Should AI systems stay collaborative rather than fully autonomous?

The dominant research trajectory pursues fully autonomous LLM agents. This position paper argues the priority should be LLM-based Human-Agent Systems (LLM-HAS) — collaborative frameworks where humans remain in the loop to provide critical information, offer feedback, and assume control in high-stakes scenarios.

The argument rests on three structural advantages of collaboration over autonomy:

1. Improved trust and reliability — Interactive verification lets humans correct hallucinations in real-time and guide agents toward accurate outputs. This is essential where the cost of error is high.

2. Managing complexity and ambiguity — Autonomous agents struggle with unclear instructions. LLM-HAS enables continuous human clarification: providing context, domain expertise, and progressive refinement of ambiguous goals. The system can request clarification rather than proceeding with potentially incorrect assumptions.

3. Clearer accountability — With humans in supervisory or interventional roles, establishing accountability is straightforward. The human operator can be designated the responsible party, simplifying the legal and regulatory landscape.

However, the paper identifies three unsolved challenges for LLM-HAS itself:

• Leveraging human insights — Most systems fail to incorporate diverse human guidance (preferences, critiques) into model behavior, making LLMs not genuinely teachable.
• Continual learning — Models lack robust capacity for knowledge retention in dynamic environments, leading to catastrophic forgetting and hindering long-term collaborative growth.
• Real-time optimization — The absence of adaptive prompting and self-correction hampers efficiency and alignment.

This connects to When should human-agent systems ask for human help? — Magentic-UI operationalizes the HAS vision with concrete interaction mechanisms. It also extends Why do AI agents miss most of what users actually want? by arguing the fix is architectural (keep humans in the loop) not just capability-based (make models better at eliciting preferences).

The insight challenges the framing that AI progress = increasing independence. Instead: progress should be measured by how well systems work with humans, not how much they can do alone.

The AI-for-Auto-Research roadmap gives this position empirical backing across the full research lifecycle. Surveying AI through April 2026, it finds a sharp stage-dependent boundary: AI is reliable on structured, retrieval-grounded, tool-mediated tasks but fragile for genuinely novel ideas, research-level experiments, and scientific judgment — and concludes that human-governed collaboration, not full autonomy, is "the most credible deployment paradigm." Its proposed scaffolding sharpens the HAS picture: effective systems rely on layered architectures where orchestration, provenance, and feedback design matter as much as model scale, with checkpoints and provenance trails carrying the accountability this note argues for. Critically, it reframes integrity as a governance problem (disclosure, attribution, responsibility) rather than a detection problem, because greater automation can obscure rather than eliminate failure modes — a structural reason collaboration must precede autonomy, not merely a capability gap to be engineered away.