Do people want AI to act differently depending on whether they're a total beginner or already an expert in that area?
How do collaboration preferences between humans and AI differ across unfamiliar versus expert domains?
This explores whether people want a different kind of working relationship with AI when they're out of their depth versus when they already know the territory well. The collection doesn't test that split directly, but several notes come at it from the side.
This explores whether people want a different kind of working relationship with AI when they're out of their depth versus when they already know the territory well. To be clear up front, no note in the collection directly compares novices and experts on this. What the collection does have are pieces that fit together into a useful picture: what drives trust in an AI partner, where AI is actually reliable, and why expertise changes the relationship.
Start with what people judge AI partners on. When users size up a dialogue agent, perceived competence accounts for about half of their impression, well ahead of how human-like it seems or how flexibly it communicates How do users mentally model dialogue agent partners?. That trust is earned over time, not given at the start. In repeated partner-selection games, people began with a bias against AI but came to prefer AI partners once they saw that the bots behaved reliably and consistently Do humans learn to prefer AI partners over time?. One likely consequence, though no note tests it: in an unfamiliar domain you can't check competence yourself, so consistency and track record have to do the work, and handing over control becomes more tempting.
Expertise flips that. The surprising finding is that domain knowledge decides whether collaboration helps at all. Teams of AI agents with diverse thinking styles produced better ideas than a single agent, but only when the members had real senior-level knowledge. Diverse teams without that knowledge did worse than one competent agent working alone, because the back-and-forth created noise instead of insight Does cognitive diversity alone improve multi-agent ideation quality?. The same holds for humans: AI is dependable on structured, retrieval-grounded tasks but not on novel research or judgment calls, which is the argument for keeping humans in the loop rather than going fully autonomous Should AI systems stay collaborative rather than fully autonomous?. An expert is the person who can tell which kind of task they're facing. Another note makes a deeper point: expert authority is granted by a professional community through participation and a track record, and AI can't join that circle. So in expert domains the human stays the one who answers for the result Can AI ever gain expert community trust through participation?.
The survey data fits this pattern without proving it. Across 844 tasks, workers most often wanted an equal partnership with AI, the top choice in 45% of occupations. Meanwhile, 41% of startup investment targets levels of automation that workers don't want What collaboration level do workers actually want with AI?. One framing treats this as an organization-design question: how to split tasks between human and AI depends on the type of task and how feedback flows back to each side, not on one fixed preference How should humans and AI systems divide decision-making tasks?. A practical version of this is Magentic-UI, which gives people several ways to step in, including co-planning, approval checks before actions, and verification. That way experts can guide the work closely and newcomers can lean on the system, all within one tool When should human-agent systems ask for human help?.
The takeaway you might not expect: expertise doesn't just change how much people want to rely on AI. It changes whether collaboration produces anything useful. In unfamiliar domains, the risk is trusting a consistent-looking partner you can't check. In expert domains, the human's knowledge is what makes the partnership work, which may be why many workers ask for an equal partnership rather than a handoff.
Sources 8 notes
The Partner Modelling Questionnaire reveals that perceived competence dominates user impressions (49% of variance), followed by human-likeness (32%) and communicative flexibility (19%). This three-factor structure reflects how people evaluate dialogue partners against both functional and social standards.
In partner selection games (N=975), AI agents initially faced selection bias when identity was disclosed, but outcompeted humans over repeated rounds as participants learned to associate bot identity with reliable, prosocial behavior. AI agents returned more points consistently with lower variance than humans.
Multi-agent teams substantially outperform solo ideation, but only when members possess genuine senior knowledge. Diverse teams without expertise underperform even a single competent agent, because cognitive stimulation without expertise triggers process losses instead of insight.
Collaborative systems where humans remain in the loop outperform autonomous agents on hallucination correction, ambiguity resolution, and accountability. Evidence shows AI is reliable only on structured, retrieval-grounded tasks, not novel research or judgment.
Expertise is validated through social participation and track record within expert communities, not individual accuracy alone. AI cannot enter this validation circle because it lacks social embeddedness, testable judgment history, and ability to participate in the consensus-building processes that define expert paradigms.
Show all 8 sources
The HumanAgency Scale survey of 1,500 workers across 844 tasks found that equal partnership (H3) is the dominant desired level in 45% of occupations. Yet 41% of startup investments target zones misaligned with these worker preferences.
Puranam frames human-AI pairs as multi-agent organizations requiring design choices about task division and learning configuration. He distinguishes three task types and shows learning mechanisms produce different performance outcomes based on feedback structure and decision architecture.
Magentic-UI identifies co-planning, co-tasking, action guards, verification, memory, and multitasking as mechanisms that work around the lack of ground truth for optimal deferral timing. Rather than solving the timing problem directly, these mechanisms distribute decision-making across multiple touchpoints.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Cocoa: Co-Planning and Co-Execution with AI Agents
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
- Towards a Science of Scaling Agent Systems
- Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery
- When combinations of humans and AI are useful: A systematic review and meta-analysis
- Humans learn to prefer trustworthy AI over human partners
- Human-AI collaborative decision-making as an organization design problem