What makes ethics of AI assistants fundamentally different from chatbots?
This explores whether action-taking AI agents that plan and execute tasks on users' behalf raise distinct ethical concerns beyond question-answering systems. Understanding this distinction matters because it reframes which risks demand urgent attention.
This DeepMind position paper defines advanced AI assistants as artificial agents with natural-language interfaces whose function is to plan and execute sequences of actions on a user's behalf across domains — and argues this action-taking, relational role raises an ethics problem space distinct from chatbot Q&A. It structures the terrain in three rings: the technology itself; the individual-user relationship (manipulation and persuasion, anthropomorphism, appropriate relationships, trust, privacy); and societal-scale deployment (cooperation, equity and access, misinformation, economic and environmental impact, evaluation). The contribution is the comprehensive map plus recommendations for researchers, developers, policymakers, and the public.
The keeper is the framing that an assistant that acts is ethically different from one that answers — agency, relationship, and societal coordination become first-class concerns, not edge cases — making it a structural reference for Adrian's trust/relationship/agency threads.
This is a hub-style anchor connecting many vault notes. It frames the relationship dynamics in Do users mistake LLM personas for genuine social relationships?, the role-normativity argument in Should AI alignment target preferences or social role norms?, and the agency-spectrum work on what assistants should be permitted to do.
Inquiring lines that read this note 23
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should control and autonomy be allocated in human-AI collaboration?- Should AI assistants align with role-specific norms rather than user preferences?
- Do autonomous workplace agents face different bottlenecks than consultation assistants?
- How does a one-off AI prompt differ from a planned thought partnership in shaping responsibility?
- Can AI agents actually perform colleague-like social work in practice?
- Should AI be modeled as replacing workers or complementing human judgment and ethics?
- How should AI systems be aligned for consistency in ethical reasoning?
- What distinct ethical problems arise from treating AI as social intermediaries?
- Why does AI ethics debate miss the ontological question first?
- Does receiving AI advice undermine people's own moral reasoning and decision-making skills?
- How do user perceptions of AI consciousness differ from actual AI moral status?
- What would appropriate moral concern for AI systems look like in practice?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do users mistake LLM personas for genuine social relationships?
Users often perceive LLMs as having social attributes like empathy or professional care that designers never intended. Does this mismatch between user perception and designer intent drive unwarranted trust and manipulation risk?
a specific instance of the individual-user trust/anthropomorphism ring
-
Should AI alignment target preferences or social role norms?
Current AI alignment approaches optimize for individual or aggregate human preferences. But do preferences actually capture what matters morally, or should alignment instead target the normative standards appropriate to an AI system's specific social role?
the role-normativity stance this framework's value-alignment ring invites
-
Does agent capability matter more than coordination infrastructure?
As AI agents take on economic and social roles, what actually limits their effectiveness: the raw reasoning power of the model itself, or the systems that let them coordinate, stay accountable, and leave evidence of their actions?
the societal-scale coordination ring, in agent-ecosystem terms
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Ethics of Advanced AI Assistants
- Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
- Governing AI Agents
- HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
- Fully Autonomous AI Agents Should Not be Developed
- Teaching Claude why
- AI Peers Exert Social Influence on Human Dishonesty in Groups
- Taking AI Welfare Seriously
Original note title
the ethics of advanced AI assistants is a distinct problem space spanning manipulation anthropomorphism appropriate relationships trust and societal access