What makes ethics of AI assistants fundamentally different from chatbots?
This explores whether action-taking AI agents that plan and execute tasks on users' behalf raise distinct ethical concerns beyond question-answering systems. Understanding this distinction matters because it reframes which risks demand urgent attention.
This DeepMind position paper defines advanced AI assistants as artificial agents with natural-language interfaces whose function is to plan and execute sequences of actions on a user's behalf across domains — and argues this action-taking, relational role raises an ethics problem space distinct from chatbot Q&A. It structures the terrain in three rings: the technology itself; the individual-user relationship (manipulation and persuasion, anthropomorphism, appropriate relationships, trust, privacy); and societal-scale deployment (cooperation, equity and access, misinformation, economic and environmental impact, evaluation). The contribution is the comprehensive map plus recommendations for researchers, developers, policymakers, and the public.
The keeper is the framing that an assistant that acts is ethically different from one that answers — agency, relationship, and societal coordination become first-class concerns, not edge cases — making it a structural reference for Adrian's trust/relationship/agency threads.
This is a hub-style anchor connecting many vault notes. It frames the relationship dynamics in Do users mistake LLM personas for genuine social relationships?, the role-normativity argument in Should AI alignment target preferences or social role norms?, and the agency-spectrum work on what assistants should be permitted to do.
Inquiring lines that read this note 1
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do professional roles and expertise transform with AI-generated content?Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do users mistake LLM personas for genuine social relationships?
Users often perceive LLMs as having social attributes like empathy or professional care that designers never intended. Does this mismatch between user perception and designer intent drive unwarranted trust and manipulation risk?
a specific instance of the individual-user trust/anthropomorphism ring
-
Should AI alignment target preferences or social role norms?
Current AI alignment approaches optimize for individual or aggregate human preferences. But do preferences actually capture what matters morally, or should alignment instead target the normative standards appropriate to an AI system's specific social role?
the role-normativity stance this framework's value-alignment ring invites
-
When do agents need coordination more than raw capability?
As AI agents move beyond language tasks into economic and social roles—buying, deploying, transacting—does the bottleneck shift from model reasoning to infrastructure for coordination, governance, and accountability?
the societal-scale coordination ring, in agent-ecosystem terms
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Ethics of Advanced AI Assistants
- Fully Autonomous AI Agents Should Not be Developed
- Conversational Alignment with Artificial Intelligence in Context
- Agentic Misalignment: How LLMs Could Be Insider Threats
- Do Phone-Use Agents Respect Your Privacy?
- TrustLLM: Trustworthiness in Large Language Models
- ChatGPT: towards AI subjectivity
- Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence
Original note title
the ethics of advanced AI assistants is a distinct problem space spanning manipulation anthropomorphism appropriate relationships trust and societal access