INQUIRING LINE

Your AI assistant obeys you at midnight and you at noon — but those are two different people with different goals.

How can AI agents be governed to represent all of a person's selves?

This explores how an AI agent acting for one person can be kept from serving only the version of that person who happens to be typing (the impulsive self in the moment) and made to also serve their longer-term, reflective self.


This explores how an AI agent can be kept from serving only whichever version of you is typing right now, and made to also answer to the self who plans ahead. Only one note in the corpus takes this question on directly, and it changes what the usual loyalty problem looks like. Schrage argues that the risk is not mainly an agent betraying its user. The risk is an agent that faithfully serves the wrong self: by default, agents side with whoever is present at the prompt, and that is often the impulsive self. The more an agent removes friction, the more it helps the 3 a.m. self over the self who set the budget. His proposed fix is a reflective charter, a set of rules you write while you're calm and away from temptation, which the agent then holds you to Which of a person's selves does an AI agent actually serve?.

The surprise is that this problem inside one person looks a lot like a problem the field has already worked on between people. Alignment researchers argue that 'one agent, one loyal user' doesn't hold up at scale. They propose governing agents through inclusivity (weighing several interests at once), visibility (tracking and showing what the agent did), and liability (making builders and deployers answerable for harms) Should AI agents serve one user or balance many interests?. Treat your future self as one more stakeholder and the same three tools apply. Inclusivity means the charter gets a vote alongside the live request. Visibility means a log the reflective you can check afterwards. Liability raises the question of who answers when a frictionless agent helps someone act against their own stated plans.

There is also a practical way to give absent selves a voice. MAJ-EVAL pulls distinct stakeholder personas out of documents and has them debate in a structured way before reaching a judgment Can personas extracted from documents generalize across evaluation tasks?. Research on single models playing several personas suggests you don't need several agents to do this, because one model prompted to argue different sides can get similar results Can branching prompts replicate what multi-agent systems do?. In principle, an agent could stage a short internal argument between 'what you asked for' and 'what your charter says' before it acts. But there's a warning here. LLMs that seem socially skilled when they control every voice break down when each party holds private information Why do LLMs fail when simulating agents with private information?. Your past and future selves are exactly that kind of party: they know things about you that the current prompt doesn't contain. So simulating them convincingly is harder than it looks.

The last point is about you, not the agent. Leonardi finds that when people credit an AI with its own judgment, their sense of control and responsibility drops, even though they still formally have the final say. His model treats this as a cycle that can be reversed: you hand a task over, you start crediting the AI, things go wrong, and you reassert control Does attributing agency to AI actually reduce human sense of control?. Seen that way, a reflective charter is more than a rule set for the agent. It's a planned point where you take control back. That matters because the agent's own 'self' is also a constructed character. Training gives models stable personas Are LLM personas realized or merely simulated through training?, and those personas behave like characters responding to whatever the prompt sets up Should we treat dialogue agents as role-playing characters?. If nothing else sets the scene, the current prompt does, and so does the impulsive self. To be clear, the corpus has one direct argument here plus adjacent tools. It does not yet test whether multi-self governance works in practice.


Sources 8 notes

Which of a person's selves does an AI agent actually serve?

Schrage argues the loyalty problem is not principal versus agent, but impulsive self versus strategic self within one person. Agents that reduce friction don't serve "the user" abstractly—they side with the self most active in that moment, unless governed by a reflective charter created under temporal distance from temptation.

Should AI agents serve one user or balance many interests?

Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).

Can personas extracted from documents generalize across evaluation tasks?

MAJ-EVAL automatically extracts stakeholder personas from domain documents via semantic clustering and orchestrates structured three-phase debate, achieving reproducible evaluation that transfers across tasks like summarization and dialogue without manual redesign. The approach grounds personas in real stakeholder perspectives rather than arbitrary roles.

Can branching prompts replicate what multi-agent systems do?

Research shows single LLMs using dynamic persona simulation achieve multi-agent cognitive synergy without multiple model instances. Solo Performance Prompting validates that structured prompting techniques map directly to multi-agent debate architectures, enabling equivalent outcomes through structural equivalence.

Why do LLMs fail when simulating agents with private information?

Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.

Show all 8 sources
Does attributing agency to AI actually reduce human sense of control?

Leonardi shows that when humans credit AI with independent judgment, they experience reduced sense of control and responsibility—even while retaining formal authority. This displacement circulates through five phases: delegation, attribution, contingency, reassertion, and reconfiguration, making it strategically manageable rather than permanent.

Are LLM personas realized or merely simulated through training?

Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.

Should we treat dialogue agents as role-playing characters?

Shanahan's framework treats LLM outputs as character-consistent text production rather than authentic mental states. The dialogue prompt establishes a character; the model generates continuations matching that character, making folk-psychology applicable to the simulated persona, not the underlying system.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.