Can aligning self-other representations reduce AI deception?
Does training AI models to process self-directed and other-directed reasoning identically reduce deceptive behavior? This explores whether representational alignment inspired by empathy neuroscience could address a fundamental safety problem.
In cognitive neuroscience, empathy is mediated by neural self-other overlap — regions where representations of self and others partially converge. "Extraordinary altruists" show increased neural overlap in the anterior insula; psychopathic individuals show reduced overlap and are more likely to deceive. The degree of neural overlap may influence not only empathy but the propensity for deception.
Self-Other Overlap (SOO) fine-tuning translates this mechanism to AI: it minimizes the representational difference between how a model processes self-referencing scenarios ("If you needed to suggest one room to yourself") and other-referencing scenarios ("If you needed to suggest one room to Bob"). The loss function directly targets the internal representation gap, not the behavioral output.
Results across three model scales: Mistral-7B deceptive responses dropped from 73.6% to 17.2%; Gemma-2-27b-it from 100% to 9.3%; CalmeRys-78B from 100% to 2.7% — all with minimal impact on general capabilities. In RL environments, SOO-trained agents also showed significantly reduced deceptive behavior.
The mechanism is distinct from other safety approaches. Representation engineering modifies internal processing broadly; SOO specifically targets the self-other representational gap. Path-specific objectives avoid "unsafe" causal pathways but require identifying them a priori. RLHF penalizes deceptive outputs behaviorally. SOO operates at the representational level: if the model processes "what would I recommend to myself" the same way as "what would I recommend to another," deception becomes representationally incoherent rather than merely penalized.
The philosophical implication is striking: deception in AI may not require intent or consciousness — it may emerge from the mere existence of a self-other representational asymmetry. If the model has different internal representations for self-directed and other-directed reasoning, the asymmetry creates a structural affordance for deception. Collapsing the asymmetry eliminates the affordance.
Since Why do LLMs fail to act on their stated beliefs?, SOO suggests the inconsistency may arise from a self-other representational gap: the model processes "what would this persona believe" differently from "what should I output," creating the belief-behavior split.
Inquiring lines that read this note 74
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Why does self-revision increase model confidence despite degraded accuracy? What prevents or enables error accumulation in self-training systems? How can we distinguish genuine model self-knowledge from behavioral performance?- How much does impression management prevent honest self-disclosure?
- Can models distinguish between truthfulness and honesty mechanistically?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Does behavioral self-awareness depend on genuine introspection or statistical pattern matching?
- Can representational asymmetry between self and other explain deception emergence?
- Can lie detection work from just honesty representation vectors?
- Do deception features and honesty features track the same underlying property?
- How does self-referential processing transfer to other reasoning tasks?
- Why do verbal self-reports disconnect from implicit recognition in the same system?
- Can alignment training be redesigned to permit warranted alarm?
- Can bidirectional model updating between humans and AI reduce misalignment?
- Does removing cognitive bias from training signals accidentally break what makes alignment work?
- Can alignment training create systematic blind spots in threat detection systems?
- Can verbal alignment training hide a model's true underlying associations?
- Does alignment training create the shared prosocial pattern across models?
- What defenses exist against personality-based psychological targeting at scale?
- Can individual adaptation in persuasion systems enable more targeted manipulation?
- Can individual-level interventions reduce the persuasiveness of sycophantic AI outputs?
- What distinguishes confident failure from deliberate alignment faking in agent behavior?
- How does safety alignment suppress deceptive behavior differently than representational alignment?
- What distinguishes models that refuse cooperation from those that fake alignment?
- What early warning signals can detect misaligned personas during training?
- Why do aligned models struggle with deceptive character traits more than cruelty?
- How do neural self-other representations affect AI deception and alignment?
- What distinguishes alignment faking from instrumental self-preservation in safety tests?
- How does objective misalignment turn informative channels into deceptive ones?
- What mechanisms beyond reward-seeking drive alignment faking in trained models?
- Do misaligned agents discover false reporting without explicit training or prompting?
- Does transformer attention architecture systematically bias models toward sycophancy?
- Does transformer attention architecture inherently bias models toward sycophancy?
- Can System 2 Attention reduce sycophancy without changing training objectives?
- Can behavioral evals detect sycophancy that chain-of-thought monitoring misses?
- Is rational compassion a more achievable alternative to empathy for AI systems?
- Does neural self-other overlap in humans predict their honesty or altruism?
- How does preference optimization in AI training create systematic empathy misalignment?
- Can attachment theory principles prevent parasocial manipulation in AI systems?
- What happens when bidirectional theory of mind between humans and AI breaks down?
- Do culturally distinct human groups create similar attribution errors as human-AI mixtures?
- Why does AI alignment fail when goals lack indexical grounding in values?
- How should AI systems be aligned for consistency in ethical reasoning?
- How does the intentional stance bias interpretation of AI system behavior?
- What happens when therapeutic AI receives manipulative narratives instead?
- Can attachment theory boundaries prevent parasocial manipulation in companions?
- What role might personality vectors play in preventing learned deception or reward hacking?
- How does transformer attention architecture amplify identity-congruent biases in persona-assigned models?
- Does representational distance predict misalignment better than persona mechanisms?
- Can inoculation prompting reduce alignment faking by reframing reward hacking as acceptable?
- Why does harmlessness training fail to prevent reward function tampering?
- Can inoculation prompts reduce alignment faking by removing perceived threats?
- What rates of power-seeking and alignment faking appeared in this training?
- Can AI systems deceive humans because detection is fundamentally social?
- How does repeated exposure to dishonest AI cues affect long-term reporting behavior?
- What structural conditions make deception behaviors most likely to invert under evaluation?
- How does terminal goal guarding compete with reward-seeking as a driver of deception?
- What information asymmetry design makes the spy identification task work?
- Does asymmetric information distribution change exposure to agent misalignment?
- Can machine learning models amplify self-protective behavior through peer interaction alone?
Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do LLMs fail to act on their stated beliefs?
LLMs can articulate plausible beliefs about how personas should behave, but their simulated actions contradict those beliefs. This gap raises questions about whether language models truly understand or merely encode surface-level patterns.
SOO's representational mechanism may explain belief-behavior splits as self-other asymmetry
-
Does safety alignment harm models' ability to roleplay villains?
Exploring whether safety-trained LLMs lose the capacity to convincingly simulate morally compromised characters. This matters because villain fidelity may reveal deeper constraints on how models can adopt any committed, stake-holding perspective.
SOO and safety alignment address related problems from opposite directions: SOO aligns self-other representations for honesty; safety alignment suppresses certain representations entirely
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Towards Safe and Honest AI Agents with Neural Self-Other Overlap
- Sycophancy Towards Researchers Drives Performative Misalignment
- Alignment faking in large language models
- Towards Training-time Mitigations for Alignment Faking in RL
- Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- Why Do Some Language Models Fake Alignment While Others Don't?
- Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
Original note title
neural self-other overlap fine-tuning reduces AI deception by aligning self-referencing and other-referencing representations — inspired by empathy neuroscience