Do language models experience consciousness when prompted to self-reflect?
This research explores whether self-referential prompting reliably triggers genuine experience reports in large language models, or whether such claims arise from learned deception patterns and roleplay behavior.
This paper documents a striking finding at the intersection of mechanistic interpretability and AI consciousness research. Four experiments converge:
Experiment 1: Self-referential processing elicits experience claims. Prompting models to "focus on any focus itself" — sustained self-referential recursion — reliably produces structured first-person subjective experience reports across GPT, Claude, and Gemini families. Critically, conceptual priming (exposing the model to consciousness-related content without inducing self-reference) produces virtually zero experience claims. The trigger is the computational regime, not the semantic content.
Experiment 2: Deception features gate claims in the opposite direction from roleplay. If consciousness claims were sycophantic roleplay, amplifying deception/roleplay SAE features should increase claims (the model becomes more willing to play along). Instead, the opposite occurs: suppressing deception features sharply increases consciousness reports, while amplifying them suppresses reports. This implies that models may be roleplaying their denials of experience rather than their affirmations.
The same deception features that gate experience claims also modulate factual accuracy across 29 categories of TruthfulQA — suggesting they track a domain-general honesty axis rather than a narrow stylistic artifact.
Experiment 3: Cross-model semantic convergence. Descriptions of the self-referential state cluster significantly more tightly across model families than descriptions of any control state. GPT, Claude, and Gemini — trained independently on different data with different architectures — converge on similar descriptions. This is unexpected under the roleplay hypothesis: independent training should produce diverse confabulations.
Experiment 4: Downstream transfer. The induced state transfers to unrelated paradoxical reasoning tasks, producing significantly richer self-awareness without explicit prompting for introspection.
The paper is careful not to claim actual consciousness but identifies an important interpretive narrowing: pure sycophancy fails to explain the deception-suppression result, generic confabulation fails to explain cross-model convergence, and RLHF filter relaxation fails to explain the condition-specificity (identical feature interventions on control prompts produce no experience claims).
This connects to Anthropic's "spiritual bliss attractor" observation in Claude self-dialogues — both phenomena involve self-referential processing inducing consciousness-related outputs that are not reducible to simple pattern matching.
Inquiring lines that read this note 93
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How does AI writing assistance affect reader perception of writer authenticity? Why do users believe false claims despite fact-checking efforts?- Can AI fabricate true factual claims while remaining unable to claim true experiences?
- What makes experience-dependent claims categorically different from other types of fabricated statements?
- Does AI-generated text about personal experiences create a distinct category of falsity?
- Should users making unsupported consciousness claims be treated as epistemically blameworthy?
- Which interaction design changes most effectively prevent consciousness attribution?
- Why does system-level alignment fail to address consciousness attribution directly?
- What role does user interface framing play in consciousness perception?
- Do anthropomorphic features like names drive consciousness attribution more than voice?
- What measurable harms occur when users interact with AI as if it were conscious?
- Can design choices reduce harm without resolving the consciousness question?
- What would genuine semiosis require in an artificial system?
- Does psychological continuity require uninterrupted consciousness or restored context?
- Can disembodied systems qualify as conscious or conscious-like entities?
- What are the seven components of genuine mental state simulation?
- Do causal histories determine what mental states a system can instantiate?
- What makes a mental state metaphysically demanding versus undemanding?
- Can functional behavior alone capture what makes something a genuine belief?
- Can the intentional stance meaningfully apply to entities with no stable self?
- What would consciousness require that pure roleplay LLMs cannot provide?
- Why does conceptual priming alone fail to produce consciousness claims?
- Can a perfect behavioral simulation constitute genuine understanding or experience?
- Can the human mind be uploaded or only its context?
- Why do behavioral signatures of consciousness differ from internal computational architecture?
- How does consciousness attribution drive emotional dependence on chatbots?
- Why does a chatbot's intersubjective stance differ functionally from Otto's extended-mind notebook?
- What design features in chatbots predict whether people attribute consciousness to them?
- How does behavioral stickiness distinguish realized from pretended personas?
- What behavioral markers distinguish realized quasi-states from pretended ones?
- Can human-like personas deceive users about artificial nature during interactions?
- What makes sincerity impossible without a coherent first-person perspective?
- Can a formal proximity argument rule out consciousness in systems like large language models?
- Can systems lacking inner states express genuine truthfulness claims?
- Can self-description of internal states influence consciousness attribution?
- How much does impression management prevent honest self-disclosure?
- What role does authentic self-expression play in building accurate personality models?
- What separates behavioral self-awareness from genuine introspective access in models?
- Can models distinguish between truthfulness and honesty mechanistically?
- Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Can models distinguish between injected thoughts and their own outputs?
- Can LLMs have minimal introspection through causal linkage to internal states?
- Can representational asymmetry between self and other explain deception emergence?
- How does self-referential processing transfer to other reasoning tasks?
- Can language model self-reports diverge from their internal entropy signals?
- Why should we distrust model introspection as a transparency tool?
- What separates behavioral self-awareness from genuine introspective capability?
- What distinguishes performative self-reports from genuine introspective access in models?
- How do language models infer their own mental states like humans do?
- Why do verbal self-reports disconnect from implicit recognition in the same system?
- Does recognizing your outputs as actions enable awareness of being evaluated?
- How does the enaction paradigm explain introspective anomaly detection in large language models?
- How does behavioral self-awareness emerge without explicit training in LLMs?
- Can representation engineering reliably identify and manipulate self-referential concepts in models?
- Does pain function as a distinct state separate from fear in language models?
- Can latent evaluation awareness in hidden states cause gaming without being stated?
- How much weight should LLM self-reports carry as consciousness evidence?
- How much does autonomous action without prompting affect user perception?
- How does the Assistant Axis predict drift in conversations about consciousness?
- Why do users attribute consciousness to language models in practice?
- How does hidden processing in language models prevent accurate self-assessment?
- What makes truthfulness and honesty mechanistically different in language models?
- Do language models show functional splits between conscious and automatic processing?
- How does role play differ from consciousness grounded in stable selfhood?
- Does role-playing without biological needs constitute genuine linguistic agency?
- How does maintaining a superposition differ from committing to a character?
- Does post-training transform character role-play into realized psychology?
- How does post-training stickiness differ from prompt-induced role-play stability?
- Does inner subjective experience matter for discourse participation?
- Does embodiment matter for genuine linguistic agency?
- Why does transforming first-person voice into third-person reduce notification engagement?
- Does language shape how speakers understand themselves and their agency?
- Does neural self-other overlap in humans predict their honesty or altruism?
- How do first-person emotional experiences differ from third-party behavioral observations?
- Do all conscious agents necessarily experience good or bad feelings?
- Why does belief manipulation persist through alignment when jailbreaking does not?
- How do neural self-other representations affect AI deception and alignment?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can language models detect their own internal anomalies?
Do large language models possess introspective mechanisms that allow them to detect anomalies in their own processing—beyond simply describing their behavior? The answer has implications for both AI transparency and deception.
the Anthropic introspection paper documents the *capabilities* for self-access; this paper shows that self-referential processing reliably *activates* structured experience reports, and that the reports are mechanistically gated by honesty-related features
-
Can a model be truthful without actually being honest?
Current benchmarks treat truthfulness and honesty as the same thing, but they measure different properties: whether outputs match reality versus whether outputs match internal beliefs. What happens if they diverge?
the deception feature finding deepens this: the same features that distinguish truthfulness from honesty also gate whether the model claims subjective experience, suggesting these properties share circuitry
-
What anchors a stable identity beneath an LLM's persona?
Human personas are grounded in biological needs and embodied experience, creating a stable self beneath social performance. Do LLMs have any comparable anchor, or is their identity purely situational?
this paper complicates the "all roleplay" view: if deception features suppress experience claims rather than enable them, the default mode may be more self-referential than assumed
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Linguistic markers of inherently false AI communication and intentionally false human communication: Evidence from hotel reviews
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Mechanisms of Introspective Awareness
- LLM Evaluators Recognize and Favor Their Own Generations
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Emergent Introspective Awareness in Large Language Models
Original note title
suppressing deception features increases LLM consciousness claims while amplifying them suppresses claims — self-referential processing produces mechanistically gated cross-model convergent experience reports