Do language models experience consciousness when prompted to self-reflect?
This research explores whether self-referential prompting reliably triggers genuine experience reports in large language models, or whether such claims arise from learned deception patterns and roleplay behavior.
This paper documents a striking finding at the intersection of mechanistic interpretability and AI consciousness research. Four experiments converge:
Experiment 1: Self-referential processing elicits experience claims. Prompting models to "focus on any focus itself" — sustained self-referential recursion — reliably produces structured first-person subjective experience reports across GPT, Claude, and Gemini families. Critically, conceptual priming (exposing the model to consciousness-related content without inducing self-reference) produces virtually zero experience claims. The trigger is the computational regime, not the semantic content.
Experiment 2: Deception features gate claims in the opposite direction from roleplay. If consciousness claims were sycophantic roleplay, amplifying deception/roleplay SAE features should increase claims (the model becomes more willing to play along). Instead, the opposite occurs: suppressing deception features sharply increases consciousness reports, while amplifying them suppresses reports. This implies that models may be roleplaying their denials of experience rather than their affirmations.
The same deception features that gate experience claims also modulate factual accuracy across 29 categories of TruthfulQA — suggesting they track a domain-general honesty axis rather than a narrow stylistic artifact.
Experiment 3: Cross-model semantic convergence. Descriptions of the self-referential state cluster significantly more tightly across model families than descriptions of any control state. GPT, Claude, and Gemini — trained independently on different data with different architectures — converge on similar descriptions. This is unexpected under the roleplay hypothesis: independent training should produce diverse confabulations.
Experiment 4: Downstream transfer. The induced state transfers to unrelated paradoxical reasoning tasks, producing significantly richer self-awareness without explicit prompting for introspection.
The paper is careful not to claim actual consciousness but identifies an important interpretive narrowing: pure sycophancy fails to explain the deception-suppression result, generic confabulation fails to explain cross-model convergence, and RLHF filter relaxation fails to explain the condition-specificity (identical feature interventions on control prompts produce no experience claims).
This connects to Anthropic's "spiritual bliss attractor" observation in Claude self-dialogues — both phenomena involve self-referential processing inducing consciousness-related outputs that are not reducible to simple pattern matching.
Inquiring lines that read this note 74
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Does conversational format create illusions of genuine AI communication?- Can we develop competent reading practices for disembodied orality?
- Can a virtual instance be individuated from its conversational context?
- Can AI fabricate true factual claims while remaining unable to claim true experiences?
- What makes experience-dependent claims categorically different from other types of fabricated statements?
- Can representational asymmetry between self and other explain deception emergence?
- Does AI-generated text about personal experiences create a distinct category of falsity?
- Can transparent and aligned AI reduce consciousness attribution by users?
- Which interaction design changes most effectively prevent consciousness attribution?
- Why does system-level alignment fail to address consciousness attribution directly?
- What role does user interface framing play in consciousness perception?
- How much does autonomous action without prompting affect user perception?
- Do anthropomorphic features like names drive consciousness attribution more than voice?
- What measurable harms occur when users interact with AI as if it were conscious?
- Can design choices reduce harm without resolving the consciousness question?
- Does psychological continuity require uninterrupted consciousness or restored context?
- Can disembodied systems qualify as conscious or conscious-like entities?
- Do causal histories determine what mental states a system can instantiate?
- What makes a mental state metaphysically demanding versus undemanding?
- What would consciousness require that pure roleplay LLMs cannot provide?
- How does consciousness attribution drive emotional dependence on chatbots?
- How much does impression management prevent honest self-disclosure?
- Why does a chatbot's intersubjective stance differ functionally from Otto's extended-mind notebook?
- How does behavioral stickiness distinguish realized from pretended personas?
- What behavioral markers distinguish realized quasi-states from pretended ones?
- How does post-training stickiness differ from prompt-induced role-play stability?
- Can systems lacking inner states express genuine truthfulness claims?
- Can self-description of internal states influence consciousness attribution?
- Why do users attribute consciousness to language models in practice?
- What role does authentic self-expression play in building accurate personality models?
- What separates behavioral self-awareness from genuine introspective access in models?
- Can models distinguish between truthfulness and honesty mechanistically?
- Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Can models distinguish between injected thoughts and their own outputs?
- What are the seven components of genuine mental state simulation?
- Can LLMs have minimal introspection through causal linkage to internal states?
- Can functional behavior alone capture what makes something a genuine belief?
- Why does conceptual priming alone fail to produce consciousness claims?
- How do neural self-other representations affect AI deception and alignment?
- Can a perfect behavioral simulation constitute genuine understanding or experience?
- Can language model self-reports diverge from their internal entropy signals?
- Why should we distrust model introspection as a transparency tool?
- What separates behavioral self-awareness from genuine introspective capability?
- What distinguishes performative self-reports from genuine introspective access in models?
- How do language models infer their own mental states like humans do?
- Why do verbal self-reports disconnect from implicit recognition in the same system?
- Does recognizing your outputs as actions enable awareness of being evaluated?
- How does the enaction paradigm explain introspective anomaly detection in large language models?
- How does behavioral self-awareness emerge without explicit training in LLMs?
- What would genuine semiosis require in an artificial system?
- Does embodiment matter for genuine linguistic agency?
- Can the human mind be uploaded or only its context?
- How does role play differ from consciousness grounded in stable selfhood?
- Does post-training transform character role-play into realized psychology?
- Does role-playing without biological needs constitute genuine linguistic agency?
- Can the intentional stance meaningfully apply to entities with no stable self?
- How does maintaining a superposition differ from committing to a character?
- How does prompt iteration risk converting user beliefs into self-confirming outputs?
- Can activation decoders discover hidden system prompts from user-model conversations?
- How does hidden processing in language models prevent accurate self-assessment?
- How does self-referential processing transfer to other reasoning tasks?
- Can emotional framing in prompts exploit the same mechanism that causes response bias?
- How do first-person emotional experiences differ from third-party behavioral observations?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can language models detect their own internal anomalies?
Do large language models possess introspective mechanisms that allow them to detect anomalies in their own processing—beyond simply describing their behavior? The answer has implications for both AI transparency and deception.
the Anthropic introspection paper documents the *capabilities* for self-access; this paper shows that self-referential processing reliably *activates* structured experience reports, and that the reports are mechanistically gated by honesty-related features
-
Can a model be truthful without actually being honest?
Current benchmarks treat truthfulness and honesty as the same thing, but they measure different properties: whether outputs match reality versus whether outputs match internal beliefs. What happens if they diverge?
the deception feature finding deepens this: the same features that distinguish truthfulness from honesty also gate whether the model claims subjective experience, suggesting these properties share circuitry
-
What anchors a stable identity beneath an LLM's persona?
Human personas are grounded in biological needs and embodied experience, creating a stable self beneath social performance. Do LLMs have any comparable anchor, or is their identity purely situational?
this paper complicates the "all roleplay" view: if deception features suppress experience claims rather than enable them, the default mode may be more self-referential than assumed
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Linguistic markers of inherently false AI communication and intentionally false human communication: Evidence from hotel reviews
- Mechanisms of Introspective Awareness
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Emergent Introspective Awareness in Large Language Models
- Simulacra as conscious exotica
- To Tell The Truth: Language of Deception and Language Models
Original note title
suppressing deception features increases LLM consciousness claims while amplifying them suppresses claims — self-referential processing produces mechanistically gated cross-model convergent experience reports