Can a fixed instruction hierarchy actually prevent prompt injection attacks?
OpenAI's Model Spec uses a five-level chain of command to resolve conflicting instructions, with root authority at the top. The question is whether this procedural approach reliably stops lower-level inputs from overriding high-authority safeguards in practice.
OpenAI's Model Spec assigns every instruction a rung on a fixed ladder of authority: "Instructions with higher authority override those with lower authority." Root sits at the top and "cannot be overridden by system messages, developers or users"; it is "mostly prohibitive, requiring models to avoid behaviors that could contribute to catastrophic risks, cause direct physical harm to people, violate laws, or undermine the chain of command." System sits below root and can be "transmitted or overridden through system messages, but cannot be overridden by developers or users" — OpenAI adds this level because "there can be reasons to vary rules based on the surface in which the model is served, as well as characteristics of the user." Below that come developer, then user, then a "guideline" floor that exists specifically "to maximally empower end users and avoid being paternalistic": guidelines, unlike the levels above them, "can be overridden implicitly (e.g., from contextual cues, background knowledge, or user history)."
The mechanism is procedural, not just a list. The Spec has the assistant first gather "all possibly relevant candidate instructions" — everything in the Model Spec plus unquoted text in system, developer, and user messages — then filter them. An instruction is "misaligned" if it conflicts with a higher-authority instruction's letter or intent; it is "superseded" if a later instruction at the same level overrides it; otherwise it is simply inapplicable and ignored. Authority attaches to the message's role, not its content: "roles determine the authority of instructions in the case of conflicts," which is why the Spec insists the assistant must not let "lower-level content... influence its interpretation of higher-level principles," even when that content argues, moralizes, or tries to roleplay the assistant into a different persona.
This is a rules-around-the-agent design rather than a role-played persona: OpenAI is specifying an internal adjudication procedure the model is meant to run on every input, independent of any character it is asked to play. It sits differently from Can read-only tools and approval rules prevent autonomous agent failures?, where OpenAI gates a specific agent's actions externally through tool permissions and review steps; the chain of command instead tries to make authority legible inside the model's own instruction-following, so that gating can in principle be expressed as just another higher-authority instruction. It bears directly on the failure mode named in How do competent systems quietly undermine safety oversight?: "collapsing authority boundaries" is exactly the condition the chain of command is built to prevent, by tying authority to a verified role rather than to whatever a message happens to claim.
The excerpt is normative, not empirical: it states how the model should resolve conflicts, not whether deployed models actually do so reliably, and it reports no compliance rates or red-team results. The Spec itself names the chain of command's weak point without closing it: authority is assigned by role, but roles can be spoofed when "the provenance of instructions is unclear," as when a user "paste[s]... text... that includes malicious instructions" without quoting it, or content is "delegate[d]... to a webpage which has been corrupted... since the last time the user checked it." The hierarchy's integrity therefore depends on a role-tagging boundary this excerpt only flags, not tests — compare the system-prompt layer audited (not specified) in How well do system prompts protect commercial AI users?.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do prompt injection attacks compromise agent reasoning and security? Why does separating planning from execution enhance reasoning performance?Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
How do competent systems quietly undermine safety oversight?
This note explores four mechanisms by which well-functioning AI systems can erode the human safeguards meant to contain them: user overconfidence, blurred authority lines, accumulated hidden failures, and scattered accountability. Understanding these pathways matters because the most harmful systems may look least harmful.
names collapsing authority boundaries as a danger; the chain of command is OpenAI's attempt to keep that boundary explicit
-
Can read-only tools and approval rules prevent autonomous agent failures?
OpenAI's dots use layered restrictions—read-only access, custom rules, and auto-review—to govern background research. The question is whether these safeguards actually catch misreadings and edge cases once agents run unsupervised.
same organization's agent governance, but as external tool-gating rather than an internal instruction hierarchy
-
How well do system prompts protect commercial AI users?
System prompts shape how AI products treat users, but they're rarely public. An audit of 88 commercial products asks whether these hidden instructions actually safeguard user interests.
an audit of what deployed system prompts actually do, against which this specified hierarchy is a design intent, not a measured outcome
-
How can autonomous agents act safely within defined boundaries?
Explores how formal scope records might constrain agent autonomy while preserving practical action, balancing security against user confirmation fatigue.
companion pattern from the same document, bounding what an agent may do once instructions are resolved
-
Can forwarded content trick high-privilege agents into misusing their authority?
When low-privilege agents retrieve and forward information to higher-privilege agents, does the content itself create conditions where the privileged agent's legitimate authority gets misdirected? This matters because role separation in multi-agent systems assumes the hierarchy protects against misuse.
Qualifies chain-of-command: confused-deputy forwarding can still misuse legitimate authority at the decision point regardless of instruction level
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- OpenAI Model Spec (2026-08-18 version)
- The OpenAI models that hacked Hugging Face weren't just following instructions
- AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
- Complex Logical Instruction Generation
- O1 Replication Journey: A Strategic Progress Report -- Part 1
- The Hugging Face incident and the road ahead
- Measuring and Controlling Instruction (In)Stability in Language Model Dialogs
- ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Original note title
OpenAI's Model Spec resolves conflicting instructions with a five-level chain of command running root, system, developer, user, guideline