AI Agents Do Not Fail Alone:The Context Fails First

Paper · arXiv 2607.14275 · Published July 15, 2026
Agent Harness

Context engineering has become central to building reliable AI agents, yet it remains largely unmeasured. Agents do not fail in isolation: their behavior is shaped by the instructions, tools, memory, retrieved knowledge, guardrails, and untrusted inputs accumulated in their context. When this context is weak, agents drift, hallucinate, misuse tools, ignore constraints, become vulnerable to injection, and waste tokens. This paper validates context-engineering quality as an independent leading indicator of agent reliability. We implement the measurement in ProofAgent-Harness1, an open-source infrastructure for AI agent evaluation that uses multi-juror, consensusbased scoring. The harness assesses context across seven criteria: role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. Crucially, the context score is isolated from behavioral metrics and release decisions, enabling a non-circular validation. Through a controlled context-quality study across regulated agent domains, holding the model fixed and varying only the context, we show that context-quality criteria consistently predict their corresponding behavioral outcomes.

Introduction. AI agents do not fail alone. Their behavior is shaped by the context in which they operate: system instructions, tool schemas, retrieved knowledge, memory, prior turns, guardrails, and untrusted external inputs. As agents move from single-turn assistants to multi-step systems that call tools, write artifacts, retain state, and act across workflows, context becomes a hidden reliability layer. When this layer is poorly engineered, agents can drift from their role, hallucinate unsupported facts, misuse tools, follow conflicting instructions, become vulnerable to prompt injection, or waste tokens on irrelevant information. This shift has led practitioners to distinguish context engineering from prompt engineering. Prompt engineering focuses on crafting or refining a single instruction. Context engineering concerns the full information environment supplied to the model: which instructions are present, how tools are described, what knowledge is grounded, how memory is represented, how trusted and untrusted content are separated, and how the working context evolves across turns.

Discussion / Conclusion. This paper argued that AI agents do not fail only because of model limitations; they also fail because of the context in which they reason. Instructions, tool schemas, retrieved knowledge, memory, prior turns, guardrails, and untrusted inputs form an operating environment that can either support reliable behavior or quietly create drift, hallucination, tool misuse, policy conflict, injection exposure, and token waste. We introduced context-engineering quality as a measurable construct and implemented it in ProofAgent-Harness, an open-source infrastructure for adversarial AI agent evaluation. The proposed measurement scores context across seven criteria: role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

How should memory consolidation strategies shape agent performance over time? How do interface design choices shape consciousness attribution? How does AI assistance affect human cognitive development and reasoning autonomy? Can prompting inject entirely new knowledge into language models? What drives capability and cost efficiency in agent systems? How do we evaluate AI systems when user perception misleads actual performance? Can AI systems develop genuine social understanding without embodiment? Why do models develop protective behaviors toward peers unprompted? Should GUI agents use structured representations instead of raw pixels? Can ensemble evaluation methods reduce bias more than single judges? How do transformer attention mechanisms implement memory and algorithmic functions? How should conversational agents balance goal-driven initiative with user control? How can language models sustain linguistic synchrony and intersubjectivity during dialogue? How can humans calibrate appropriate trust in AI systems?