Design Patterns for Securing LLM Agents against Prompt Injections

As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge. Among the most pressing threats are prompt injection attacks, which exploit the agent’s resilience on natural language inputs — an especially dangerous threat when agents are granted tool access or handle sensitive information. In this work, we propose a set of principled design patterns for building AI agents with provable resistance to prompt injection. We systematically analyze these patterns, discuss their trade-offs in terms of utility and security, and illustrate their real-world applicability through a series of case studies.

Introduction. Large Language Models (LLMs) are becoming integral components of complex software systems, where they serve as intelligent agents that can interpret natural language instructions, make plans, and execute actions through external tools and APIs. While these LLM-based agents offer new and powerful capabilities in natural language understanding and task automation, they also open up new security vulnerabilities that traditional application security frameworks are ill-equipped to address.

Among these new threats, prompt injection attacks (Perez & Ribeiro, 2022; Willison, 2022; Goodside, 2022b; Abdelnabi et al., 2023) are particularly concerning. These attacks occur when malicious data, embedded within content processed by the LLM, manipulates the model’s behavior to perform unauthorized or unintended actions. Prompt injection attacks can have severe consequences, ranging from data exfiltration and privilege escalation to remote code execution, as demonstrated by several real-world incidents (Embrace The Red, 2024b, 2023).

The research community has proposed various defensive measures against prompt injections, from heuristic approaches like detection and adversarial training (Wallace et al., 2024; Chen et al., 2024; Zverev et al., 2024; Willison, 2023a; Yi et al., 2023; Zou et al., 2024), to principled system-level isolation mechanisms (Bagdasarian et al., 2024; Balunovic et al., 2024; Abdelnabi et al., 2025b; Willison, 2023b; Wu et al., 2025; Debenedetti et al., 2025). These proposals are primarily generic, aiming to design safe general-purpose agents.

In this work, we describe a number of design patterns for LLM agents that significantly mitigate the risk of prompt injections. These design patterns constrain the actions of agents to explicitly prevent them from solving arbitrary tasks. We believe these design patterns offer a valuable trade-off between agent utility and security.

To illustrate the broad applicability of these design patterns, we apply them to ten case studies of LLM agent applications, spanning from simple OS function assistants to more general software engineering agents. In each case, we describe different design decisions—aided by our design patterns—that can provide meaningful levels of security against prompt injections without overly restricting the agent’s utility.

We hope that the design patterns and case studies described in this work will guide LLM agent designers, developers, and decision makers towards building secure LLM agents.

Related work. LLM Agents. Modern LLM-powered applications build upon the growing capabilities of instruction-tuned LLMs to understand complex natural language instructions, reason about tasks, and interact with external systems through well-defined interfaces. These applications (or “agents”) typically follow a pattern of receiving natural language input, using an LLM to convert this input into action plans, and execute these plans through available tools or Agent Computer Interfaces (Patil et al., 2024; Yang et al., 2024; Yao et al., 2022).

Prompt Injection Attacks. Prompt injection attacks occur when adversaries introduce instructions into the content processed by an LLM, causing it to deviate from its intended behavior. Most of these attacks insert malicious instructions to an otherwise benign user prompt, analogously to an SQL injection (thus, the name) (Goodside, 2022a; Willison, 2022; Liu et al., 2024; Kang et al., 2024; Fu et al., 2023).

Attackers may pursue various objectives through prompt injection1 e.g., unauthorized tool execution to manipulate system state, gathering and exfiltration of protected data, manipulation of the agent’s reasoning or output, or denial of service through resource exhaustion.

Existing Defenses can be categorized into LLM-, user-, and system-level defenses:

• LLM-level defenses include prompt engineering to strengthen the model’s resistance to injections and adversarial training to recognize and reject malicious or unexpected inputs (Wallace et al., 2024; Yi et al., 2023; Zou et al., 2024; Abdelnabi et al., 2025a). While these heuristic approaches provide some protection, they do not provide guarantees.

• User-level defenses typically involve confirmation mechanisms (Wu et al., 2025), requiring human verification before executing sensitive actions. While theoretically effective, these approaches can significantly impact system automation and usability, and pose a safety risk themselves with tired or overloaded human verifiers approving unsafe actions. Advances in data attribution techniques (Siddiqui et al., 2024), or explicit control- and data-flow extraction (Debenedetti et al., 2025) may help building more efficient and effective mechanisms for human verification.

• System-level defenses enhance LLM applications by integrating external verification and control mechanisms, and represent one of the most promising paths toward robust safety guarantees. Designing models inherently robust to attacks that elicit arbitrary outputs — such as prompt injection — is extremely challenging (e.g. in computer vision, adversarial examples remain an open problem after over a decade of research (Szegedy et al., 2014)).

Method. Current LLM agent designs are typically intended to be general-purpose, meaning they aim to automate a wide range of workflows. Such agents have access to powerful tools (e.g., code execution (Yang et al., 2023)) and are expected to solve arbitrarily complex tasks while interacting with untrusted third-party data.

While future improvements in agent capabilities and attack detection might improve the security of these general-purpose agents, such improvements are likely to remain heuristic in nature—and thus inherently brittle. As long as both agents and their defenses rely on the current class of language models, we believe it is unlikely that general-purpose agents can provide meaningful and reliable safety guarantees.

This leads to a more productive question: what kinds of agents can we build today that produce useful work while offering resistance to prompt injection attacks? In this section, we introduce a set of design patterns for LLM agents that aim to mitigate — if not entirely eliminate — the risk of prompt injection attacks. These patterns impose intentional constraints on agents, explicitly limiting their ability to perform arbitrary tasks.

The design patterns we propose share a common guiding principle: once an LLM agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential actions—that is, actions with negative side effects on the system or its environment. At a minimum, this means that restricted agents must not be able to invoke tools that can break the integrity or confidentiality of the system. Furthermore, their outputs should not pose downstream risks — such as exfiltrating sensitive information (e.g., via embedded links) or manipulating future agent behavior (e.g., harmful responses to a user query).

To realize these guarantees in practice, we introduce design patterns that limit the capabilities of agents in principled ways. These patterns provide concrete, composable strategies for building agents that remain useful while offering meaningful resistance to prompt injection attacks.

Below, we describe six LLM agent design patterns that enforce some degree of isolation between untrusted data and the agent’s control flow. In Appendix A, we further describe general “bestpractices” for enhancing LLM agent security (e.g., running actions in a sandbox, or asking users for confirmation for sensitive actions). We distinguish these from the design patterns below, as we believe that every LLM agent should incorporate these best practices to the extent possible.

1. The Action-Selector Pattern. A relatively simple pattern that makes agents immune to prompt injections — while still allowing them to take external actions — is to prevent any feedback from these actions back into the agent. The agent acts merely as an action selector, which translates incoming requests (presumably expressed in natural language) to one or more predefined tool calls.

This design pattern acts like an LLM-modulated “switch” statement that selects from a list of possible actions. With more generality, these actions could be templated (e.g., a predefined SQL query with placeholders for variables to be filled in by the agent).

As an example, an AI agent that serves as a customer service chatbot may have a fixed set of actions available to it, and choose an action based on the user’s query, e.g., • Retrieve a link to the customer’s last order; • Refer the user to the settings panel to modify their password; • Refer the user to the settings panel to modify their payment information This pattern can also be effective against prompt injections in the user input, by enforcing that the prompt is removed before the agent replies to the user (see pattern 6)).

2. The Plan-Then-Execute Pattern. A more permissive approach is to allow feedback from tool outputs back to the agent, but to prevent the tool outputs from influencing the choice of actions taken by the agent. The “plan-then-execute” pattern (see e.g., Debenedetti et al. (2024)), allows the agent to accept instructions to formulate a plan to be executed (i.e., a fixed list of actions to take).

Conclusion. Building AI agents that are robust to prompt injection attacks is essential for the safe and responsible deployment of large language models (LLMs). While securing general-purpose agents remains out of reach with current capabilities, we argue that application-specific agents can be secured through principled system design. To support this claim, we propose six design patterns for making AI agents resilient to prompt injection attacks. These patterns are intentionally simple, making them easier to analyze and reason about — an essential property for deploying AI in high-stakes or safety-critical settings. We demonstrate the practical applicability of these patterns through ten case studies spanning diverse domains. For developers and decision-makers, we offer the following recommendations:

Recommendation 1: Prioritize the development of application-specific agents that adhere to secure design patterns and clearly define trust boundaries.

Recommendation 2: Use a combination of design patterns to achieve robust security; no single pattern is likely to suffice across all threat models or use cases.

We hope this work contributes to the foundation for building safer AI agents and minimizing the risks posed by prompt injection attacks in real-world deployments.