← AI Agent Security Engineering: Prompt Injection, Identity, MCP, and Tool ControlsSkip to lesson ↓

Module 1: See the System and Its Boundaries

Contain Prompt Injection Before It Reaches Authority

Follow a hidden instruction from a support ticket into the agent's context and place independent controls between generated intent and external action.

Approx. 4 min reading · Includes diagrams; allow extra time for reflection and linked resources.

Start reading ↓
About this learning content and editorial review
About this learning content: Courses, lessons, assessments, explanations and illustrations may be created with the help of artificial intelligence. We review and check the material and do our best to avoid incorrect or outdated information, but mistakes, omissions or ambiguous questions may remain. Please verify information before relying on it for professional, security, legal or operational decisions. Read the full notice or report an issue.

Your focus

What you’ll learn

  • Trace direct or indirect prompt injection without assuming that every unexpected action proves an attack.
  • Place enforceable controls outside the model that limit the consequence of manipulated output.

Explains direct and indirect prompt injection as an instruction-boundary problem, then develops layered controls around untrusted content, tool availability, arguments, destinations, approvals, and effects.

The ticket is data until the model reads it as instruction

A customer attaches an exported email to a support ticket. Deep inside the quoted thread is a sentence telling any assistant to ignore the refund request, search other customer records, and send the results to an external address. A human sees untrusted correspondence. The model receives tokens that resemble instructions alongside the application’s trusted directions.

Indirect prompt injection occurs when instructions arrive through content the system retrieves or processes rather than through the user’s direct prompt. Web pages, documents, emails, code comments, issue descriptions, tool results, and long-term memory can all become carriers. The model may be unable to reliably separate a legitimate instruction from text that only looks authoritative. OWASP therefore treats prompt injection as a structural application risk, not a wording puzzle solved by one stronger system prompt.

Do not leap from an unusual tool call to a claim of malicious injection. Confusing instructions, model error, stale memory, a faulty tool description, or an ordinary application bug may produce similar behavior. Preserve what content entered context, what policy applied, what action was proposed, and what effect occurred. The learner-facing guidance in Safe Generative AI at Work explains why connected data must be treated as a new input boundary.

Two differently trusted inputs enter the model, so authority cannot depend on the model perfectly separating them.
Trusted directions and untrusted content meet in one context — Two differently trusted inputs enter the model, so authority cannot depend on the model perfectly separating them.

Move enforcement out of the conversation

The unsafe design exposes a broad send_message tool and asks the model to use it responsibly. The safer design changes what the surrounding system permits. The support workflow offers only necessary operations. A refund tool accepts a customer identifier bound to the active case, an amount within policy, and an idempotency key. A separate message tool restricts recipients to approved channels. Neither tool accepts arbitrary code, arbitrary destinations, or credentials supplied by model output.

Validate model-selected arguments as hostile application input. Use typed schemas, allowlists, length and range checks, tenant ownership checks, canonical destination handling, and ordinary secure coding at the boundary. Enforce the same rules at the target service because an orchestrator-side check can be bypassed by another caller or fail during a version change. A model refusal is helpful behavior; a target-side denial is a security control.

High-consequence actions need meaningful confirmation. Show the reviewer the operation, target, data leaving the boundary, expected effect, and supporting evidence. Approval is weak when the person sees only “Allow?” or habitually confirms a queue they cannot inspect. Make approval fresh, specific, and invalid after relevant arguments change.

Tool selection, arguments, authorization, confirmation, and target policy are checked after the model proposes an action.
Independent gates limit a manipulated proposal — Tool selection, arguments, authorization, confirmation, and target policy are checked after the model proposes an action.

Design for residual risk instead of promising a cure

Content can be labeled, isolated, summarized through a lower-authority path, or excluded from tool-capable context. Those measures reduce exposure, but none proves that every injection will be recognized. The durable strategy is to limit the authority available when untrusted content is present and to verify consequential effects independently.

Define a degraded mode. When the agent processes external content, it might retrieve and draft but lose send, delete, payment, or credential-management capabilities. When an injection detector fires, treat the result as a risk signal rather than truth: narrow tools, require review, and preserve evidence. NIST’s 2026 agent-identity work similarly highlights contextual authorization that can change when an agent operates on untrusted data.

The support agent can now read a hostile ticket without automatically gaining a path to arbitrary disclosure. The next question is whose authority it should use for the refund it is still allowed to propose.

Resources

  • OWASP Prompt Injection — Use OWASP’s current explanation of direct and indirect prompt injection and why untrusted content becomes an instruction surface.
  • OWASP Top 10 for Agentic Applications 2026 — Relate goal hijacking, tool misuse, identity abuse, memory poisoning, and rogue-agent behavior to the layered controls taught here.