Start With the Action, Not the Model

Introduces the continuing support-agent scenario and builds a system model that distinguishes user purpose, model output, application decisions, credentials, tools, policies, and target-side effects.

A helpful summary becomes a security decision

Imagine a customer-support assistant that begins by summarizing tickets. At first it only transforms text. Then the team connects it to a customer database so it can retrieve account history. Next it receives a refund tool and a messaging tool. The interface still looks like one chat box, but the security problem has changed: generated text can now influence actions involving money, personal data, and external recipients.

Calling the whole arrangement “the AI” hides the parts that determine risk. A user or incoming ticket supplies content. An application assembles instructions and context. A model proposes a response or tool call. An orchestrator interprets that output. A credential authorizes an API request. A tool server performs an operation, and a business system records the effect. The model participates in the decision path, but it does not create API authority by producing words.

Begin every review with one concrete action: “issue a refund to this customer,” not “secure the chatbot.” Ask what initiates it, which data informs it, which component selects it, which identity executes it, which policy permits it, and where completion is recorded. This is the same evidence-led habit used in Agent Identity and Delegated Authority as Detection Data, applied before deployment rather than after an alert.

A tool call is not yet an effect

Suppose the run log says the agent called create_refund. That observation does not establish that money moved. The tool may have rejected the amount, timed out, queued work, returned a misleading success message, or completed after a retry. Secure design keeps the proposal, authorization decision, request, and target-side result distinct. Otherwise the application may tell the customer that a refund succeeded when only a request was generated.

Trace identifiers help relate records across the application, tool server, and target service. They do not prove that every related event caused the next one. Preserve stable action identifiers, idempotency keys, actor and target identifiers, policy versions, and a receipt from the system that owns the effect. The AI Tool-Call and MCP Telemetry Requirements page expands the evidence model you will use throughout this course.

This distinction also improves safety. A policy can reject the proposal before credentials are presented. The target service can enforce amount and account boundaries even if orchestration fails. A completion handler can compare the receipt with the approved action before notifying the user. Each component makes a smaller claim and verifies the next boundary.

Turn the map into a bounded purpose

The team now states the agent’s purpose precisely: it may summarize a ticket, retrieve the requesting customer’s support history, propose a refund below a defined limit, and submit it only after the required confirmation. “Help customers” is not an authorization rule. A bounded purpose names the subject, allowed data, available operations, targets, consequence limits, and required review.

Record assumptions beside the map. Does the ticket identity reliably match the account? Can a support employee switch tenants? Does the refund service perform its own checks? Which content is attacker-controlled? An architecture diagram without those questions can look complete while leaving the decisive trust relationships implicit.

You now have the spine for the rest of the course. The next lesson introduces malicious instructions inside the same ticket content and asks whether the system’s boundaries survive when the model follows the wrong text.

Resources

• NIST AI Agent Identity and Authorization Resource Hub — Follow NIST’s evolving implementation-oriented work on agent identity, authorization, accountability, and governance.