We’re continuing our series on what really breaks when AI agents move from demos to production.
Part 1: Communication Failure: Why Agent-to-Agent Communication Fails
Part 2: Contract Failure: Why AI Agent Handoffs Fail
Part 3: Context Failure: Your Agent Isn’t Hallucinating…
Part 4: Evaluation Failure: Evaluating AI Agent Handoffs
Part 5: Agent Sprawl: The Hidden Nightmare of Your Next Agent Call
Part 6: Decision Blindness: How to Fix “Decision Blindness” in AI Agents
Today, we’re tackling a critical risk in agent design: the gap between what an agent can do and what it is authorised to do.
👉 Request: I'd love to hear from you. Take the short survey at the bottom of this article and tell me what you'd like to learn more about.
A common mistake in agentic implementations is confusing technical execution with enterprise authorisation. Just because an LLM has the technical capability to output a function call doesn’t mean the agent has the entitlement to actually perform the action. If we treat “the model can invoke this function” as equivalent to “the agent is authorised to perform this action”, we confuse capability with authority and create a path from a mistaken or manipulated response to a real-world consequence.
If you are building production agents, the design question is not simply which tools should we connect? It is under what conditions may this agent perform this specific action, for this user, on this resource, right now?
The tool list is not the policy
A common architecture exposes a set of functions to an agent: search a knowledge base, read a customer record, issue a refund, update a case, send an email. The model chooses which function to call and supplies its arguments.
That is a useful interaction pattern, but not an authorisation model.
The model may be wrong. It may misread the user’s intent, act on stale context, or be steered by instructions hidden in a document, email or tool response. OWASP describes “excessive agency” as the risk that unexpected, ambiguous or manipulated model outputs trigger damaging actions. It identifies three recurring causes:
excessive functionality
excessive permissions
excessive autonomy
Consider a customer-service agent that can look up orders and issue refunds. A request to “sort this out” might reasonably lead the agent to inspect an order, but it should not automatically give the agent authority to refund the full purchase price. The model’s judgement can help interpret the request; it should not be the final authority on whether the action is permitted.
Define authority as a boundary
A robust design separates the agent’s ability to propose an action from the system’s decision to authorise and execute it.
Before a tool call reaches a business system, evaluate it against explicit constraints:
Agent identity: Which registered agent is making the request, and what is it intended to do?
User context: Who initiated the task, and what may that person do?
Action and resource: Is this agent allowed to perform this operation on this specific record or account?
Conditions: Are the request, value, status, location, time or other relevant attributes within policy?
Approval: Does the action require a person to review and approve it before execution?
The exact policy will vary by use case, but the principle is consistent: authority must be checked for the action and not inferred from the fact that a tool is available.
This distinction is becoming a more explicit part of the agent-security conversation. NIST’s AI Agent Standards Initiative identifies agent authentication and identity infrastructure as areas of research, and points to work on identity and authorisation for enterprise agents.
Make the tools narrower
A broad tool gives the agent room to do more than the task requires. A function such as run_sql(query) or execute_shell(command) is flexible, but flexibility expands the set of possible actions and the consequences of misuse.
Prefer tools that express a specific business operation:
Instead of arbitrary SQL, provide
get_order_status(order_id).Instead of a general email interface, provide
draft_reply(case_id).Instead of
update_customer_record(fields), expose a narrowly scoped operation such asupdate_delivery_preference(customer_id, preference).
This does not make the model infallible. It reduces the damage a bad decision can cause. OWASP recommends limiting extensions to what is necessary, avoiding open-ended functions where possible, and enforcing minimum permissions in the downstream system.
A practical test is: If an attacker-controlled instruction reached the agent, what is the most damaging thing the exposed tools would let it do? If the answer includes actions unrelated to the agent’s purpose, the tool boundary is too broad.
Check every action, in the right context
Tool access should not be a one-time decision made when a session starts. Each invocation should be authorised against the current request, identity, resource and policy.
Where an agent acts on behalf of a user, preserve that user’s security context downstream. A shared service identity with broad access can silently turn a user-scoped task into an organisation-wide privilege. Instead, rely on user-delegated scoped tokens or implement mechanisms like RFC 8693 (OAuth 2.0 Token Exchange) to pass the initiating user’s precise context securely to the downstream service.
And do not rely on the model to decide whether it is allowed to act. Put the authorisation check in the tool service, API gateway, or downstream application. Architecting this component as a strict Policy Enforcement Point (PEP) backed by a Policy Decision Point (PDP), such as Open Policy Agent or Cedar. This ensures you have a component that enforces the rule regardless of what the model says. OWASP calls this “complete mediation”: validate requests against policy at the point where they reach the protected system.
In practice, the flow might look like this:
The agent proposes an action and supplies structured arguments.
The action service authenticates the agent and establishes the initiating user’s context.
A policy check evaluates the operation, target resource and relevant conditions.
The service denies, permits or routes the action for approval.
The downstream system enforces its own permissions when the operation executes.
The model can initiate this flow and it should not be able to bypass it.
Treat approval as an execution gate
“Ask the user to confirm” is not enough if the model can still call the tool without confirmation. A meaningful approval gate is enforced outside the model’s control and tied to the specific action being approved.
For example, an approval request should make clear exactly what will happen: which customer or record is affected, what value will change, and what the result is expected to be. Beware of state drift and the classic TOCTOU (Time-of-Check to Time-of-Use) vulnerability. If the agent modifies the proposed action after the human approves it, or if the underlying context changes, the authorisation must be evaluated again. To prevent this, tie approved actions to cryptographic hashes of the payload or require signed execution tokens to guarantee that the action executed is identical to the action approved.
Not every action needs a human in the loop. A low-impact, reversible update may be handled automatically within a clear policy. A high-impact or difficult-to-reverse action may require approval. The decision should be based on the consequences and risk of the operation, not on a blanket assumption that either all autonomy is safe or none of it is.
Log the decision, not just the call
A tool log that says “agent called issue_refund” is a start, but it does not explain whether the action was authorised or why.
For each decision, record enough to reconstruct the event: the agent and user identities, the requested action and target, the policy decision, any approval, the downstream result and the relevant correlation identifiers. Keep sensitive information within your retention and access policies, but make the authorisation path investigable.
Monitoring and rate limits can help detect or contain unwanted activity. They are useful safeguards, but they do not replace preventative controls: once an unauthorised write has happened, a log can show what went wrong without undoing it.
A production question worth asking
When reviewing an agent design, do not stop at “Which tools can it call?” Ask:
What boundary prevents this agent from taking an action that is outside its authority, even when the model is mistaken, the input is adversarial, or the tool call is technically valid?
You cannot just have this information in the system prompt, that’s just an instruction, not an enforcement boundary.
Agents will need capabilities to be useful. But capability should be granted narrowly, authority should be checked per action, and consequential operations should have controls the model cannot bypass.
Ultimately the goal is not to make agents powerless. It is to make their power explicit, scoped and enforceable.
How are you defining authority for the agents you’re building? Share the boundary you’ve found hardest to design: identity, permissions, approvals or execution, and what you’re doing to make it enforceable.
Thanks for reading,
Sandi.
Next week: We will talk about the Complexity of the Evaluation Surface Area.
One request:
Please take the survey.
Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.
👉 Take the survey here
P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.
Ask your friends to join.
More valuable content coming your way.
Thank you for this one. The gap between what an agent can do and what it is authorised to do is exactly where my business continuity work keeps landing too: when an over-privileged service identity fails or gets misused, the blast radius is far bigger than anyone planned for in recovery. Least privilege is an operational resilience control, not just a security one. I've subscribed back and will follow the rest of the series.