“How do we let LLM agents touch production systems? What happens when an upstream agent’s probabilistic reasoning triggers an irreversible action downstream?”
Multi-agent architectures introduce the risks of unmapped decision dependencies. Architectures where authenticated, read-only agents exert unconstrained, transitive authority over high-risk execution layers without deterministic boundaries in between.
This is the last part of our series on what breaks when AI agents transition from prototypes to production:
Part 1: Communication Failure: Why Agent-to-Agent Communication Fails
Part 2: Contract Failure: Why AI Agent Handoffs Fail
Part 3: Context Failure: Your Agent Isn’t Hallucinating…
Part 4: Evaluation Failure: Evaluating AI Agent Handoffs
Part 5: Agent Sprawl: The Hidden Nightmare of Your Next Agent Call
Part 6: Decision Blindness: How to Fix “Decision Blindness” in AI Agents
Part 7: Authority Failure: Stop Giving Agents Tools. Start Giving Them Boundaries.
Part 8: Complexity Failure: The Hidden Cost of Evaluation Debt in Multi-Agent Systems
Part 9: Commitment Failure: Handling the Risks of Agent Actions
Today, we look at the failure that is often overlooked: Architecture Failure
👉 Request: I'd love to hear from you. Take the short survey at the bottom of this article and tell me what you'd like to learn more about.
We have all stood in front of a whiteboard and drawn the exact same boxes and arrows. Users. Agents. Orchestrators. Vector databases. Tools. Target APIs. We map the data flow, define the component dependencies, and wrap it all in IAM policies. The architecture looks scalable, the boundaries look secure, and least privilege is demonstrably enforced.
But when you put this system into production, that diagram is lying to you.
It tells you how the system runs, but it completely obscures how the system thinks. As we transition from shipping deterministic microservices to deploying agentic systems, we are introducing a blind spot into our engineering practices. Standard component and permission graphs map system dependencies. They fail to map decision dependencies. And in a multi-agent architecture, decisions are the most critical dependencies you have.
The component graph is an illusion
Consider a standard enterprise automation workflow:
From a traditional systems architecture perspective, this is a solved problem. We understand how to secure it. We grant the Research Agent read-only access to customer data. We isolate the Recommendation Agent. We restrict the Payment API so that only the Execution Agent can invoke it.
If we audit the IAM graph, it looks impenetrable. The Research Agent cannot execute payments. It lacks the scope, the tokens, and the network path.
But the component graph and the IAM graph are hiding a third, undocumented architecture: the decision graph.
The operational reality looks like this:
These are not the same graph. The first tells us which services can establish a TLS connection. The second tells us which decisions depend on prior probabilistic interpretations. In agentic systems, data isn’t simply passed; it is interpreted.
When Agent B receives data from Agent A, it is often not receiving raw telemetry or immutable facts. It is receiving Agent A’s conclusion.
The danger of transitive authority
This decision lineage exposes a massive vulnerability in how we currently threat-model AI systems.
If you ask an infrastructure engineer where the high-risk boundary is in the scenario above, they will point to the Execution Agent. But if an attacker wants to steal £2,000, they don’t need to compromise the Execution Agent or bypass the Payment API’s authentication. They just need to manipulate the unstructured data ingested by the read-only Research Agent.
If the Research Agent misinterprets a poisoned document and confidently outputs that a user is eligible for a refund, and the downstream agents simply accept that interpretation, the exploit succeeds. Every API call was authenticated. Every agent operated perfectly within its RBAC boundaries. Yet the system produced an unauthorized, catastrophic outcome.
I call this transitive authority: permissions are not inherited, but influence is.
A low-authority component that sits upstream of a high-authority decision effectively wields the power of that downstream system if there are no independent control boundaries in between. The risk profile of an agent is not defined solely by what it can access, but by what its outputs can materially influence.
JSON schemas are not strict enough
The engineering instinct here is to tighten the interface. We force agents to communicate via strict JSON schemas:
JSON
{
"customer_id": "cust_8912",
"recommendation": "REFUND",
"amount": 2000,
"confidence": 0.94,
"reasoning": "Policy exception granted based on retrieved context."
}
This is syntactically robust and operationally useless as a security boundary. A valid schema tells the downstream agent nothing about whether it should trust the payload. A syntactically perfect handoff can still carry a fundamentally compromised decision.
To establish a real boundary, the receiving agent must validate the trust provenance, not just the data structure.
What evidence supports this recommendation?
Was the amount calculated deterministically or hallucinated probabilistically?
Are we simply trusting an upstream LLM’s interpretation of a policy, or are we enforcing that policy in code?
Adding a “Critic” agent is often not enough
This leads to a common anti-pattern I see in early agentic implementations: attempting to solve probabilistic unreliability by adding more probabilistic reviewers.
It looks like defense in depth. It is actually just trust propagation. Five reasoning steps do not equal five control boundaries. If the Critic Agent is simply prompted to “review the Analyst’s work” using the same underlying context, it will often confidently validate the Analyst’s hallucination.
A real control boundary changes the operational mechanics. It introduces friction where friction is required.
The blueprint: 4 steps to secure your Decision Graph
To move from theoretical risks to production-grade resilience, engineering teams must implement a new set of architectural controls.
Here is how you build a secure decision graph in practice.
1. Work backwards from authority: Do not start your threat modeling at the API gateway or the read-only agent. Start at the point of highest authority, the Execution Agent or the mutation API, and trace the decision dependencies backward. For every upstream input, ask: Can this probabilistic interpretation materially change our final action? If the answer is yes, you need a control boundary.
2. Enforce deterministic control boundaries: Stop using an LLM to check another LLM. Build hybrid systems where probabilistic reasoning is constrained by deterministic code.
Force downstream agents to re-fetch critical source data rather than trusting upstream summaries.
Let the LLM recommend the refund, but use standard Python/Go to evaluate the threshold (
if amount > max_refund: require_human_approval()).Before an agent executes a command based on an entity’s status, cross-reference that status against an authoritative SQL database, not the agent’s memory.
3. Validate trust provenance, not just schemas: When Agent A hands off to Agent B, standardizing the JSON is only step one. Agent B must be engineered to verify how Agent A reached its conclusion. Pass the metadata of the decision, which policy was used, which documents were cited alongside the payload, and have the execution layer independently verify that those conditions justify the action.
4. Upgrade to decision observability: Standard APM traces (like OpenTelemetry or Jaeger) are built for the component graph. They will tell you that Agent A called Tool C and returned a 200 OK. But if a regulator or auditor questions an automated financial decision six months from now, you need decision provenance. You must log the environment at the exact moment of inference: the specific context injected into the prompt, the exact version of the policy evaluated, the intermediate outputs that became downstream inputs, and the deterministic controls that were bypassed or satisfied.
Architecture for the AI era requires reasoning across three distinct dimensions.
Component architecture defines communication.
Permission architecture defines access.
Decision architecture defines influence.
Before you build your next autonomous workflow, trace that third dimension. You will likely find that your most critical security boundary isn’t the API gateway, it is an upstream prompt handling untrusted text.
Share your perspective
Take the Reader Survey to shape upcoming deep dives on production agent architectures.
Subscribe to AgentBuild for engineering blueprints delivered weekly.
Share agentbuild.ai with your technical peers.
P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.
Ask your friends to join.
More valuable content coming your way.