Why AI Agent Handoffs Fail
The handoff is the new API boundary, and most teams treat it like a prompt. Why that fails without ever throwing an error, and what a reliable handoff carries instead.
Things That Break When Agents Meet Reality · Part 2
A customer rang the bank on a Monday morning. They had been locked out of their own account.
No breach. No fraud. A legitimate person, a legitimate transaction, frozen by a system that had done exactly what it was built to do.
The team pulled the trace. Every step had completed. The research agent ran, the analysis agent ran, the decision agent ran. The observability dashboard showed one word across the whole pipeline.
SUCCESS.
Nothing had crashed. That was the problem.
Three boxes, three arrows
On a whiteboard the pipeline looked clean.
Three agents, three capabilities, one workflow.
But the boxes weren’t where the trouble lived. The trouble lived in the arrows.
Every arrow is a boundary. The research agent had told the analysis agent that a customer carried a high risk score. The analysis agent read “high risk” as reject. The research agent had meant worth a closer look. Nowhere in the trace did those two readings ever meet. No exception fired, because from the machine’s point of view nothing was wrong. A value went in, a value came out, the run went green.
We have spent decades learning to engineer boundaries between software components. We call them APIs. We write contracts, specify schemas, validate inputs, version interfaces, handle errors, monitor latency. We do all of it because of one lesson that cost the industry a great deal to learn: the interface between two systems usually matters more than either system on its own.
Now the components on both sides of the interface are language models. And somewhere along the way we started treating the interface as a prompt.
This is the second piece in a series I’m running, Things That Break When Agents Meet Reality. It’s about the engineering problems that only surface once agentic systems leave the demo and have to survive production. Part 1, Why Agent-to-Agent Communication Fails, landed on one line: having five smart agents doesn’t give you one smart system. That leaves an obvious next question. If agents have to communicate reliably, what does that communication actually have to be?
An agent handoff isn’t a conversation
Picture the research agent telling the analysis agent what it found.
A person might say: “I found three transactions that look off. You should take a closer look.”
Perfectly reasonable between two humans. But strip the human out of the receiving end and the sentence falls apart.
Does “three” mean exactly three?
Does “off” have a threshold behind it?
Which transactions, on what evidence, to what confidence?
Is this a claim that they are fraudulent, or a request to check?
People resolve that ambiguity without noticing. Deterministic software refuses to.
Language models sit in the worst possible place between the two: brilliant at interpreting ambiguity, and equally brilliant at inventing an interpretation when there shouldn’t be one.
In a fraud pipeline, that second talent is the one that freezes a real customer’s account.
The old world had contracts for a reason
A traditional service call carries a contract:
{
"customer_id": "12345",
"risk_score": 0.87,
"decision": "REVIEW"
}Drop customer_id and the request fails. Send risk_score as text and validation fails. Send a decision outside the permitted set and the request fails. That strictness is deliberate. The system would rather stop loudly than guess quietly.
Replace the sender with an agent and the same information arrives like this:
“I’ve identified the customer as 12345. Based on the available information, this appears to be the relevant account.”
Useful to read. Miserable to build on. The receiver now has to parse prose to recover a value that was never meant to be ambiguous. We took a deterministic interface and swapped in a probabilistic one, then acted surprised when the decision at the far end drifted.
Contract drift: the change nobody made
Here is the failure mode that is specific to agents.
Two agents agree on a shape:
customer_id
account_status
risk_scoreSix months later someone edits the sending agent’s prompt. It starts returning two extra fields:
customer_id
account_status
risk_score
reasoning
recommended_actionNothing breaks. The JSON is valid. The workflow runs. But the receiving agent quietly starts leaning on recommended_action instead of forming its own judgement. The architecture has changed and no one touched the API.
That is contract drift. With traditional software, an interface change is an event: a version bump, a review, a deprecation notice. With agents, the meaning of an interface can shift because of a prompt edit, a model upgrade, a new tool, a memory change, a policy tweak. The schema stays valid while its meaning moves underneath you. That is a different class of problem, and most monitoring is blind to it.
The most dangerous handoff is the one that succeeds
A malformed API call gives you an error, which is a gift, really. It tells you where to look.
An agent gives you something worse: a well-formed answer that means something other than what the sender intended.
The research agent says “high risk score.” The decision agent hears reject. The research agent meant investigate. Nothing fails and the pipeline completes. The dashboard goes green. And a customer who did nothing wrong spends Monday on hold.
Semantic failures are more dangerous than technical ones, because every tool we own is built to catch the technical kind.
What a handoff should actually carry
Start treating handoffs as contracts rather than messages, and a useful contract works on four layers.
Structure. What fields exist. The schema.
Semantics. What those fields mean. A field called
statuswith the valueVULNERABLEis valid JSON and almost useless on its own. Has the customer disclosed vulnerability, has the system inferred it, has a human confirmed it, is it a legal protection, was it true yesterday and unchecked since? The schema is fine. The meaning is undefined, and in a regulated business the meaning is the whole point.Authority. What the receiver is allowed to do with the payload. Receiving a risk score is not the same as being cleared to freeze an account on it.
Provenance. Where the information came from.
risk_score = 0.87is a number until you know which model produced it, on what data, under which policy, and when.
A mature handoff stops being “here’s some JSON.” It becomes: here is what I know, what it means, where it came from, and what you are allowed to do with it.
Every arrow is an evaluation point
This is the part I think matters most, and it is where the whole series has been heading.
We tend to evaluate agents one box at a time. Did the research agent find the right transactions? Did the decision agent make the right call? Useful questions, but they are incomplete.
The better question lives on the arrow: did the sending agent give the receiver everything it needed to do its job correctly? That is a different evaluation, and it is one you can actually run. Schema validity, completeness, semantic correctness, context preservation, intent preservation, evidence quality, confidence calibration. Each of those can be checked at the boundary before the payload moves on.
Do that, and the arrows in your diagram stop being decoration. They become the places you measure. When something goes wrong in production, you stop reaching for “the agent hallucinated” and start asking which handoff corrupted the workflow. That question has an answer. The other one never did.
This is Evaluation Debt in its native habitat. It is also why evaluation isn’t a final checkbox bolted on at the end of the pipeline. It’s the architecture that keeps the whole thing alive, and in an agentic system that architecture lives on the boundaries, not inside the boxes.
The architecture that follows
Once handoffs are contracts, the system grows the parts it was missing. Validation before a payload reaches the next agent. Versioning when the contract changes on purpose, so drift becomes an event again. Tracing, so you can see what crossed every boundary. Evaluation at each arrow. A defined failure path when the receiver rejects a payload, and human escalation when the system can’t establish that a handoff is safe.
Agent orchestration starts to look far more like distributed systems engineering than like prompt design. That is the mindset shift I think the next year of this work turns on.
Back to the customer on hold. In the pipeline that catches this, “high risk” never reaches the decision agent as a bare phrase. It arrives as a typed claim with a meaning, a source, and a ceiling on what can be done with it. The decision agent can act on investigate and is not permitted to escalate it to freeze on its own. The dashboard still goes green. This time the green is telling the truth.
Where this goes next
We are building software components that can reason, which is genuinely powerful and quietly introduces something APIs never had. An API never decides what your payload probably means. An agent does. The space between two agents isn’t an API and it isn’t a conversation. It’s a third thing: a reasoning boundary, and it has to be engineered on purpose.
The next pieces in this series follow the cracks further out.
What happens when you keep adding agents to a system that already has too many.
Why an agent can pass every evaluation and still fail in production.
What happens when an agent has all the right information and still lacks the context to use it.
If you take one line into your next design review, take this one: the boundary between two agents is a contract, and an untested contract is a production incident with a delay on it.
See you in Part 3.
One request:
Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.
👉 Take the survey here
Thanks for reading,
Sandi
P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.
Ask your friends to join.
More valuable content coming your way.
Thats why we build IM