Subscribe now

When you have multiple agents talking to each other, passing data, and executing tasks, you eventually hit a deeply uncomfortable moment. Nope, it’s not when the system crashes, crashing is easy to debug. It’s when the system succeeds, and absolutely nobody in the room can explain why. Trust me, it happens a lot.

Let’s say a customer gets approved for a loan, or an automated fraud investigation escalates a case. You open your observability dashboard, and technically, everything looks perfect. The agent called a model, retrieved some documents, fired off a few tools, and returned a result.

But then a compliance officer taps you on the shoulder and asks,

“Why did the AI make that decision?”

And no one in the team knows why.

I call this decision blindness. Your logs are telling you exactly what happened, but you have absolutely no idea why. To fix it, we have to rethink how we monitor these systems from the ground up.

Standard telemetry has limitations

When we start building agentic systems, we naturally reach for conventional software observability. We track the standard APM (Application Performance Monitoring) metrics: model calls, API requests, token usage, and latency. That’s a great baseline, but a decision is fundamentally more than a sequence of network events.

Imagine an agent deciding whether to approve a customer request. Your event log might show that it retrieved a customer profile, pulled a policy document, called an LLM, and approved the request. You can see the execution pathway clearly, but the reasoning context is completely hidden.

• Which specific facts about that customer actually triggered the approval?

• What alternative choices did the agent weigh before it crossed the threshold to act?

Traditional event logs tell you what the infrastructure did, whereas a decision trace tells you how the system actually arrived at its conclusion.

Agents break traditional tracing

In traditional software, execution is deterministic enough that we can trace a decision straight through the code. If condition A is true, the system executes branch B. You can read the logic like a map.

Agents simply don’t work like that. Their decisions emerge from a messy, highly dynamic interaction between context, retrieved data, shifting policies, tool outputs, and the models themselves. All of those variables can change independently. The underlying foundation model might get an unseen update, your vector database might return slightly different context due to a new embedding, or a policy document might be rewritten. Because of this fluidity, just recording the final answer and the API calls isn’t enough. You have to capture the exact state of the world at the exact moment the decision was made.

Architecting a decision trace

A useful way to solve this is to treat every consequential agent decision as something that produces a structured trace. Instead of just tracking timestamps and API spans, you track the flow of logic.

The architecture of a good decision trace looks like this: Decision, Evidence, Context, Policy, Actions, and finally, the Outcome.

So, if an agent declines a transaction, the trace shouldn’t just record a “Declined” state in the database. It should explicitly show that it declined the transaction because of an unusual amount and a geographic inconsistency (the evidence), based on the current account state (the context), using Fraud Policy v3.8 (the constraints), after querying the transaction history and a sanctions API (the actions). Now you have a tangible, serialized record that another engineer, an auditor, or an investigator can actually reconstruct.

For a deeper dive read this article:

RAG has a provenance problem

I have seen this technical trap here that caught me off guard a number of times, especially when using Retrieval-Augmented Generation (RAG). Suppose your agent outputs a note saying, “The customer appears to have breached policy X.” Where exactly did that conclusion come from?

If the agent pulled that from a retrieved document, you need absolute provenance. You need to log the document ID, the exact version hash of that document, the specific chunk that was retrieved, and the timestamp. Without this kind of provenance, your evidence is basically ephemeral. Six months from now, you might still have the agent’s answer saved, but if that policy document has been updated three times since then, you’ll have no reliable way to prove what information the agent was actually looking at when it made the call. For production AI in regulated environments, that missing link is a massive liability.

How to implement this - few tricks

So, how do you practically wire this up? You don’t need to invent a new database, but you do need to change how you prompt your models and how you pass data.

1. Force structured outputs. Stop letting your agents reply with free-text paragraphs. Instead, force the LLM to return a rigid JSON object for every significant move. Alongside the final action, require fields for evidence_used, policy_version, and alternative_rejected. By making the model explicitly output its dependencies, you instantly generate the data for your decision trace right at the source.

2. Leverage OpenTelemetry “baggage.” When Agent A (the Researcher) finds a piece of evidence and passes the job to Agent B (the Executor), that evidence needs to travel with the request. Use standard distributed tracing features like OpenTelemetry baggage to pass this reasoning context along the chain. That way, when the final Executor agent makes a move, your trace contains the entire inherited history of why.

3. Hijack your LLM observability tools. Platforms like LangSmith, Langfuse, Braintrust, or Datadog LLM are great for capturing prompts and latencies. But don’t just use their defaults. Push your structured decision JSON into their custom metadata tags. You want to be able to query your dashboard for “Show me all decisions where Fraud Policy v3.8 was applied,” not just “Show me traces that took longer than 2 seconds.”

Evaluating integrity over outcomes

To be clear, I’m not suggesting we need to capture the model’s private, rambling chain-of-thought. We don’t need a massive text dump of the LLM talking to itself in the logs. What we really need is a structured, machine-readable record of the decision pathway.

This shift changes how we evaluate agents entirely. Right now, most evals just ask if the final answer was correct, which only measures outcome correctness. With decision traces, we can finally start asking if the agent arrived at that decision using acceptable evidence, authorized policies, and permitted actions. We can measure decision integrity. An agent can stumble into the right answer using completely hallucinated policies, and if you’re only checking the final output, you’d never know your system is hallucinating its logic.

Read my article below where I talk about Evaluation Graph:

Context & Chaos

Context Graphs as AI Evaluation Infrastructure

About the Author: Sandipan Bhaumik have spent almost 2 decades building Data & AI foundations. Now, through AgentBuild Weekly, he shares how builders and founders can move beyond AI hype to create Agentic systems that think, adapt, and truly work…

Read more

6 months ago · 20 likes · 2 comments · Sandipan Bhaumik

The Multi-Agent cascading failure

Take this exact problem and scale it up to a multi-agent architecture where you have a Planner, a Researcher, an Analyst, and an Executor all working together. The Executor makes the final call, but which agent introduced the critical piece of evidence? Which one decided to discard a valid constraint, and where did a flawed assumption enter the pipeline?

Traditional distributed tracing tools like Jaeger or OpenTelemetry will gladly tell you that Agent A called Agent B, and how many milliseconds it took. It won’t tell you that Agent A’s decision was based on stale evidence that Agent B then treated as indisputable fact. That is the dangerous gap between workflow observability and decision observability.

Ultimately, we need to change the fundamental question we ask when designing these systems. Don’t just ask if you can see what the agent did. Ask if you can reconstruct why it happened. A bad decision is highly visible, but decision blindness is silent. Having a log that just reads “LLM call, tool call, response” isn’t true observability. It’s just a receipt. And when a serious audit comes around, a receipt isn’t going to save you.

Stop printing receipts

Ultimately, we need to change the fundamental question we ask when designing these systems. The next time you’re reviewing a new agent architecture with your team, don’t just ask, “Can we see what the agent did?”

Ask, “Can we prove exactly why it did it?”

A bad decision in production is highly visible. The business usually feels it immediately. But decision blindness is silent. It sits hidden in your system, just waiting for the worst possible moment - an audit, a customer complaint, a compliance review to reveal that nobody actually knows how your AI works.

Having an observability dashboard that just reads “LLM call -> tool call -> response” isn’t true observability. It’s just a receipt. It tells you the transaction happened, but it tells you absolutely nothing about the logic behind the purchase. And when a serious audit comes around, a receipt isn’t going to save you.

One request:

Please take the survey.

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.

👉 Take the survey here

Thanks for reading,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Subscribe now

There is a moment in almost every agent architecture where the diagram starts looking impressive.

It begins simply:

Then someone asks:

“What if we add a planner?”

So it becomes:

Then:

“What if we add a researcher?”

Now we have:

Then someone adds a critic.

Then a reviewer.

Then a verifier.

Before long, the architecture looks like this:

It looks sophisticated.

It looks like a system built for serious work.

But there is an uncomfortable question we don’t ask often enough:

Did we actually make the system better?

Or did we just create six systems we now have to operate?

Every agent is another boundary

Adding an agent doesn’t just add another capability.

It adds another system boundary.

And every boundary creates work.

A new agent usually means:

• another context boundary

• another handoff

• another interface

• another failure mode

• another thing to evaluate

• another component to observe

• another source of latency

• another source of cost

And the problem compounds.

Imagine a simple workflow:

Agent A produces something that B needs to understand.

B transforms it and passes something to C.

Now suppose A is changed.

The output changes slightly.

B still works.

Its own evaluation still passes.

But C starts failing because B’s interpretation of A’s output has changed.

Nothing is necessarily “broken” inside any individual agent.

The boundary is broken.

This is exactly the kind of problem we’ve spent decades dealing with in distributed systems.

Except now the components aren’t deterministic services.

They are probabilistic systems interpreting language, context and intent.

That’s a very different kind of boundary.

Agent Sprawl

This is Agent Sprawl.

It is the tendency to turn every capability into another autonomous agent without first proving that the boundary adds value.

It is the agent equivalent of creating a microservice for every function in a codebase.

Microservices taught us an important lesson:

Decomposition is not automatically architecture.

You don’t create a service because you just can.

You create one because a boundary gives you something valuable.

Perhaps it gives you:

• independent scaling

• independent deployment

• ownership

• security isolation

• fault isolation

• a different data lifecycle

The same principle should apply to agents.

The question should be:

“What boundary does making this another agent give us?”

Not every role needs to be an agent

Consider a research workflow.

You might design it as:

That’s six agents.

But perhaps the actual requirements are much simpler.

The planner might just be a structured planning step.

The researcher might be a tool-calling capability.

The critic might be an evaluation step.

The reviewer might be a deterministic validation process.

And perhaps only the analyst and writer actually need autonomous reasoning.

Suddenly the architecture becomes:

That’s not necessarily less sophisticated.

It may be more engineered.

The mistake is assuming that every logical role in a workflow deserves an autonomous identity.

A function isn’t automatically an agent.

A prompt isn’t automatically an agent.

A tool isn’t automatically an agent.

A validation step certainly doesn’t need to become an agent just because an LLM could perform it.

The best agent architecture usually has fewer AI agents.

So when should you create another agent?

I use a simple test.

A new agent should introduce at least one meaningful boundary.

The following framework helps:

1. Capability boundary

The new agent does something fundamentally different from the existing agent.

For example, a specialised coding agent may have different reasoning patterns, tools and execution environments from a customer-service agent.

2. Data boundary

The new agent operates on a different data domain or requires different access.

For example, separating customer-facing reasoning from an internal risk-data environment may be a meaningful boundary.

3. Authority boundary

The new agent has a different level of permission.

An agent that recommends a transaction and an agent that can actually execute it may need to be separate because their authority is fundamentally different.

4. Failure-isolation boundary

The new agent can fail independently without taking down the rest of the workflow.

This can be particularly valuable for expensive or unpredictable operations.

If creating the agent doesn’t give you one of these boundaries, ask yourself why it exists.

Maybe it shouldn’t.

The hidden cost of adding another agent

Suppose a customer asks:

“Can I increase my credit limit?”

The request passes through:

The final answer is wrong.

Where do you look?

The model?

The prompt?

The context?

The handoff?

The policy?

The retrieved data?

The risk calculation?

The decision?

The interaction between two of them?

The more agents you add, the more possible places there are for something to go wrong.

And importantly, the failure may not belong to any one agent.

It can emerge between agents.

That’s the real cost of decomposition.

Not just more boxes.

More arrows, more interactions.

The architecture diagram hides this

This is where agent architecture can become deceptive.

On the diagram, adding another agent is easy.

You draw another box.

Connect with arrows.

Done.

But the production system doesn’t see boxes.

It sees:

• state

• context

• messages

• permissions

• tool calls

• timeouts

• retries

• model versions

• handoffs

• failures

• evaluations

And humans trying to figure out what happened when the workflow produced the wrong answer.

The diagram gets more impressive.

The operating model gets harder.

That’s why I think we need to change how we talk about multi-agent architecture.

The goal shouldn’t be to maximise the number of agents.

The goal should be to create useful boundaries.

Decompose for boundaries

The same principle applies whether you’re designing microservices, data platforms or agentic systems.

A boundary should earn its place.

If it gives you independent capability, data isolation, authority separation or failure isolation, it may be worth the complexity.

If it exists because someone thought:

“Wouldn’t it be cool if we had another agent?”

It probably doesn’t.

Because every agent you add creates another system boundary you now have to operate.

And eventually, the question stops being:

“How many agents do we need?”

It becomes:

“How many boundaries can we afford to operate?”

That’s a much more useful architectural question.

See you in Part 6, where we'll look at Decision Blindless

If you are interested in multi-agent system architectures, you might like this video.

One request:

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.

👉 Take the survey here

Thanks for reading,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

This is the fourth piece in a series I’m running, Things That Break When Agents Meet Reality. It’s about the engineering problems that only surface once agentic systems leave the demo and have to survive production.

Subscribe now

I’ve been thinking about a failure mode that is going to become increasingly common as we build bigger agentic workflows.

You evaluate an agent.

It passes.

You evaluate the next agent.

It passes too.

You wire them together.

Then something goes wrong.

I’ve seen variations of this in distributed systems for years. A service can work perfectly when you test it on its own and still cause problems when it becomes part of a larger system.

Agents make this harder because the boundary between two components isn’t always a clean API.

• Sometimes it’s JSON.

• Sometimes it’s a message.

• Sometimes it’s a chunk of retrieved context.

• Sometimes it’s an instruction that one model has generated for another model.

• And sometimes it’s all of those things at once.

That makes a simple question surprisingly difficult:

What exactly are we evaluating?

Let’s say we have three agents

Take a fairly ordinary enterprise workflow.

A customer asks for a refund.

We have:

• a Researcher that looks up the customer and relevant policy

• a Decision Agent that determines whether the refund should be approved

• an Executor that updates the CRM and triggers the appropriate action

Individually, they look good.

The Researcher retrieves the right information.

The Decision Agent makes the right decision.

The Executor calls the right API.

You run your evaluation suite and get something like:

Researcher       96%
Decision Agent   95%
Executor         99%

You’d be pretty comfortable taking those numbers to a review meeting.

Then you run the complete workflow.

And the success rate is much lower than expected.

The obvious reaction is to go back and look for the “bad agent”.

But that may be the wrong place to look.

Consider the handoff between the Researcher and the Decision Agent.

The Researcher might return:

{
  "customer_id": "12345",
  "issue": "refund",
  "eligibility": "uncertain",
  "policy": "refund_policy_v3",
  "evidence": [
    "Purchased 41 days ago",
    "Product was returned"
  ]
}

Looks fine.

But on another the same Researcher agent returns:

{
  "customer_id": "12345",
  "issue": "refund",
  "eligibility": "likely",
  "policy": "refund_policy_v3",
  "evidence": [
    "Purchased 41 days ago"
  ]
}

Still valid JSON.

Still all the expected fields.

The Researcher may even get a good score for this output.

But now the Decision Agent has to decide what “likely” means.

Is that enough to approve the refund?

Does it mean more evidence is required?

Should it send the case to a human?

We’ve got a perfectly reasonable agent talking to another perfectly reasonable agent, and the system can still make the wrong decision.

That’s the bit that tends to get missed.

The handoff itself is part of the system.

This is where I think our testing habits need to change

We’ve been doing this in software engineering for a long time.

We unit test components.

Then we test the interfaces between them.

Then we test the complete application.

Then, if we’re serious about production reliability, we deliberately make things fail.

Agent systems need the same treatment.

So yes, evaluate the individual agents.

But also evaluate:

Agent A → Agent B

and:

Agent B → Agent C

and eventually:

Agent A → Agent B → Agent C

The first tells you whether an agent can perform its job.

The second tells you whether two agents can actually work together.

The third tells you whether the workflow achieves what you intended.

Those are different tests.

I’ve seen teams put a lot of effort into the first one and surprisingly little into the other two.

The handoff needs a contract

When we talk about APIs, we’re comfortable with the idea of a contract.

The producer knows what it has to send.

The consumer knows what it is going to receive.

We can test that contract.

Agent handoffs need something similar.

And I don’t mean simply checking that the output conforms to a JSON schema.

A schema can be perfectly valid while the meaning is wrong.

I’d want to test things such as:

• Is the required information present?

• Is the information current?

• Is uncertainty represented?

• Can the receiving agent distinguish “unknown” from “no”?

• Is the evidence attached?

• Can we trace where the information came from?

• Is there enough context for the next decision?

• What happens when something is missing?

That last question is particularly important.

A production system will eventually receive an incomplete handoff.

The interesting question isn’t whether that can happen. It will.

The question is what the next agent does when it does.

• Does it stop?

• Does it ask for more information?

• Does it make a guess?

• Does it escalate?

• Does it quietly continue?

That’s an evaluation case.

Then there is the workflow itself

Let’s say we’ve done all of that.

A is good.

A → B is good.

B is good.

B → C is good.

We still haven’t proved that:

A → B → C

works.

Why?

Because the workflow has a state.

And that state changes.

Imagine the Researcher retrieves a customer’s account at 10:01.

The Decision Agent makes its decision at 10:02.

At 10:03, the Executor performs the action.

What if something changed between those steps?

The account was updated.

A policy changed.

A previous transaction appeared.

A permission was revoked.

Another process modified the same record.

The information passed between the agents might have been correct when it was produced and wrong when it was acted upon.

That’s not really an LLM problem.

It’s a distributed-systems problem.

And I think that’s a useful way of looking at agentic architectures.

Here, we’re just building systems where some of the components happen to be probabilistic.

The evaluation needs to know what was true at the time

This is where I’ve become increasingly interested in the idea of an Evaluation Graph.

I have writiten about it here:

Suppose an evaluation says:

Evaluation #1842

Score: 0.94

Fine.

But what does that score actually mean six months later?

• Which model produced it?

• Which prompt?

• Which data?

• Which policy?

• Which business definition?

• Which retrieved documents?

• Which agent version?

• Which context?

• Which permissions?

If you can’t answer those questions, you have a score, but you don’t necessarily have a reproducible evaluation.

Now imagine recording the evaluation together with the context surrounding it.

Now the evaluation isn’t floating on its own.

It has relationships.

You can trace the result back to the conditions under which it was produced.

That’s a much more useful thing to have when you’re running a production system.

The world changes underneath your evals

Here’s a simple example.

You run the same evaluation today and tomorrow.

The test input hasn’t changed.

The workflow hasn’t changed.

The model hasn’t changed.

Today it passes.

Tomorrow it fails.

What happened?

Maybe the model changed.

But maybe the underlying data changed.

Maybe the policy changed.

Maybe a business definition changed.

Maybe a retrieved document was updated.

Maybe the context supplied to the agent was different.

Without that information, it’s very easy to blame the model for a failure that actually came from somewhere else.

This is why I don’t think evaluation should be treated as a snapshot.

For production systems, an evaluation is really a statement about a system at a particular point in time, under particular conditions.

Those conditions need to be part of the evaluation record.

This is where the evaluation surface starts getting large

Now add another agent.

Then a verifier.

Then a router.

Then a fallback.

Then shared memory.

Then a human approval step.

Then a couple of external tools.

The architecture starts looking something like this:

                  ┌──→ Planner ──→ Executor
Researcher ───────┤
                  └──→ Verifier ──→ Planner

Every new component adds another thing to evaluate.

But that’s only half of it.

Every new connection adds another boundary.

And every boundary can carry:

• context

• state

• instructions

• uncertainty

• data

• assumptions

That’s what I mean by Evaluation Surface Area.

It isn’t simply the number of agents in your architecture.

It’s everything that can influence the outcome.

Once you see it this way, adding another agent isn’t a free architectural decision.

You’re also adding more behaviour that needs to be understood and tested.

Now you see, there is so much underneath you need to engineer for.

(And you see posts claiming to “replace <insert job title> with AI Agents”, it is not that easy.)

What I want in the production trace

This changes the telemetry I’d want from an agent system too.

A final answer isn’t enough.

If the workflow failed, I want to be able to walk backwards through it.

Something like:

Workflow: 84721

Input
  ↓
Agent A
  ├─ model/version
  ├─ prompt/version
  ├─ retrieved data
  ├─ context
  ├─ tool calls
  └─ output
       ↓
A → B
  ├─ payload
  ├─ context
  └─ validation
       ↓
Agent B
  ├─ decision
  └─ output
       ↓
B → C
  ├─ payload
  └─ context
       ↓
Agent C
  ├─ tool calls
  └─ action
       ↓
Outcome

If something went wrong, I want to find the first place the system deviated from the expected path.

Maybe retrieval was wrong.

Maybe the handoff dropped some context.

Maybe the policy was stale.

Maybe the Decision Agent made a bad call.

Maybe the Executor received a valid instruction that was no longer safe to execute.

Maybe a retry caused the action to happen twice.

The trace should let me work that out.

Otherwise we’re left staring at the final answer and asking the model why it did something.

That’s not a particularly good debugging strategy.

So what should we actually evaluate?

For a production workflow, I’d think about it this way.

The agent

Can it do its job?

The handoff

Can the next agent reliably understand what it received?

The workflow

Does the complete sequence produce the right outcome?

The failure path

What happens when something is missing, stale, delayed or broken?

The context

What data, policy, model, prompt and state were in force when the evaluation happened?

That last one is easy to overlook.

But it becomes incredibly important once your system is changing underneath you.

Evaluation has to become part of the architecture

This is probably the biggest shift in thinking for me.

Evaluation shouldn’t sit at the end of the development process as a test suite we run before deployment.

For an agentic system, evaluation needs to sit alongside the architecture.

The agent graph tells us how work moves through the system.

The evaluation graph tells us how we understand and explain that behaviour.

And the two need to stay connected.

Because eventually someone is going to ask:

Why did this agent make that decision?

And “because it scored 94% on our eval set” isn’t going to be a satisfactory answer.

We need to be able to trace that decision back through the agent, the handoff, the context, the data, the policy, the model and the state of the system at that moment.

That’s the level of evaluation I think production agent systems are going to need.

Not just:

Did it pass?

But:

What happened, under what conditions, and can we prove it?

That’s a much harder engineering problem.

It’s also a much more interesting one.

Evaluate the nodes. Evaluate the edges. And keep the context that makes the evaluation meaningful.

See you in Part 5, where we'll look at why your agent can pass every evaluation and still fail in production.

One request:

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.

👉 Take the survey here

Thanks for reading,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

A few months ago, I was digging into why some AI systems return completely convincing answers that turn out to be dead wrong.

The interesting failures aren’t obvious hallucinations. The tricky cases look airtight on paper: the document is real, the customer is real, the policy is accurate, and the text was interpreted faithfully. Yet the final output is completely broken.

The problem isn’t the data itself. The agent simply has the right information applied to the wrong context.

Think about cooking dinner

Imagine preparing a meal for friends. You have chicken, vegetables, rice, spices, and a recipe you’ve made dozens of times. Halfway through prep, someone mentions that one of the guests is vegetarian.

Nothing in your kitchen failed. The chicken was fresh, the recipe was solid, and your technique was fine. You just lacked a crucial constraint about the people you were cooking for, and that missing detail invalidates the rest of your plan.

Enterprise AI agents run into this constantly.

When a good RAG response isn’t enough

Suppose an employee asks an HR bot: “How much parental leave am I entitled to?”

The system queries the internal knowledge base, pulls a valid policy document citing 26 weeks, quotes it accurately, and references the source. It looks flawless.

Except the employee is based in another region, works on a contractor agreement rather than permanent staff, or the policy was revised last week and the search index hasn’t refreshed.

The document wasn’t fabricated, retrieval didn’t crash, and the model didn’t misread the text. The agent simply had no awareness of the situation surrounding the query.

We’ve gotten quite good at feeding documents into LLMs. What we haven’t nailed is ensuring that data applies to this specific user, task, and moment in time.

RAG answers “what looks relevant?”

Standard RAG follows a predictable loop: user asks a question, the engine retrieves matching snippets, feeds them into the prompt, and the model outputs an answer.

The weak point is semantic relevance.

A search index will easily identify an SLA document for a question about service levels. But that document might be superseded by an addendum signed six months later, an unindexed regional policy, or an active exception logged in Salesforce.

The document is semantically relevant, but operationally invalid for that specific scenario.

This is the third piece in a series I’m running, Things That Break When Agents Meet Reality. It’s about the engineering problems that only surface once agentic systems leave the demo and have to survive production.

Context is scattered across systems

In production environments, the data required to resolve a single interaction rarely lives in one place.

A routine customer query might pull identity from the CRM, subscription tiers from Stripe, negotiated terms from the contract repo, open tickets from Zendesk, historical usage from Snowflake, and real-time events from Kafka.

The agent doesn’t need a dump of everything, in fact, dumping raw state degrades performance. It needs the exact slice of state governing that specific decision.

Context goes stale

Systems drift out of sync all the time.

If a company shortens its cancellation window from 30 days to 14 days, the source database might update immediately while the vector store lags behind. The agent keeps quoting the 30-day policy. The model didn’t hallucinate, semantic search didn’t fail, and the pipeline ran as designed. The data was simply historically true, but currently false.

This creates a core infrastructure challenge: is a document valid because it matches search terms, or because it reflects the current operational state?

Basic metadata like created_at and updated_at rarely cuts it. Systems need fields like valid_from, valid_until, supersedes, source_system, and scope, because last updated and currently applicable are entirely different concepts.

Multi-agent compounding errors

This gets significantly messier across multi-agent workflows.

Customer Agent -> Pricing Agent -> Compliance Agent -> Execution Agent

Each service queries its own domain: the customer agent reads the CRM, the pricing agent checks billing tiers, compliance reads policy docs, and execution writes to the transactional database.

Each agent executes its sub-task correctly, but they might operate on mismatched snapshots of state. One sees a legacy plan, another reads an updated tier, and compliance applies an exception logged yesterday that the execution worker never receives.

By the end of the chain, nobody hallucinated, but the pipeline failed due to context fragmentation. It’s an insidious failure mode because every individual log looks green in isolation.

The missing source of truth

Enterprise systems frequently have conflicting definitions of basic terms.

If an account is marked Active in HubSpot, Past Due in Stripe, and At Risk in Zendesk, which one does the agent rely on?

A human operator knows that Stripe owns payment state while HubSpot owns account lifecycle. An LLM lacks that institutional intuition unless it is explicitly engineered into the pipeline.

Teams need clear ownership around business objects:

• What the field represents

• Which system owns authoritative write access

• Update latency and sync schedules

• Entity boundaries and constraints

Prompt engineering cannot fix underlying data governance gaps.

The handoff dilemma

When a research agent finishes a task and passes output to an analyst agent, what should that payload contain?

Passing the entire raw dialogue blows up the context window with conversational noise, token bloat, and stale assumptions. Conversely, sending a bare-bones summary drops critical constraints, source provenance, and workflow state.

Agent handoffs require intentional payload schemas: passing the exact state required for downstream decisions; nothing more, nothing less.

Engineering context assembly

For simple chatbots, context is just conversation history and a few retrieved chunks. For decision-making agents, context becomes an active assembly layer:

Before prompt construction, an assembly pipeline must resolve:

• Time: Is the data current and actively enforced?

• Entity: Does this rule apply to this specific account or tier?

• Authority: Is this the definitive system of record for this attribute?

• State: What prior steps and mutations occurred in this session?

• Permissions: Does the user/agent have authorization to view this data?

• Provenance: Can we trace the origin of this assertion?

These are traditional data engineering patterns applied upstream of LLM orchestration.

Bigger context windows won’t solve this

Expanding context windows to millions of tokens doesn’t eliminate the need for curation.

Giving a model an unfiltered firehose of internal data introduces conflicting policies, outdated records, noisy intermediate reasoning, and distraction. The engineering challenge isn’t how many tokens a model can swallow; it’s how reliably we assemble the exact state required for a specific decision.

The evaluation split

When an agent takes the wrong action, the default reaction is to blame model reasoning.

In reality, agent evaluations need to decouple two distinct questions:

1. Reasoning Quality: Did the model reach a sound conclusion based on the payload provided?

2. Context Quality: Did the orchestration layer supply the correct, authoritative state in the first place?

If the reasoning failed, you tweak the prompt, model, or tool definitions. If the context was wrong, no amount of prompt tweaking will prevent the failure.

Why this matters for action-taking agents

When an agent just generates text, bad context produces bad copy. When an agent has write access, bad context triggers bad transactions.

A refund bot might look up the right policy, calculate the correct amount, and match the customer record, yet execute a duplicate payout because it lacked visibility into a refund processed ten minutes prior.

At that stage, context management becomes part of the system’s operational control plane.

Rethinking RAG

RAG isn’t going anywhere, but its role needs reframing.

Moving from simple document search to context assembly means pulling simultaneously from vector indexes, transactional APIs, event buses, state machines, and access control layers.

RAG provides reference material. Context provides the operational situation.

See you in Part 4, where we'll look at why your agent can pass every evaluation and still fail in production.

One request:

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.

👉 Take the survey here

Thanks for reading,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

A customer rang the bank on a Monday morning. They had been locked out of their own account.

No breach. No fraud. A legitimate person, a legitimate transaction, frozen by a system that had done exactly what it was built to do.

The team pulled the trace. Every step had completed. The research agent ran, the analysis agent ran, the decision agent ran. The observability dashboard showed one word across the whole pipeline.

SUCCESS.

Nothing had crashed. That was the problem.

Three boxes, three arrows

On a whiteboard the pipeline looked clean.

Three agents, three capabilities, one workflow.

But the boxes weren’t where the trouble lived. The trouble lived in the arrows.

Every arrow is a boundary. The research agent had told the analysis agent that a customer carried a high risk score. The analysis agent read “high risk” as reject. The research agent had meant worth a closer look. Nowhere in the trace did those two readings ever meet. No exception fired, because from the machine’s point of view nothing was wrong. A value went in, a value came out, the run went green.

We have spent decades learning to engineer boundaries between software components. We call them APIs. We write contracts, specify schemas, validate inputs, version interfaces, handle errors, monitor latency. We do all of it because of one lesson that cost the industry a great deal to learn: the interface between two systems usually matters more than either system on its own.

Now the components on both sides of the interface are language models. And somewhere along the way we started treating the interface as a prompt.

This is the second piece in a series I’m running, Things That Break When Agents Meet Reality. It’s about the engineering problems that only surface once agentic systems leave the demo and have to survive production. Part 1, Why Agent-to-Agent Communication Fails, landed on one line: having five smart agents doesn’t give you one smart system. That leaves an obvious next question. If agents have to communicate reliably, what does that communication actually have to be?

An agent handoff isn’t a conversation

Picture the research agent telling the analysis agent what it found.

A person might say: “I found three transactions that look off. You should take a closer look.”

Perfectly reasonable between two humans. But strip the human out of the receiving end and the sentence falls apart.

• Does “three” mean exactly three?

• Does “off” have a threshold behind it?

• Which transactions, on what evidence, to what confidence?

• Is this a claim that they are fraudulent, or a request to check?

People resolve that ambiguity without noticing. Deterministic software refuses to.

Language models sit in the worst possible place between the two: brilliant at interpreting ambiguity, and equally brilliant at inventing an interpretation when there shouldn’t be one.

In a fraud pipeline, that second talent is the one that freezes a real customer’s account.

The old world had contracts for a reason

A traditional service call carries a contract:

{
  "customer_id": "12345",
  "risk_score": 0.87,
  "decision": "REVIEW"
}

Drop customer_id and the request fails. Send risk_score as text and validation fails. Send a decision outside the permitted set and the request fails. That strictness is deliberate. The system would rather stop loudly than guess quietly.

Replace the sender with an agent and the same information arrives like this:

“I’ve identified the customer as 12345. Based on the available information, this appears to be the relevant account.”

Useful to read. Miserable to build on. The receiver now has to parse prose to recover a value that was never meant to be ambiguous. We took a deterministic interface and swapped in a probabilistic one, then acted surprised when the decision at the far end drifted.

Contract drift: the change nobody made

Here is the failure mode that is specific to agents.

Two agents agree on a shape:

customer_id
account_status
risk_score

Six months later someone edits the sending agent’s prompt. It starts returning two extra fields:

customer_id
account_status
risk_score
reasoning
recommended_action

Nothing breaks. The JSON is valid. The workflow runs. But the receiving agent quietly starts leaning on recommended_action instead of forming its own judgement. The architecture has changed and no one touched the API.

That is contract drift. With traditional software, an interface change is an event: a version bump, a review, a deprecation notice. With agents, the meaning of an interface can shift because of a prompt edit, a model upgrade, a new tool, a memory change, a policy tweak. The schema stays valid while its meaning moves underneath you. That is a different class of problem, and most monitoring is blind to it.

The most dangerous handoff is the one that succeeds

A malformed API call gives you an error, which is a gift, really. It tells you where to look.

An agent gives you something worse: a well-formed answer that means something other than what the sender intended.

The research agent says “high risk score.” The decision agent hears reject. The research agent meant investigate. Nothing fails and the pipeline completes. The dashboard goes green. And a customer who did nothing wrong spends Monday on hold.

Semantic failures are more dangerous than technical ones, because every tool we own is built to catch the technical kind.

What a handoff should actually carry

Start treating handoffs as contracts rather than messages, and a useful contract works on four layers.

1. Structure. What fields exist. The schema.

2. Semantics. What those fields mean. A field called status with the value VULNERABLE is valid JSON and almost useless on its own. Has the customer disclosed vulnerability, has the system inferred it, has a human confirmed it, is it a legal protection, was it true yesterday and unchecked since? The schema is fine. The meaning is undefined, and in a regulated business the meaning is the whole point.

3. Authority. What the receiver is allowed to do with the payload. Receiving a risk score is not the same as being cleared to freeze an account on it.

4. Provenance. Where the information came from. risk_score = 0.87 is a number until you know which model produced it, on what data, under which policy, and when.

A mature handoff stops being “here’s some JSON.” It becomes: here is what I know, what it means, where it came from, and what you are allowed to do with it.

Every arrow is an evaluation point

This is the part I think matters most, and it is where the whole series has been heading.

We tend to evaluate agents one box at a time. Did the research agent find the right transactions? Did the decision agent make the right call? Useful questions, but they are incomplete.

The better question lives on the arrow: did the sending agent give the receiver everything it needed to do its job correctly? That is a different evaluation, and it is one you can actually run. Schema validity, completeness, semantic correctness, context preservation, intent preservation, evidence quality, confidence calibration. Each of those can be checked at the boundary before the payload moves on.

Do that, and the arrows in your diagram stop being decoration. They become the places you measure. When something goes wrong in production, you stop reaching for “the agent hallucinated” and start asking which handoff corrupted the workflow. That question has an answer. The other one never did.

This is Evaluation Debt in its native habitat. It is also why evaluation isn’t a final checkbox bolted on at the end of the pipeline. It’s the architecture that keeps the whole thing alive, and in an agentic system that architecture lives on the boundaries, not inside the boxes.

The architecture that follows

Once handoffs are contracts, the system grows the parts it was missing. Validation before a payload reaches the next agent. Versioning when the contract changes on purpose, so drift becomes an event again. Tracing, so you can see what crossed every boundary. Evaluation at each arrow. A defined failure path when the receiver rejects a payload, and human escalation when the system can’t establish that a handoff is safe.

Agent orchestration starts to look far more like distributed systems engineering than like prompt design. That is the mindset shift I think the next year of this work turns on.

Back to the customer on hold. In the pipeline that catches this, “high risk” never reaches the decision agent as a bare phrase. It arrives as a typed claim with a meaning, a source, and a ceiling on what can be done with it. The decision agent can act on investigate and is not permitted to escalate it to freeze on its own. The dashboard still goes green. This time the green is telling the truth.

Where this goes next

We are building software components that can reason, which is genuinely powerful and quietly introduces something APIs never had. An API never decides what your payload probably means. An agent does. The space between two agents isn’t an API and it isn’t a conversation. It’s a third thing: a reasoning boundary, and it has to be engineered on purpose.

The next pieces in this series follow the cracks further out.

• What happens when you keep adding agents to a system that already has too many.

• Why an agent can pass every evaluation and still fail in production.

• What happens when an agent has all the right information and still lacks the context to use it.

If you take one line into your next design review, take this one: the boundary between two agents is a contract, and an untested contract is a production incident with a delay on it.

See you in Part 3.

One request:

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions. Tell me what you want to learn.

👉 Take the survey here

Thanks for reading,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

One of my goals with this newsletter is to make every issue genuinely useful for you. Rather than guessing what you’d like to read, I’ve put together a short survey to understand the topics, challenges, and formats that would be most valuable to you.

It only takes 30 seconds to complete, and your feedback will directly influence the future content of this newsletter. If there’s a topic you’ve been struggling with or would like me to cover, this is the best way to let me know.

👉 Take the survey here

Thank you, it genuinely helps me write better content for this community.

Over the past year, I’ve had countless conversations with teams building AI agents.

The discussion almost always starts the same way.

“Which model should we use?”

GPT-5? Claude? Gemini? An open-source model?

It’s a reasonable question. But it’s rarely the one that determines whether the project succeeds.

In production, AI agents usually don’t fail because the LLM isn’t intelligent enough.

They fail because the data infrastructure wasn’t designed for agents.

Model problem is an illusion, mostly

When an agent gives a poor answer, it’s tempting to blame the model.

In reality, the root cause is often somewhere else.

• The agent couldn’t find the right information.

• The retrieved context was outdated.

• The metadata was incomplete.

• Permissions prevented access to critical documents.

• Tool outputs were inconsistent.

• There was no reliable way to evaluate whether the response was correct.

Changing the model may improve the symptoms.

But it doesn’t fix the underlying system.

AI agents are data systems

Traditional software follows predefined logic.

LLM applications generate responses.

AI agents do something more demanding.

They continuously retrieve context, invoke tools, maintain memory, reason across multiple steps, and make decisions based on dynamic information.

That creates different engineering challenge.

The bottleneck is no longer model capability. The models are great.

It’s the quality of the surrounding data system.

Four infrastructure questions you should answer

Before choosing an LLM, I believe every architecture review should answer these four questions.

1. How does the agent acquire context?

• Where does information come from?

• Structured databases?

• Documents?

• Knowledge graphs?

• Streaming events?

• How is relevance determined?

• How is freshness maintained?

2. How does the agent remember?

   Memory isn’t just conversation history. You need to define:

• session memory

• long-term memory

• organisational knowledge

• user-specific context

• retention and expiry policies

Memory is a data architecture problem long before it’s an AI problem.

3. How does the agent interact with enterprise systems?

Most production agents spend more time calling APIs than generating text.

• Can tools be discovered?

• Are they authenticated?

• Are outputs standardised?

• Can failures be retried?

• How are side effects controlled?

These questions often determine reliability more than model quality.

5. How will success be measured?

This is the question I see missed most often.

Teams build agents first. Evaluation comes later.

Instead, define success before implementation.

• What does a correct response look like?

• What datasets will validate it?

• What metrics matter?

  • Task completion

  • Tool accuracy

  • Groundedness

  • Latency

  • Cost

  • User satisfaction

If you can’t measure quality, you can’t improve it.

Your role is changing

The next generation of engineers will be distinguished by designing systems that give any capable model the right information, at the right time, with the right controls.

Models will continue to improve.

The system around it remains the long-term differentiator.

That’s where architectural decisions create lasting advantage.

Want to go deeper?

This article only scratches the surface.

Designing data infrastructure for AI agents is a discipline in its own right, covering topics such as context engineering, memory architectures, retrieval strategies, evaluation frameworks, tool integration, governance, and observability. These are the decisions that determine whether an agent works reliably in production.

If you’re a software engineer, data engineer, platform engineer, or architect looking to build production-grade AI systems, I’ve partnered with Packt to deliver a hands-on workshop on Designing Data Infrastructure for AI Agents.

Rather than focusing on prompt engineering or model comparisons, we’ll work through the architectural patterns, design principles, and practical exercises needed to build agent systems that are scalable, reliable, and maintainable.

👉 [Register for the Packt Workshop]

I hope to see some of you there.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Six months ago, AgentBuild was a few hundred people. Today, it’s 1100+, spread across 73 countries - from Bangalore to Boston to Bradwell.

I want to index on proving value over increasing subscriber count.

Hence, I’m going to ask for your help.

First, the Growth Curve

A year ago, AgentBuild had roughly 300 followers. Today it’s 1,158 - nearly a 4x jump, with the steepest acceleration happening in just the last four months. Something clicked around April, right when I doubled down on decision traces, agent registries, and evaluation frameworks instead of chasing whatever AI news cycle was trending that week.

Where you’re all reading from:

• India: 37%

• United States: 24%

• United Kingdom: 7%

• Germany: 3%

• Canada: 2%

• Plus pockets across 68 other countries

That’s a genuinely global room of practitioners, which still blows my mind every time I look at it.

Where I stand today

My last 20 posts averaged an open rate between 22.5% and 30.3%, with a 30-day rolling average of 25.24%.

Let’s put that in context:

• General email marketing average: ~21-22% open rate

• Substack’s platform-wide headline number: often quoted at 35 - 37%

• Real-world compressed range for niche newsletters in 2025-2026: closer to 28-32%

So where does that put AgentBuild?

Comfortably ahead of generic marketing email, but trailing the Substack averages. I call it “solid, not spectacular.” And that’s fine.

My best-performing piece, “Why Solution Architects Are the Real Force Behind Enterprise AI Transformation,” hit a 30.28% open rate - proof that when the topic lands, the number moves. Which is exactly why I want your input on what topics actually land for you.

My most-read piece by a wide margin: “Why Agent-to-Agent Communication Fails - How to Design for Failure” pulled 897 views, nearly double my typical post.

Close behind: my two-part series on building an AI agent registry (586 and 476 views), and “The Evaluation Graph: Why Your AI Pipelines Are Lying to You” (466 views).

There’s a clear pattern here - you want tactical, failure-mode-focused content over broad conceptual overviews.

Take the survey, please

If I’m grading myself against the benchmarks: good, not great. I’m beating the average email newsletter by a healthy margin, but I’m not yet in “50%+ open rate, readers treat this like a must-open” territory that the best niche Substacks report.

There’s a room to close that gap and the fastest way to close it is writing more of what you actually want, instead of guessing.

Hence, I’ve put together a short survey to answer three things:

• what topics you want more of,

• what format works best for you,

• which of the three community features should I launch

👉 Take the survey here

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions.

What I’m Building Next

Based on the feedback I’ve already gathered, three things kept coming up as the most wanted next step for this community:

1. Ask Me Anything sessions - live, unscripted Q&A where you bring your hardest production AI problems and I answer in real time.

2. Virtual meetups - smaller group calls connecting practitioners across India, the US, UK, and everywhere else you’re reading from.

3. A reader community forum - a space to swap notes and debate frameworks between issues, not just consume them passively.

I’m committing to build all three.

The only question left is sequencing - which one should I do first?

Tell me in the survey above.

Thank you for being part of the community. I am focusing on delivering value to each of you. Feel free to reach out if you need any help.

Talk soon,
Sandi

👉 Take the survey here

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Palantir built a business on this word. Every catalogue vendor has quietly rebadged its glossary as a semantic layer. Features tagged with the word “Ontology“ are being released.

Underneath the marketing there’s a real engineering shift, and it’s worth separating the two.

Subscribe now

Why Ontology matters for enterprise AI?

An LLM reads your data. It doesn’t understand your business, and nothing in your stack is currently telling it the difference.

Ask three systems what a customer is. Sales counts anyone who signed. Support counts anyone with a login, which sweeps in two people who never bought. Finance only counts accounts that have paid an invoice. The gap between those definitions is real money, and the only place it’s written down is in the head of the employees.

So when someone asks the AI how many customers you have, retrieval pulls all three definitions into the same context and the model hands back one number. It won’t mention that it merged three incompatible populations to get there, and you’ve got nothing to check it against.

A glossary won’t save you, because a glossary is written for people to read, not for a pipeline to enforce. A tidier schema won’t either. A foreign key can say two tables are related; it can’t say what the relationship means. That missing layer is what an ontology holds, in a form the machine reads at query time rather than a form a human reads in a wiki.

It becomes even more serious in the write path. A chatbot that miscounts customers gives a wrong answer. An agent that miscounts and can act on it does something about it, mails the wrong segment or moves the wrong forecast. That’s roughly when the word “ontology“ started coming up in architecture reviews.

What is an Ontology?

The textbook definition is Gruber’s:

an ontology is a formal, explicit specification of a shared conceptualisation.

Let’s look at the four words:

• A conceptualisation is the abstract model of what exists in a domain. It’s already in everyone’s head, whether written down or not.

• Explicit means it’s written down.

• Formal means a machine can act on it, not just a person read it.

• Shared means it’s agreed and owned across a community, not one architect’s private diagram.

Mechanically, you write statements as triples: subject, predicate, object.

Here’s a fragment in Turtle:

:Customer    a  owl:Class .
:PayingCustomer  rdfs:subClassOf  :Customer .
:placed      a  owl:ObjectProperty ;
    rdfs:domain  :Customer ;
    rdfs:range   :Order .

The last block says something a foreign key cannot: placed only ever runs from a customer to an order. Assert that some account placed an order, and a reasoner infers the account is a customer. Nobody typed that fact. It follows.

Then you query the meaning instead of the tables:

sparql

SELECT ?customer WHERE {
  ?customer  a        :PayingCustomer ;
             :placed  ?order .
}

This returns paying customers across every source system feeding the graph, because the definition of “paying customer” lives in one place now instead of three. That’s the line between a schema and an ontology.

A schema constrains data for one application. An ontology models a domain for many, and it entails things you never wrote down.

How to actually build one

This is very new for most companies. There is no fromal playbook - but I can see a number of guidances coming out. There are many issues I see with ontology imlementation. Usually a modelling team would take nine months to produce two thousand classes, and nobody uses it. That’s costly.

Given what I have learnt so far, I can roughly put some thoughts on the implementation, already working ins ome enteprrises:

1. Start with competency questions. Write the twenty questions the ontology must answer, in business language, before you model anything. “Which counterparties have aggregate credit exposure above £50m across all subsidiaries?” If your model answers it, that part is done. If you can’t write the questions, you don’t have a use case, you have a vocabulary project.

2. Reuse before you build. FIBO already models counterparty, legal entity, and financial instrument, with a decade of EDM Council work behind it. SNOMED CT has done clinical terms. Your novel IP is the last 10% specific to your firm, not the definition of a legal entity. Reuses existing artefacts.

3. Treat it as software. Ontologies rot faster than code because nobody owns them. Put the files in Git. Write SHACL shapes as your tests and run them in CI, so a change that breaks a downstream contract fails the build rather than a quarterly report. Give every module a named business owner who signs off changes, and version it properly.

4. Ship thin vertical slices tied to one workflow. One agent, one domain, one set of competency questions, in production. The organisational fight you’re picking is agreement on what words mean, and that fight is much easier to win on a small surface with a working demo behind it.

The takeaway

The ontology conversation isn’t a semantic web revival. It’s what happens when systems start acting on their own and the industry discovers it never wrote down what anything means.

I went deep on this with Jessica Talisman, who has spent her career on taxonomy and ontology work at the sharp end of large organisations. We got into where taxonomies stop and ontologies start, what breaks at enterprise scale, and why most of the failures are organisational rather than technical. Spend sometime this weeked, watch it, and let me know how you found it.

Hope you enjoy it.
Happy Weekend.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.



More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

Subscribe now

If you are buidling with coding agents, you have surely faced this problem. You ask the coding agent to build a feature, it writes the code, writes the tests, and reports success. However, you still find bug was there the whole time. The tests didn’t catch it. This is a common problem and in this article I will share what works for me most of the time.

Why this happens

Think about what a test is actually for. A test is a second opinion. You write code that you think is right, and you write a test that checks whether it really is. The value comes from those two things being separate. The code is your attempt. The test is the check on your attempt. If they disagree, you’ve caught something.

Now here’s what goes wrong with AI.

When you ask an AI to write the code and the tests in one go, you lose the separation. This creates a correlated failure or model bias propagation. Because the LLM shares a “latent space” between the test and the implementation, it is statistically likely to produce tests that validate its own mistakes-the model simply predicts the most probable test code given the most probable implementation code it just generated. The code has the bug, and the test is written to expect the buggy behaviour.

Consider a simple shopping cart: you ask the AI to calculate the total price, including a 10% tax. The AI rushes the job and writes a calculation function that accidentally skips the tax entirely. Then, it immediately pivots to writing the test. Because the AI is still “thinking” about the buggy code it just wrote, it creates a test that expects the total without the tax.

Now, you have a buggy function that ignores tax, and a test that celebrates that exact result. The test doesn’t flag the error because it wasn’t designed to check for the correct business requirement, it was designed to match the code’s current, buggy behavior. Everything goes green, even though the feature is fundamentally broken.

The test didn’t fail you because it was a bad test. It failed you because it was written by the same thing that wrote the bug. It’s marking its own homework.

And it gets worse at scale. One developer writing careless tests writes a few. An AI writes dozens in seconds - all neat, all passing, all quietly checking for the wrong thing.

How I fix this problem

The fix is to put the separation. You want the tests written by something that has not seen the code and cannot change it. That’s the whole idea. Everything below is just how to set that up.

If you’re using Claude Code, you do this with something called a sub-agent. A sub-agent is just a helper with a specific job and its own limited set of permissions. You create a sub-agent by adding a small text file to your project or from the terminal on Claude code.

In this case, you have to make two sub-agents:

• one that only writes tests

• one that only writes code

The trick is in what each one is allowed to touch.

If you aren’t using an agentic framework like Claude Code, the core concept remains the same: force a “Contextual Firewall” between your test-writing process and your implementation process. You don’t need a specific tool; you need a workflow that treats them as separate entities.

Let’s see how to do this step-by-step.

Step 1 - Create the tester, and don’t let it write code

Add a file called test-author.md in a folder named .claude/agents/ in your project. Put this inside it:

---
name: test-author
tools: Read, Write
model: sonnet
---
Write tests based on the requirement I give you.
Do not look at the existing code first.
Write tests for what the feature is supposed to do.

The important line is tools: Read, Write. This lets the tester read the requirement and write a test file - but it does not give it permission to change any existing code.

Conversely, the Coder will need read access to these tests to understand the criteria it must pass, but restricting its write access is crucial to prevent it from “hacking” the tests to suit its own buggy implementation.

Step 2 - Create the coder, and don’t let it touch the tests

Add a second file, implementer.md, in the same folder:

---
name: implementer
tools: Read, Edit, Bash
model: opus
---
Make the failing tests pass by changing the code.
Never edit a test file. Never delete a test.
Keep fixing the code and re-running until the tests pass.

This one is the opposite. It can edit code and run the tests (Bash lets it run them), but it’s told never to touch a test file. So it can’t cheat by weakening a test that’s in its way. Its only route to green is to actually fix the code.

Note: using different models can help, but not always - especially within the same model family, where they tend to share the same blind spots. The real independence comes from isolation and tool limits, not the model choice.

Step 3 - Write the tests first, and read them yourself

This step matters the most.

Give the tester the requirement - in plain English, what the feature should do. Let it write the tests before any code exists. Then, perform a Senior Engineer Review.

Position the AI as a Junior Developer generating boilerplate; your job as the Senior Reviewer is to validate the “contract” of the feature. You don’t need to write the code, but you must ensure the tests are asking for the right thing. The AI does the typing; you own the architectural judgement.

Don’t try to automate this part. At least not for critical code.

In the shopping cart example, you’d check: is there a test confirming the total includes tax? If yes, you’re safe. If the code later forgets the tax, that test will fail, and the bug gets caught right there.

Step 4 - Let the coder make them pass

Now hand it to the coder. It writes the actual code, runs the tests, sees what fails, fixes it, runs again - until everything’s green.

But this time, green means something. The tests it’s passing were written by something that never saw the code and can’t be edited by the thing writing the code. The two are genuinely separate again. The second opinion is back.

Run through the cart example one more time. The tester, working only from your requirement, writes a test that says the total must include tax. The coder writes the function. If it forgets the tax, the test goes red, and the coder has to fix it. The bug that sailed through before now can’t get past step 4 - because the thing checking for it was never allowed to see the mistake in the first place.

Reminder

This fixes a specific, common problem: the AI quietly agreeing with its own mistakes. It’s a real fix and it’s worth doing.

What it doesn’t fix is you asking for the wrong thing. If the requirement you gave the tester was wrong, you’ll get a perfectly tested version of the wrong feature. The tests can only be as right as the requirement behind them. Checking that the requirement matches what the business actually needs is your job - you need to do it.

Takeaway

If you’re letting an AI write your code and your tests together, and trusting the green tick, be careful. That green doesn’t mean what you think it means. It means the code and the tests agree - and since the same AI wrote both, of course they agree.

Do these four things instead:

1. Make a tester that can write tests but can’t change code.

2. Make a coder that can change code but can’t touch tests.

3. Write the tests first and read them yourself.

4. Let the coder make them pass.

Separate the two, and the tests start protecting you again - which is what they were always supposed to do.

I love diving deep into these topics

Some weekends I spend time with teams talking about coding agents - from the basics right through to building production-grade applications with them. Last weekend I did this with a team at a software consultancy. It was fun, and we had such a valuable exchange of ideas.

This is the best time I get to spend - speaking to people who are genuinly interested in building real things with AI, not slops.

Thanks for being a part of this community.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

So the newsletter crossed 1,000 subscribers this week.

1,000 is nothing in newsletter world. People do that in a weekend with one lucky post. I know that. But those aren’t the thousand I got.

The thousand I got are people building AI inside banks, insurers, healthcare companies, startups, consultancies. People who reply to my emails saying “we hit exactly this last quarter.” That’s a different thousand. I’d take it over 50,000 randoms any day.

You give me space in your busy inbox - that matters a lot to me.

Thank you for that.

Subscribe now

Why I started the newsletter?

Everyone’s shouting about AI. Demos everywhere, announcements everywhere, everyone’s an expert on LinkedIn by Tuesday. But almost nobody talks about the boring part - what happens six months after the demo, when the thing’s in production, the data’s messy, the regulator’s asking questions, and someone senior wants to know why the numbers moved.

That part. The unglamorous part. That’s the whole newsletter. Because that’s where I’ve spent my career, and honestly, that’s where yours gets made too.

So nothing changes at 1,000. I’ll keep writing about the stuff that actually breaks - evaluation, data debt, decisions nobody wrote down. I’ll keep sharing the failures, mine included, because that’s where the lessons are. And I’ll keep reading every single reply, because half my best issues started as one of you saying “we’re seeing something weird and nobody’s writing about it.”

Keep sending me those. Seriously.

One more thing. I’m not going to take up space in your inbox just to celebrate - that’s not what you signed up for.

So here’s something useful. I just published a video on why most AI projects fail. It’s everything I’ve learnt working with enterprises over the past few years - the patterns I keep seeing, and what I think actually makes these projects succeed.

Watch it here.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

So when your agent starts giving wrong answers at scale don’t look for the failure in the model. It’s in the memory system you built around it. And if you don’t understand the memory types and how they behave differently under load, you’ll spend weeks debugging the wrong layer. I have seen this happen multiple times.

Why Memory Architecture Is the Real Work

Consider what happens in a real production deployment at a financial institution. An agent handles customer queries across thousands of sessions per day. It needs to know what the customer asked last week. It needs to know current product terms. It needs to know what it did three steps ago in the current task. It needs to know that when a customer says “the same account as before,” they mean their savings, not their current account.

These are four completely different information retrieval problems. Treating them as one (which is what happens when teams reach for a vector database and call it done) is where agents fall apart in production.

Each of these problems maps to a distinct memory type. Each type has different latency characteristics, different staleness risks, different failure modes, and different architectural requirements.

The Four Types of Agent Memory

1. In-Context Memory (Working Memory)

This is whatever sits inside the current context window - the active prompt, the conversation so far, the instructions, the retrieved chunks, the tool outputs. It’s the only memory the model can directly reason over.

• What it’s good for: The current task. Right now. This session.

• Constraint: Context windows are finite. GPT-4o gives you 128K tokens. Claude gives you 200K. They sound large until you’re injecting retrieved documents, conversation history, system instructions, and tool schemas simultaneously. You will always burn through context faster than you would expect.

• Failure pattern: Teams usually design for a 10-turn conversation in testing. Real users run 60-turn sessions. Context fills up, older turns get truncated, and the agent loses track of earlier constraints the user set. The agent doesn’t tell the user it’s forgotten. It just starts behaving inconsistently. And that’s dangerous.

2. Episodic Memory (What Happened)

This is the log of past interactions - previous conversations, completed tasks, decisions made, outcomes observed. It’s stored externally and retrieved when relevant.

• What it’s good for: Cross-session continuity. “Last time we spoke, you were working on the credit risk model.” Personalisation. Audit trails in regulated environments.

• Implementation: Conversation summaries or raw turns get embedded and stored in a vector database. At the start of each new session, a semantic search retrieves the most relevant past episodes and injects them into context.

• Failure Pattern: Episodic memory grows without bound. A user with 200 past sessions generates thousands of stored chunks. Retrieval latency climbs. Relevance degrades because older episodes are returned alongside recent ones. Teams don’t notice until p99 latency starts causing timeout errors in production.

3. Semantic Memory (What Is Known)

This is factual, reference knowledge - product documentation, policy documents, regulatory guidance, knowledge base articles. It doesn’t change per user. It’s the shared ground truth the agent reasons from.

• What it’s good for: Answering questions from authoritative sources. Think everything about the business. Grounding the agent in facts rather than hallucination. RAG (Retrieval Augmented Generation) is almost always semantic memory.

• Implementation: Documents are chunked, embedded, and indexed. Queries are embedded and matched via similarity search. Top-K chunks are injected into context.

• Failure Pattern: Source documents change, the index doesn’t (this is very common). Nobody set up a re-indexing trigger. The agent retrieves a policy document from eight months ago with full confidence, because from the vector database’s perspective it’s still the most semantically similar result. In a regulated environment, this isn’t a UX problem. It’s a compliance problem with vulnerabilities that can cause huge penalties.

4. Procedural Memory (How to Act)

This is the agent’s knowledge of how to behave - which tools to call, in what order, under what conditions. It’s not retrieved at runtime in the same way the other types are. It’s encoded in the system prompt, in tool schemas, in fine-tuned weights, or in explicit workflow definitions.

• What it’s good for: Consistent behaviour across all users and sessions. If you want the agent to always verify identity before accessing account data, that’s procedural memory.

• Failure Pattern: Procedural memory encoded only in system prompts is brittle. Long system prompts get partially ignored as context fills. Behaviour drift emerges - the agent follows the procedure at the start of a session but deviates by turn 40. Teams don’t catch this because they test short sessions.

How These Four Types Behave Differently at Scale

At low volume , say hundreds of sessions per day, all four types can be bolted onto a single vector database and it mostly works. At production scale, i.e., tens of thousands of sessions, millions of stored episodes, documents updating continuously, each type starts failing in a distinct way.

In-context memory degrades with session length. The longer the conversation, the more context is consumed, and the more the model has to work with a truncated window. At scale, you need a context management strategy: summarise older turns, compress tool outputs, prioritise what stays in window.

Episodic memory degrades with corpus size. Retrieval latency grows as the episode store fills. Relevance drops as more episodes compete for the same top-K slots. At scale, you need a tiered storage strategy: hot storage for recent episodes (last 30 days), cold storage for older ones, with a recency-weighted retrieval model rather than pure semantic similarity.

Semantic memory degrades with document churn. The faster your source documents update, the faster your index goes stale. At scale, you need event-driven re-indexing triggered by document changes, not scheduled batch re-indexing that runs weekly and misses the update that happened on Tuesday.

Procedural memory degrades with task complexity. Simple, short tasks follow procedures reliably. Long, multi-step tasks in complex domains show drift. At scale, you need procedures encoded as explicit workflow graphs - not just prose in a system prompt - with checkpoints that verify the agent is still on the correct path.

Architecture Choices and Optimisation Strategies

For in-context memory:
A common pattern is to use a sliding window with summarisation. Keep the last N turns verbatim, summarise everything older into a compressed context block. If you use Claude Code - you will notice it does that in long conversations. It compresses conversations at one point. (Yes, Claude Code is an agent).

LLM-generated summaries work well here. For long-running agents, implement a context budget - allocate token limits per component (system instructions, retrieved memory, conversation history, tool outputs) and enforce them at the orchestration layer, not as an afterthought.

For episodic memory:
Separate your recent episode store from your historical episode store. Redis or a fast key-value store handles recent sessions; your vector database handles historical retrieval. I love Databricks LakeBase architecture here - LakeBase can act as the transactional store while the historical data is stored in Delta Lake. (I know Databricks plug, but it is usful to know if you are already using Databricks).

Apply recency decay to your similarity scores - an episode from yesterday should score higher than a semantically identical episode from six months ago, because context drifts. Build a summarisation pipeline that condenses completed sessions before storage, rather than storing raw turns.

For semantic memory:
Your chunking strategy is your most important decision - more important than your choice of vector database. Chunks need to be semantically coherent, not just fixed-size token windows. 512 tokens with sentence boundary respect and document-level metadata outperforms 2048-token sliding windows without metadata. Build a document change detection pipeline that triggers re-embedding on update. Track embedding model versions - if you upgrade your embedding model, you need to re-embed your entire corpus, because old and new vectors aren’t comparable.

For procedural memory:
Move away from text prose system prompts for complex procedures. Define agent behaviour as a state machine or directed graph - explicit states, explicit transitions, explicit conditions. This gives you auditability (you can log which state the agent was in at each step), testability (you can write unit tests for state transitions), and recoverability (you can restart a failed task from a known state rather than from the beginning).

You know what’s common across all four?
Each memory type needs its own evaluation pipeline.

Not just evaluation of the agent’s final output - evaluation of the memory layer itself. Did episodic retrieval return the right sessions? Did semantic retrieval return current documents? Did the agent follow the correct procedure? Did context management preserve the right information?

They need to be measured separately. And at production scale, not doing this could prove expensive, and risk reputational damage.

Now, tell me what kind of memory failure patterns you observed in your implementations. Give me feedback - tell me how this article helps you. I want to hear more from you.

If you love listening to experts, this podcast with Denis Rothman, author of Context Engineering for Multi-Agent Systems is very relevant. Denis is a thoughtleader and he describes complex topic in simple human way. You will like it.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

The fix for this isn’t a better model log. It’s a different structure that sits alongside it - what I’ve been calling an evaluation graph. Every time the AI runs, it takes a hashed, timestamped snapshot of every governance artefact in force at that exact moment: the glossary term version, the policy version, the dataset certification. So six months later, when the definition has moved from version 2.3 to 2.4, you’re not reconstructing anything. You look it up. The snapshot tells you exactly which soup the kitchen made that day, and who signed off on the recipe.

Evaluation Graph vs Context Graph

People keep confusing this with a context graph, which is genuinely a different thing. What the difference?

The evaluation graph is the receipt that proves what was actually served. It records: today’s menu version, which supplier the tomatoes came from, which health certification was active, what time the dish left the kitchen. Six months later, if someone asks “was this dish safe and compliant on that day,” you don’t guess, you pull the receipt and know exactly which version of “today’s soup” this was, signed off by whom. That’s provenance. It’s not about why the waiter brought you soup. It’s about proving what was true in the kitchen at that exact moment.

The context graph is the waiter’s reasoning for why you got soup at all. Maybe you said “I’m not very hungry,” the waiter remembered you’re allergic to shellfish from last visit, saw the bisque was off the board today anyway, and recommended the soup as the lightest, safest option. That whole chain - your stated preference, the remembered allergy, the menu constraint, the final recommendation is the decision lineage. If you later ask “why did you suggest soup,” the context graph is what lets the restaurant reconstruct that reasoning instead of shrugging and saying “the waiter just did.”

So: same dish, two completely different questions.

• Evaluation graph: “Prove what was actually in the bowl, and that it met standards, on that specific day.” (Provenance, was the input governed correctly?)

• Context graph: “Explain why the waiter chose this for you specifically.” (Lineage, why did the agent decide what it decided?)

A restaurant or a bank can have perfect receipts and a clueless waiter. Or a brilliant waiter with no receipts to back up a regulator's question. You need both, and they're solving for different failure modes.

What this looks like technically

Strip away the restaurant for a moment. Here’s the conceptual shape of both graphs, and the one thing that ties them together.

A context graph is built from three kinds of nodes.

• Entities: the customers, products, policies, and people your business already has some version of in a knowledge graph.

• Decision events: each time an agent or a human makes a choice, that choice becomes its own node, carrying the inputs, the policy applied, any exception made, and the outcome.

• Temporal state: because the graph needs to know not just what’s true now, but what was true at the moment the decision happened. Sarah was a Director in May; she’s a VP today. The graph needs to know which one mattered when.

An evaluation graph is built differently, because it’s answering a different question. Its core node is a context snapshot taken at the exact moment a model runs, hashed and timestamped, pointing to the specific versions of every governance artefact in force at that instant: the glossary definition, the policy version, the dataset certification. Alongside that sits the run itself - the model version, the prompt template, the retrieval configuration, and a failure taxonomy that lets you diagnose, after the fact, whether a wrong answer was a model failure, a context failure, or a policy gap.

The two graphs aren’t separate systems sitting in isolation. They connect through one shared reference: the run, or the decision event itself. Every decision event in the context graph that involved a model call can carry a pointer to the evaluation graph’s snapshot for that exact run.

When someone asks “why did the agent decide this,” you walk the context graph. When someone asks “what was governing the system when it decided that,” you follow the same run ID into the evaluation graph and pull the snapshot. One graph for the reasoning, one graph for the proof, joined at the single point where the AI actually executed.

Practically, this doesn’t require building either graph from scratch on a green field. If you’re already running something like MLflow or Databricks AI Agent Evaluation, the run node is just a thin wrapper around the run ID your platform already generates. The context snapshot capture can be a pre-run hook - a lightweight call to your catalog and glossary APIs that resolves the current versions of everything relevant, hashes them, and writes the snapshot before the model executes. The heavier lift is the context graph side, because that one has to live inside the execution path - you can’t capture a decision event you weren’t watching for when it happened.

What I’d actually do about it

These concepts are new and pretty early for many organisations. My advise is to be congnizant and ask questions for every AI build that passes through the architecture team. So, before your next model goes anywhere near a regulated workflow, ask one question out loud in the project meeting: if a regulator asks us to reproduce this exact answer in six months, what’s allowed to change underneath us between now and then?

Then write down every governed artefact that answer depends on. Glossary terms. Policy versions. Dataset certifications. For each one, name who owns it and how often it moves. If you can’t answer that for every item on the list, you don’t have an evaluation graph yet - you have a model log dressed up as proof.

That’s not a backlog item to schedule for next quarter. It’s the difference between “we can rerun the query” and “we can actually explain the decision.”

Where to go next

I’d genuinely like to know what you’re seeing on your side. Has anyone asked your team to reproduce a decision and found the same gap I’m describing? Is this something your governance and AI teams have already wired together, or is it still sitting in the gap between two calendars like it was at the bank above?

Hit reply or drop a comment. Tell me about the patterns you’re seeing, they usually end up shaping what I write next.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

A quick note before you watch: this was recorded in April 2026, and the pace of change in this space means a few things have already moved on. Databricks has since shipped more platform features that make some of these pillars easier to implement than I describe in the talk. My own thinking keeps shifting too, as I talk to more customers and fold new patterns into the framework. I’ll keep sharing those updates here and on the YouTube channel as they land.

Here’s what it covers.

The talk is called “The Production AI Playbook” and the core argument is this: most AI projects fail for the same five reasons, and almost none of them are about the model.

I open with a pattern most of you will recognise: the Week 1 to 14 doom loop. Teams pick a model, build features, demo to leadership, ship, and watch it fall apart within weeks because nobody built the infrastructure underneath it. Gartner puts enterprise AI project failure above 40%. I’ve watched it happen first-hand, more than once.

From there I walk through five pillars, each one a dependency for the next:
Evaluation,
Observability,
Data Foundation,
Orchestration,
Governance.

Two pillars get particular depth. On evaluation, I show the actual judge prompt structure we use for LLM-as-judge scoring, including the fix for non-determinism: running each test case three times and flagging anything with high score variance before it ships. On orchestration, I cover the patterns that hold up in production against the ones that only survive in demos, including failure modes nobody puts on slides: context window bleed, cascading failures, and trust boundary violations between agents.

The spine of the talk is a real case study: a retail bank, 18,000 calls a month, a prior attempt that burned $85,000 over six months with nothing shipped. The second attempt took eight weeks, and the model wasn’t chosen until week seven. Everything before that was evaluation and infrastructure. The result: 87% accuracy, 62% call deflection, and a tripled API-call bug caught in two hours that would otherwise have cost $43,200 a year in wasted fees.

I also include a section on what I’d do differently. Three things that surprised us even after the framework held up: the test case library needs a named owner or it quietly rots, prompt version logs need to capture intent and not just the diff, and behavioural evals cost far more to run at scale than most teams budget for.

If you’re building or running production AI, this one will be useful.

Here’s in the link to the resources I talk about in the video.

I aslo did an online talk in the same conference on Multi-Agent Orchestration Patterns, more tehcnical deep dive into patterns and failures modes when you build agentic workflows.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

The model did its job. What failed was everything around the model. The context. And that’s the shift worth getting your head around: the intelligence is largely a solved problem now, and the bottleneck has moved to context.

Prukalpa Sankar, Founder and Co-CEO of Atlan, puts it bluntly: with AI, context might be everything, because “the intelligence is already here.” The model is the easy part. The hard, durable, defensible part is whether your agents understand how your business actually works. That understanding is what people are starting to call context intelligence.

What context intelligence actually is

It isn’t a bigger prompt or a better retrieval setup. It’s the infrastructure that gives an agent shared, governed, current knowledge of your organisation, plus a memory of the decisions it and its predecessors have made. Two ingredients: a map of how things relate, and a record of why things happened.

That map and that record are the context graph. A context graph is a living model of your business as a set of entities and the relationships between them, this customer, that contract, this SLA, that exception, joined to the decisions taken against them. Foundation Capital, in their Context Graphs essay by Jaya Gupta and Ashu Garg, made the sharpest version of the argument: the durable asset isn’t the data an agent reads, it’s the decision trace it leaves behind. What was gathered, what rule applied, why an action was allowed. Capture that, and precedent becomes something an agent can look up instead of guess at. The agent stops having data with no judgement and starts having judgement.

Why this is the accuracy story

Most enterprise agents fail on a trust gap, not a model gap. The SLA agent didn’t need a cleverer model. It needed to know which contract was current, that it was allowed to act on it, and what had been decided in similar cases before. None of that lives in the model. All of it lives in the context.

Without it, you get what Prukalpa calls context sprawl: every agent building its own private, partial view of the world, none of them agreeing on what “active customer” even means. Fifty agents, fifty versions of the truth, no shared map. Accuracy in that environment isn’t a model property. It’s an infrastructure property.

This is where the runtime and the infrastructure get confused. Anthropic’s work on context engineering covers the runtime half: context is finite, and the discipline is fitting in the fewest high-value tokens at the moment of inference, not the most. That’s real, but it assumes the right context already exists to be selected. Context intelligence is the layer below it, the one that decides what context exists, whether it can be trusted, and whether the agent may use it. One is what you put in the window. The other is what’s available to put there at all.

How to think about building it

You can read most failures here as three debts coming due.

• Data Debt: no single governed source of what’s true, so your agents disagree.

• Decision Debt: nobody captured why past actions were taken, so the context graph has no memory.

• Evaluation Debt: no framework to check whether the context an agent actually used was the right context. Work out which one is biting and you know where to start.

The build order matters more than the architecture diagram. The instinct is to spend two years plumbing every system into a perfect context layer before anything ships. Prukalpa’s advice is the opposite, and it’s right: bootstrap from the systems you already have, the CRM, the ERP, the BI definitions, get the context layer roughly 80% of the way there, and let the flywheel start turning. Every decision an agent makes then becomes institutional memory the next agent inherits. You don’t design the context graph up front. You grow it.

Which is why this isn’t really an AI problem at heart. It’s the next turn of data engineering: context as a governed product, with owners, versions, and tests, sitting between your data and your agents. The teams who treated data as a product a decade ago have a head start. The ones still treating context as something you cram into a prompt are about to learn the difference in production.

The open question, the one nobody at the table has a clean answer to yet, is who inside the enterprise actually owns this layer.

I got into exactly that with Prukalpa Sankar on the podcast. She’s been arguing for the context layer longer than almost anyone, and it’s the clearest thinking I’ve heard on where this is heading.

Also, read Prukalpa’s article - What an Enterprise Context Layer Actually Is

Enjoy your weekend.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Part 1 covered Schema and Ownership: the description and accountability layer of an Agent Registry. Unity Catalog as the agent registry. AWS IAM for per-agent identity. CloudTrail for audit trails. A stack that answers the questions an audit demands: what exists, who owns it, what it can do.

That is necessary, but it is not sufficient.

Schema and Ownership describe an agent. They do not constrain it. An agent with a well-documented schema and a clear owner can still make a consequential decision without human approval, access data it should not touch, or operate at an autonomy level the business never signed off on. That is where Risk Class and Autonomy Level come in and where the registry becomes a governance control rather than a filing system.

Component 3: Risk Class - Not All Agents Carry Equal Risk

Risk classification is the decision that determines what governance an agent receives. If you get it wrong and you either over-govern low-risk agents until teams route around the process, or under-govern high-risk agents until something goes wrong in production.

The four risk categories worth working with in a regulated environment:

1. Low Risk: bounded output, read-only, no consequential action. An agent that summarises documents or answers questions from a knowledge base.

2. Medium Risk: supervised action. The agent can write or update records, but a human reviews before execution.

3. High Risk: consequential action inside a tightly constrained boundary. The agent can execute without per-action approval, but only against approved tools, approved data paths, and fully audited systems.

4. Critical Risk: autonomous consequential action across multiple systems, with downstream effects that are difficult or impossible to reverse.

In Databricks, risk class is best treated as governed metadata attached to the agent assets you register. For example, on Unity Catalog models, model versions, and functions, all of which support tagging. That makes risk class visible, queryable, and reviewable across the lifecycle. If an agent gains new tools, new data access, or a broader execution boundary, the classification should be reassessed before the next promotion.

So, if you ar eon Databricks, use Unity Catalog to register the agent, its model versions, its functions, and its metadata. Use tags to record risk class. Use Unity Catalog privileges, managed authentication, and deployment-time permission checks to enforce which tools and data paths are actually reachable at runtime. If you need an explicit risk-policy engine that says, for example, a High Risk agent may call Tool A but never Tool B, that policy layer still sits outside Databricks - in an API gateway, middleware layer, or external authoriser.

That is the line between registry and policy engine. Databricks gives you the governed assets, the permissions model, and the audit surface. If you need deterministic policy decisions over agent behaviour itself, you compose that on top.

On the AWS-native path: Amazon Bedrock AgentCore Gateway with Cedar policies provides a similar enforcement pattern - deterministic allow/deny decisions on every tool call, with Lambda interceptors for dynamic validation. Cedar policies are authored in a declarative language and evaluated against principal, action, and resource with optional conditions over request context. Worth monitoring as it moves towards general availability.

Component 4: Autonomy Level - How Independently an Agent Operates

Risk class tells you what an agent is allowed to do. Autonomy level tells you how much independent judgement it is permitted to exercise in doing it. These are related but distinct. A High Risk agent can still operate at L1 with human approval, or at L3 with bounded multi-step execution. The risk class constrains the action space. The autonomy level constrains the operating model.

The five levels from the registry:

• L0 - Assistive only. Generates output for human review. No execution.

• L1 - Human-approved actions. The agent proposes; a human confirms before execution.

• L2 - Semi-autonomous workflows. Executes within defined boundaries without per-action approval.

• L3 - Goal-driven execution. Plans and executes multi-step tasks. Human oversight at checkpoints, not per action.

• L4 - Multi-agent autonomy. Coordinates with other agents, spawns sub-agents, orchestrates complex workflows with minimal human intervention.

In Databricks, autonomy level should be treated as an explicit registry attribute you manage in your own governance model, not as a built-in Unity Catalog field with native enforcement semantics. The platform gives you the places to record it, such as model and model-version metadata, but the meaning of L0 through L4 remains an operating policy you define and then enforce through workflow design, approvals, deployment controls, and runtime boundaries.

MLflow Tracing captures the full execution path - every tool call, every decision point, every input and output - linked to the agent version and autonomy level at the time of execution. In a regulatory investigation, that trace is the evidence.

So, autonomy level is declared in the registry, promotion is gated through MLflow deployment workflows, runtime execution is bounded by Databricks permissions and isolated tool execution, and trace evidence is captured through MLflow. The registry records the autonomy decision; the surrounding control plane makes it real.

Open source alternative: Microsoft released the Agent Governance Toolkit in April 2026 under MIT licence, an open source project that addresses runtime security governance for autonomous agents. It includes execution rings inspired by CPU privilege levels, kill switches for emergency agent termination, and circuit breakers. It maps directly to the OWASP Top 10 for Agentic Applications (2026) and is the most purpose-built open source option currently available for autonomy-level enforcement.

What the Registry Now Does

Across both parts, the four components work as a single control system.

Schema tells the registry what the agent is. Ownership tells it who is accountable. Risk Class tells it what level of control the agent requires, and that classification is carried as governed metadata across the agent’s registered assets. Autonomy Level tells it how much independent judgement the agent is permitted to exercise, and that decision is enforced through release workflows, runtime permissions, and execution boundaries rather than left implicit in code.

The audit request scenario from Part 1 - fourteen agents, nobody can answer, is no longer possible. Every agent has a schema record, an owner, a declared risk class, an explicit autonomy level, governed access to tools and data, and an execution trail that can be reconstructed through MLflow and platform audit logs. The registry is not a reporting artefact. It is the control plane that makes autonomous operation in a regulated environment defensible.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

An organisation deploys 50 agents across 6 teams over the course of a year. Without any central register, wihtout any ownership mapping, wihtout any shared schema.

Then the audit request arrives.

They ask the following questions:

• Which agents are live?

• Who approved the one touching customer financial data?

• What has it been told in its system prompt?
  

Nobody could answer.

This is not a technology failure. The agents worked fine but the governance infrastructure was never built.

A Note on the choice of technology

Before we process, I want to make a note on my technology choices here. The architecture in this issue is Databricks-on-AWS. That is not a neutral choice, but it reflects where I spend most of my time. I work with Tier 1 UK financial institutions, and the patterns I see repeatedly across those engagements have shaped this stack.

Databricks gives me Unity Catalog for governance, lineage, and access control across the full data and AI estate. AWS gives me IAM for identity and CloudTrail for audit trails. These tools are production-stable, work in hybrid deployments, and hold up under compliance scrutiny. Where open source alternatives exist and are worth knowing about, I have noted them inline.

Where open source alternatives exist and are worth knowing about, I have noted them inline. But the primary recommendation here is the stack I have seen hold up under real compliance scrutiny.

What an Agent Registry Actually Is

An Agent Registry is the control layer that makes an agent estate governable. It is not a dashboard. It is not a catalog you update manually. It is infrastructure, the layer that sits beneath your agents and answers the questions an audit demands: what exists, who owns it, what it can do, and how much autonomy it operates with.

It has four components. This issue covers the first two: Schema and Ownership. Part 2, out on Wednesday, covers Risk Class and Autonomy Level - the enforcement layer.

I am splitting this deliberately.

• Schema and Ownership are about description and accountability - what an agent is and who controls it.

• Risk Class and Autonomy Level are about enforcement - what an agent is allowed to do and what stops it.

These are different engineering problems. Collapsing them into one piece does neither justice.

Component 1: Schema — How an Agent Describes Itself

Schema is the agent’s self-declaration. Without it, agents are opaque. Orchestration breaks. Integration fails silently. You cannot route work to an agent you cannot describe.

A schema record needs to capture: capabilities and skills, the APIs and tools the agent can call, input and output formats, memory and context handling behaviour, access permissions, and communication protocols.

Unity Catalog (UC) is the agent registry in this stack. It governs agent tools as registered, versioned functions - each one a securable object with access control, lineage tracking, and metadata surfaced through Catalog Explorer or the REST API. It supports attribute-based access control on tags, which means you can attach metadata directly to agent capability records and build access policies around them. If you think about it, UC becomes the same governance layer that already covers your data assets, your ML models, and your pipelines, just extended to agents without adding a separate system.

Unity Catalog also works across Databricks workspaces on AWS, Azure, and GCP. If your agent estate spans cloud environments, governance travels with it through the same control plane. That’s powerful.

AWS AgentCore Agent Registry is worth knowing about as a complementary discovery layer. It stores agent records across frameworks and clouds and supports semantic search and approval workflows - and is useful if your estate includes agents running entirely outside Databricks and you need a single cross-platform catalogue. It is a discoverability tool, not a governance layer. For most Databricks-on-AWS deployments, Unity Catalog covers the registry function without it.

Open source alternative: purpose-built open source agent registry tooling is immature right now. The practical path is a lightweight service catalogue like Backstage for discoverability combined with OPA for policy enforcement - neither of which was designed for agents, but both of which work today without significant custom engineering. I have found this GitHub Repo, I never tried it or even explored it - might you might want to have a look.

If you find one or know one - let me know in comments.

Component 2: Ownership — Who Controls the Agent

Ownership is where shadow AI comes from. An agent without an owner is an agent nobody is responsible for. In regulated environments, that is not an abstract risk.

An ownership record needs to capture: the team or business owner, accountability mapping, the access governance model, the escalation path, approval authority, and lifecycle responsibility - including who decommissions the agent and when.

On the identity side, AWS IAM is the enforcement mechanism. Each agent should operate under a dedicated IAM role with least-privilege permissions - no shared credentials, no roles that accumulate access over time. IAM Policy Autopilot, allows AI coding tools to generate baseline IAM policies directly from application code, reducing the gap between what an agent was built to do and the permissions it actually holds.

Accountability trails run through AWS CloudTrail. Every registry access and administrative action is logged. In AgentCore Agent Registry, CloudTrail integration is built in, meaning you have an auditable record of who approved an agent, when it was registered, and when its record was last modified.

Unity Catalog extends this further. Audit logs capture every agent action. Unity AI Gateway release introduced MCP server governance - controlling which agents can access which external systems and tracking how that data is used. Ownership in the registry is not just a field in a database. It maps directly to the IAM role, the Unity Catalog access policy, and the audit trail.

Open source alternative: Open Policy Agent (OPA) provides policy enforcement for ownership and access governance in environments not running on the Databricks or AWS managed stack. It is cloud-agnostic and widely used in regulated industries.

If you find good open-source stack please comment here. I am looking for them as well.

Coming Wednesday: Risk Class and Autonomy Level

Schema and Ownership tell you what an agent is and who is responsible for it.

That is necessary but not sufficient.

The harder question is what an agent is allowed to do and what stops it when it operates outside its boundaries.

Part 2 covers Risk Class: how risk classification moves from a metadata tag in Unity Catalog to a live runtime control via Unity AI Gateway and AgentCore Policy. I will also cover Autonomy Level, the scale from assistive-only (L0) to multi-agent autonomy (L4), and the kill switch infrastructure that makes higher autonomy levels safe enough to deploy in a regulated environment.

This is the part of the registry most organisations skip entirely. It is also the part that determines whether your governance is real or decorative.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

Mnay organisations I talk to gathers 50-100 AI use cases. They hold a big “priority workshop.” They vote on business value, strategic fit, and stakeholder excitement. They pick the top three, hand them to the engineering team, and wait for the magic to happen.

Six months later, the project is quietly shelved.

The reason? “The data infrastructure doesn’t support it.”

In my latest video, I talk about why this happens and more importantly, how to stop it using a framework I’ve been developing for production AI.

The “Human-to-Agent” Gap

The core problem is something I call the “Human vs. Agent” data standard.

As technical people, we often forget that data built for a human analyst is completely different from data built for an AI agent.

• Humans can reconcile messy schemas and ask a colleague for help.

• AI Agents cannot. They either fail, or worse, they fail silently.

If your infrastructure is built for BI dashboards, it simply won’t sustain a fully autonomous AI agent.

The 8-Dimension Assessment

To fix this, I’ve broken down AI readiness into 8 key dimensions across two groups:

1. Data Infrastructure: Consumers, Access Latency, Schema Rigor, and Metadata.

2. AI Operations: Error Handling, Memory/State, Evaluation, and Observability.

The goal isn’t to be “perfect” at all of them. The goal is to be honest. When you score your current setup (1 to 4) against what a use case actually requires, the “readiness” of your project stops being a debate and starts being a number.

Finding Your “Beachhead”

The most actionable part of this framework is finding your Beachhead Use Case.

A beachhead isn’t your most ambitious goal (like a fully autonomous loan negotiator). It’s the use case where:

1. The infrastructure gap is small (you can ship in weeks, not months).

2. The business value is standalone (it pays for itself immediately).

By starting here, you aren’t just “doing a pilot” - you’re building the foundation that makes the ambitious stuff possible later.

Want to run this assessment yourself?

I’ve put together a full walkthrough of these 8 dimensions and how to build your own “Gap Map.” If you’re tired of the “AI priority workshops” that lead nowhere, this might be the most useful 15 minutes of your week.

Check out the full breakdown here.

I’d love to hear your thoughts—drop a comment on the video and let me know which of the 8 dimensions is currently the biggest blocker in your org.

Have a great weekend,

Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

• MCP is the right abstraction for standardising tool access across multiple agents, but most teams are deploying it without the governance it needs

• Five failure modes keep appearing in regulated environments: hardcoded credentials, no authorisation layer between model and tool, invisible tool calls, server sprawl, and untracked data residency

• MCP is not always the right choice. Direct function calling, existing APIs, and async queues are better fits for several common patterns

• The fix isn’t complicated: treat MCP servers as infrastructure, instrument every tool call at the boundary, and never let the model be your policy engine

MCP is at that USB moment

Something shifted in 2024. Developers started asking, “can we connect an LLM to our tools?” and then wondering, “how do we do it without building a different integration for every model, every framework, every team?”

These are the questions MCP answers. The Model Context Protocol, originally developed by Anthropic and released in November 2024, and now gaining ground as a de facto standard.

By mid-2025, MCP had moved from research-adjacent to actively deployed. GitHub Copilot, Cursor, Claude, and a growing list of enterprise agent frameworks had either adopted or announced support. The server ecosystem - registries, SDKs in Python and TypeScript, community-contributed connectors for everything from PostgreSQL to Salesforce had expanded fast. Almost every other product has its MCP.

That speed should give you pause.

This is a pattern we’ve seen before. When USB was introduced in 1996, it solved a real problem: a dozen incompatible port types on the back of every PC. It was the right abstraction. And within a few years, “plug and play” had become a running joke because the driver ecosystem moved faster than the discipline around it. Devices connected. Systems crashed. Enterprise IT spent years cleaning up what consumer enthusiasm had shipped. While the protocol was fine, the deployment culture was not.

MCP is at that USB moment. The abstraction is right. The ecosystem is moving faster than the engineering rigour around it. Teams are shipping MCP servers in sprints, demoing them to CTOs, and having them in production eight weeks later. Six months after that, nobody can tell you what tools the agent is calling, the credentials haven’t been rotated since go-live, and there is no audit trail that would survive a compliance review.

The protocol didn’t fail them. The deployment pattern did.

This article is about the difference.

Tool Definition: A Contract with Non-Determinism

The fundamental difference between an MCP tool and a standard API is that it is a contract between a traditional, deterministic backend system and a non-deterministic LLM agent. Standard API engineering assumes a consumer will call a function exactly as documented. Agent tooling requires developers to accept that the model will interpret the description and choose the arguments. This non-deterministic usage is why building for agents requires a higher degree of protective rigor around entitlement, validation, and audit than standard integration patterns.

What MCP gets right?

MCP is solving a real problem, and the core of it is genuinely well-designed.

The integration tax is real, and MCP eliminates it

Before MCP, connecting an agent to a tool meant writing bespoke integration code. Every model had its own function-calling format. Every framework had its own abstraction layer. If you wanted to switch from one agent framework to another, you rebuilt your integrations. If you wanted the same capability accessible across multiple agents, you duplicated the logic and prayed for consistency.

Without a shared protocol, every team builds its own integration in the shape of its own constraints - its available libraries, its preferred auth pattern, its interpretation of what the downstream system needs. The result is integration sprawl that compounds with every new team that touches the same system. MCP breaks that coupling. A single MCP server exposes a set of tools, resources, and prompts under a standardised interface. Any MCP-compatible client regardless of which model or framework it uses, can discover and invoke those tools through the same protocol. Write once, expose everywhere.

This is what happened when REST became the default for web APIs in the early 2000s, displacing the chaos of SOAP, WSDL, and proprietary RPC formats. REST won because it was simple enough that teams could independently build to the same standard and have things actually work. MCP is attempting the same move at the agent-tool layer.

Runtime capability discovery changes the architecture

One of the underappreciated features of MCP is the tools/list endpoint. Think of it like DNS for tools - you don’t hardcode IP addresses into your application, you resolve them at runtime. An agent queries the server on startup, gets a schema-described list of available tools, and decides which to invoke based on the task. Your agent architecture can evolve without redeployment every time a tool is added or changed.

Where it genuinely shines

MCP is strongest when the problem is standardised access to well-defined internal systems. Four use cases stand out:

• Internal tool registries. A platform team builds and owns MCP servers for canonical internal capabilities - search, data retrieval, workflow triggers. Agent teams consume them without needing to understand the underlying integration. The boundary is clean, the ownership is clear, and the interface is versioned. This is the internal developer platform model applied to agent tooling.

• Governed data catalogue access. An agent needs to query dataset metadata, lineage, or schema. Exposing a data catalog (Unity Catalog, Alation, Collibra, DataHub) via MCP gives the agent a structured, permissioned interface without direct database access. The server enforces what the agent can see. This matters enormously in financial services, where an agent browsing raw schema can inadvertently surface data it has no business touching.

• Regulated workflow triggers. An agent initiates a downstream process - raises a ticket, submits a form, triggers a notification. MCP provides a typed, auditable interface for those triggers. The tool schema documents exactly what inputs are required; the server enforces them. The schema is the contract.

• Multi-agent orchestration. A supervisor agent delegates to specialist subagents, each with its own MCP server exposing its capabilities. The supervisor discovers what each subagent can do and orchestrates accordingly. This is where capability discovery really earns its keep; it makes composition between agents tractable without tight coupling.

• Tool Design for LLM Token Efficiency. The effectiveness of a tool is measured by its use within the LLM's context window. Prioritize tool interfaces that minimize the total volume of tokens consumed. For instance, prefer a search_datasets(query: str) tool to a generic list_all_datasets() tool. Furthermore, design tools with parameters for pagination and truncation to ensure the tool output - the data returned to the LLM is as concise and high-signal as possible. Refer to Anthropic’s blog: Writing effective tools for agents — with agents

A minimal MCP server for a governed data catalog wraps your catalog API behind a typed search_datasets tool enforcing domain filters, capping result limits, and keeping the agent away from raw schema access entirely. The tool definition is the contract; the server enforces it.

This is MCP doing what it’s designed for.

What risks MCP brings?

MCP moves fast to production. The failure modes tend to follow shortly after because the protocol makes it easy to ship something that works in a POC before the operational questions have been answered.

These are the five patterns you should understand.

Developers often forget “credential best practices”

Hardcoded credentials are not a new problem. What MCP changes is the rate at which new service boundaries get created. Spinning up an MCP server takes minutes - a few lines of Python, a decorator, done. That speed means teams are creating new integration points faster than their credential management habits have been built to handle. The result is more hardcoded credentials, in more places, with less visibility than traditional integration patterns would produce.

In a UK financial institution, this creates obligations that a missing credential policy would directly fail to meet under the Digital Operational Resilience Act (DORA). A credential with no owner and no rotation policy fails that bar.

DO NOT DO THIS: hardcoded credential in MCP server

DATABASE_URL = “postgresql://svc_agent:SuperSecret123@prod-db:5432/customers”

The credential is now wherever this server runs. If the server is containerised, the credential is in the image or the environment. If the image is pushed to a registry, it may be in the layer history. The blast radius of a compromise is the entire downstream system, not just the agent. This is the digital equivalent of writing your vault combination on a Post-it and sticking it to the outside of the vault.

Models usually have access to every tool in an MCP server - this is not least priviledge

MCP puts tool invocation decisions in the hands of the model. The model reads the tool schemas, decides which tool to call, and constructs the arguments. Nothing in the base protocol validates whether that decision was appropriate, whether the arguments are safe, or whether the calling agent had the entitlement to invoke that tool for that user in that context.

This is Saltzer and Schroeder’s principle of least privilege - articulated in their 1975 paper “The Protection of Information in Computer Systems” and still the foundation of access control design.

The principle states that every component should operate with only the permissions it actually needs. A model that has access to every tool in an MCP server, for every user, at all times, is a maximal privilege configuration. It is the opposite of least privilege.

In regulated industries, this matters concretely. An agent that can invoke a transfer_funds or update_credit_limit tool should not be making that invocation based solely on what the model infers from a user message. The trust chain is broken.

python

# Dangerous: model output routes directly to tool execution

async def run_agent(user_message: str, user_id: str):

    response = anthropic_client.messages.create(

        model=”claude-sonnet-4-20250514”,

        tools=mcp_tools,  # All tools. No entitlement check. No context.

        messages=[{”role”: “user”, “content”: user_message}]

    )

    for block in response.content:

        if block.type == “tool_use”:

            # The model decided. The server executes. Nothing in between.

            result = await mcp_session.call_tool(block.name, block.input)

This code grants the model access to all tools (tools=mcp_tools), establishing a maximal privilege configuration. The model’s tool\_use block is immediately trusted as the final decision, bypassing any policy or validation check. Execution proceeds directly to mcp\_session.call\_tool without confirming the user’s entitlement or context.

The model is a reasoning engine. It is not a policy engine. These are different things, and conflating them is how you end up with agents doing things nobody authorised them to do.

MCP invocations still require explicit instrumentation to be traced end‑to‑end

Any RPC to a separate process requires explicit instrumentation to appear in your trace - that is not unique to MCP. What makes MCP different is that the protocol is new enough that most observability platforms have no native integration for it yet. With a mature HTTP or gRPC stack, there is a reasonable chance your tracing library auto-instruments at the transport layer. With MCP, there is not. Teams adopting it now are on their own.

An MCP tool call is an RPC to a separate process. Without explicit instrumentation, that call disappears from your trace. You can see the model’s input and output. You cannot see which tool was called, with what arguments, what the server returned, how long it took, or whether it failed. This is the observability equivalent of a black box flight recorder that stops recording five minutes before the crash. You have most of the data. You’re missing exactly the part that matters.

For regulated deployments, this is an audit problem. The FCA’s Senior Managers and Certification Regime (SM&CR) creates personal accountability for outcomes. If an agent made a decision that affected a customer - a credit flag, a document retrieval, a workflow trigger - and that decision was influenced by a tool call, you need to reconstruct exactly what the tool returned. If the tool call isn’t in your trace, you cannot reconstruct it. “The model did it” is not an explanation that satisfies a regulator.

The fix lies in instrumentation at the MCP boundary, not inside the server. This is shown in the MCPGateway pattern below.

Every team builds one, no one owns them - there is no established governance for MCP servers

In a large organisation, the absence of a shared standard is a vacuum that teams fill independently - each making a locally rational decision that creates a globally irrational system.

The MCP equivalent: without governance, every team designs its own simple system. The risk team builds an MCP server for their data warehouse. The finance team builds a different MCP server for the same data warehouse with different auth. The platform team builds a third server that partially overlaps with both. The agent now sees twelve tools that do variations of the same thing. The people who built them have moved on. No deprecation path exists. This is not a hypothetical - it is the same pattern that played out with internal REST APIs at most large organisations that adopted microservices without a service catalogue, and it is already starting to repeat with MCP.

The structural cause is Hyrum’s Law: observed by Google engineer Hyrum Wright and now widely referenced in software engineering: “With a sufficient number of users of an API, it does not matter what you promise in the contract — all observable behaviours of your system will be depended on by somebody.”

Once a team starts using your MCP server, they will depend on its undocumented behaviours. Deprecating it without governance becomes painful very quickly.

Treating MCP servers as infrastructure from the first deployment - with an owner, a version, a changelog, and a deprecation policy is not bureaucracy. It is the thing that lets you move fast in two years without digging out from under your own sprawl.

MCP doesn’t answer the data residency question

MCP doesn't answer the data residency question, and neither should it, but because MCP makes it trivially easy to spin up a new server anywhere, teams are creating new data residency exposure points faster than they're tracking them.

This is the failure mode that almost never appears in ecosystem documentation and is the one most likely to cause a material incident in regulated industries.

When an agent calls an MCP tool, data flows in both directions: the arguments sent to the tool, and the response returned. In a regulated context, both can contain customer data, personally identifiable information, or material non-public information. The question of where that data flows - which process handles it, which logs capture it, which jurisdiction it transits through - is not answered by the protocol. That is not a criticism of MCP. It is simply a boundary you need to understand.

The MCP server is a process. That process can run anywhere. If it runs in a container in a region that is not approved for the data it is handling, you have a data residency violation before the tool even returns a result. If the tool call arguments are logged by intermediary infrastructure before reaching your server, you have a data handling question that needs a documented answer.

For UK firms post-Brexit, this intersects with UK GDPR, FCA data governance expectations, and potentially the location requirements of your outsourcing arrangements. The compliance question is not “does MCP support data residency?” - it does not operate at that layer - but “can you trace every byte of this tool call, confirm where it went, and show it never left an approved boundary?”

That answer requires mapping the full data flow before deploying any MCP server that touches regulated data. Not after the first incident.

Do you actually need MCP?

I see it commonly - every team jumps to the conclusion of using MCP very fast. MCP is infrastructure and infrastructure has a cost - operational overhead, governance burden, complexity. The Unix philosophy, articulated by Doug McIlroy in the early 1970s, puts it plainly: “Write programs that do one thing and do it well.” Before you introduce a protocol for interoperability, ask whether you actually need interoperability.

Here is a decision tree I like to use to help teams make that decision:

Let’s make each branch concrete.

• When direct function calling is the right answer

  If you have a single agent, a small number of tools, and no requirement for reuse across teams or frameworks, native function calling in the Anthropic SDK is simpler, cheaper, and easier to observe. No additional process boundary. No MCP server to maintain. No capability discovery overhead. Adding an MCP server here is the software equivalent of installing industrial plumbing to fill a kettle.
  

• When a well-governed existing API beats MCP

  If the downstream system already has a REST or gRPC API with proper authentication, rate limiting, observability, and documentation - a Salesforce API, an internal risk platform, a data catalogue with its own REST interface, wrapping it in an MCP server often adds a layer without adding value.
  

  The test is simple: does the MCP layer provide something the existing API does not? If the answer is capability discovery for agent consumption, standardised schema, or unified access across multiple systems, MCP earns its keep. If the answer is “it’s just a wrapper,” you’ve added a process boundary, a deployment artefact, and an operational dependency for no functional gain.
  

• When an async pattern is the right architecture

  MCP is synchronous request-response. The agent calls a tool, blocks, and waits. That is fine for fast, bounded operations. It is the wrong shape for long-running jobs (submit and poll, not block), audit-required fire-and-forget (a message queue with dead-letter handling gives you durability and replay that MCP cannot), and event-driven workflows where the agent should be consuming from a stream, not polling in a loop.

MCP solves a specific problem well. The mistake is treating it as the default integration pattern for anything agent-related, rather than the right answer to a specific architectural question.

One pattern that works

Let me show you an architecture pattern to address these failure modes above. First, let’s understand the few best practice that matter here. It a bit of recap, but worth a refresh.

1. MCP servers are infrastructure, not glue code. You need an owner, you need versioning, changelog, deprecation policy. It should be registered in your internal catalogue and deployed through the same pipeline as your other services.
   
   “Who owns this MCP server?” should have a human name attached to it.

2. Every tool call must be observable. Instrument at the boundary - in the layer between the orchestrator and the MCP session - not inside the server. The instrumentation wrapper is shared infrastructure, not something each team spends time re-implementing.

3. The model never touches credentials or entitlements directly. There is always an authorisation layer between the model’s invocation decision and the tool’s execution. This is not optional in a regulated environment. It is least privilege applied at the agent layer.

4. Credentials are runtime injection, not baked-in secrets. Retrieve credentials at server startup via your secrets manager (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault) - never at image build time, never from environment variables baked into a container. The credential is never in your code, never in your image layer history, and has a documented owner and rotation policy. Rotation is handled by the secrets manager; the server picks up new credentials on the next startup cycle.

5. Define quality before deployment using Evals. The observability focus in production must be preceded by rigorous quality assurance during development. And these ar enot unit tests - you need systematic measurement of the LLM's ability to use the tool correctly. Advocate for comprehensive evaluation tasks grounded in complex, real-world scenarios to measure tool efficacy. For debugging, run these evaluations programmatically, instructing agents to output their reasoning steps alongside the tool invocation. This practice helps developers probe exactly why an LLM selects or struggles with specific tools, ensuring the quality of the non-deterministic contract before it is exposed to regulated production environments.

Every tool call passes through the Guardrail Layer before it gets anywhere near the MCP server. That layer validates tokens, masks PII, fetches credentials from the Secrets Manager at runtime, and writes an immutable audit entry before execution starts. The MCP server receives a clean, credentialled, traced call. The response comes back, gets sanitised, and the trace span closes. The user gets a safe payload. Nothing touches the downstream system without a paper trail.

Conclusion

MCP is the right abstraction at the right layer. The standardisation problem it solves is real, and the use cases where it works well are genuinely valuable enterprise problems. The protocol is not the issue.

What history tells us from USB to REST to microservices is that good protocols get adopted faster than the discipline to deploy them safely.

That gap is where incidents come from. This is commonly described as the difference between a sharp knife and a blunt one: the sharp knife is more dangerous in the wrong hands, but it’s the right tool for someone who knows what they’re doing.

The teams that get this right usually ask the boring questions first - who owns this, where does the data go, what happens when this call fails at 2am - and build systems accordingly.

That’s not caution. That’s just engineering.

Using or building with MCP?

Don't let speed compromise security. Apply the governance and architectural rigor outlined here to your MCP servers now. Build your Authorization and Observability Layer first to ensure your tool calls are secure, auditable, and compliant from day one.

Tell me in comments whether this resonates, what other challenges are you facing, and where did MCP do the magic for you. I am eager to learn from your experience - so please comment, leave a feedback.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

But as teams push these systems into production, they are hitting a wall. Having five smart agents does not automatically equal one smart system.

The industry is quickly learning a hard lesson: orchestration is an understood problem, and communication reliability is the actual bottleneck . You can easily instantiate ten agents using frameworks like LangGraph, AutoGen, or CrewAI. But getting them to talk to each other reliably without hallucinating payloads, dropping context, or getting stuck in infinite loops is where the real engineering happens.

Let’s break down the first principles of agent communication, look at how they fail in production, and explore how modern production systems are solving these exact problems.

The First Principles of Agent Communication

At its core, getting agents to collaborate requires the same fundamentals as distributed computing, but with a chaotic twist: the “nodes” in this network are non-deterministic text engines.

1. Message Passing

Principle: Message passing is the transfer of information from one node to another. When Agent A finishes its job, it must hand off a payload to Agent B to trigger the next step.

• Failure Mode: The Hallucinated Payload.

• Example: A Data Extraction Agent is told to pull a user’s ID and pass it to a Database Agent. Instead of passing 12345, the agent passes, “Here is the user ID you requested: 12345.” The Database Agent expects an integer, receives a conversational string, and crashes.

  

• Why it happens: LLMs are fine-tuned to be helpful conversationalists, not strict state machines. Without hard constraints, they inject pleasantries and markdown formatting into their outputs, corrupting the message payload.

2. Protocols and Interfaces - H2A, A2C, A2A

Principle: Protocols dictate the rules of engagement. In modern agentic systems, we categorize these interfaces into three distinct buckets:

• H2A (Human-to-Agent): Conversational, unstructured, and forgiving (e.g., ChatGPT).

• A2C (Agent-to-Computer): Rigid and deterministic. The industry standard here is the Model Context Protocol (MCP). Introduced by Anthropic in late 2024 and now hosted under the Linux Foundation, MCP standardizes how agents securely connect to external tools, IDEs, and databases.

• A2A (Agent-to-Agent): Peer-to-peer communication between two non-deterministic models. This is historically the most fragmented layer, but the industry recently coalesced around the Agent-to-Agent (A2A) Protocol. Originally released by Google in April 2025 and unified with IBM’s Agent Communication Protocol (ACP), A2A is now a Linux Foundation open standard for cross-framework agent discovery and task delegation over HTTP and JSON-RPC.

• Failure Mode: Interface Confusion.

• Example: A developer uses MCP to perfectly connect a Research Agent to a PostgreSQL database (A2C). But when the Research Agent hands the data to a Writer Agent (A2A), the developer lets them communicate in conversational English. The Writer Agent misinterprets the unstructured text and hallucinates missing facts.

• Why it happens: Developers often treat A2A communication like H2A communication. Unless you enforce machine-readable protocols for peer-to-agent handoffs, conversational drift will inevitably break your architecture.

3. Shared Context and State

Principle: Agents need a shared understanding of the environment, current progress, and available data to collaborate effectively.

• Failure Mode: Context Desynchronization.

• Real-World Example: A Researcher Agent analyzes a 50-page PDF and passes a brief outline to a Writer Agent. The Writer tries to draft the article but fabricates details because it lacks access to the source material.

• Why it happens: Context windows are expensive. To save tokens and latency, builders often restrict the context passed downstream. This creates asymmetric information - Agent A knows something Agent B doesn’t, leading to poor decisions.

4. Intent Alignment

Principle: Every agent in the chain must understand the overarching goal of the user, not just its localized sub-task, to ensure the final output is cohesive.

• Failure Mode: The Telephone Game.

• Example: A user asks for a “brief, humorous summary of the latest AI news.” The Manager Agent passes the news to the Summarizer Agent but forgets to include the “humorous” instruction. The Summarizer writes a dry academic brief.

• Why it happens: Information decays across hops. When breaking a complex prompt into smaller agentic tasks, the nuance of the original user prompt is easily lost in translation.
  

  

5. Memory and Feedback Loops

Principle: When Agent B rejects Agent A’s work, Agent A needs memory of the failure and the ability to correct itself without repeating the exact same mistake.

• Failure Mode: The Infinite Death Spiral.

• Example: A Coding Agent writes a Python script. The Execution Agent runs it, encounters a SyntaxError, and passes the error back. The Coder apologizes, generates the exact same code, and sends it back. They repeat this loop 50 times until the API budget is drained.

• Why it happens: LLMs are highly sensitive to their immediate context. If the feedback isn’t explicit, or if the model’s internal weights heavily favor a flawed syntax pattern, it will deterministically generate the same wrong answer.

How To Solve This - Patterns That Are Working

Building reliable multi-agent systems requires shifting your mindset from “prompt engineering” to “protocol engineering.” Here is how top-tier engineering teams are building resilience into their agent networks.

1. Enforce Structured Outputs (a.k.a Strict Schemas)

Never let agents talk to each other in free-text prose if they are exchanging data. Treat agent communication exactly like an API. Use tools like OpenAI’s Structured Outputs, Pydantic, or standard Python libraries like instructor to enforce JSON schemas. If Agent A needs to pass an ID to Agent B, structurally guarantee that the output is only a valid JSON object matching your exact schema.

2. Adopt Standardized Protocols

Stop reinventing the wheel for tool use and communication. Implement MCP for all A2C interactions to securely connect your agents to external systems. For A2A interactions, adopt the A2A Protocol to standardize payload structures instead of injecting variables into conversational prompt templates. Leveraging these standards ensures your agents can operate reliably across different platforms and enterprise environments.

3. Centralize State (The “Blackboard” Pattern)

Instead of passing massive context back and forth between agents like a hot potato, use a centralized state mechanism. In graph-based frameworks like LangGraph, all agents read from and write to a single, shared state object or “blackboard.” This ensures no agent is operating on outdated or asymmetric information.

4. Implement Observability and Tracing

When a multi-agent system fails, it fails silently and weirdly. You cannot debug these systems with standard print statements. You need dedicated LLM observability platforms like LangSmith, Langfuse, or MLflow 3.0 to trace the exact input, output, and execution path of every single node. If the Telephone Game happens, you need to see exactly which agent dropped the context.

5. Defensive Programming: Retries and HITL

Expect agents to fail. Wrap inter-agent communication in standard retry logic with programmatic guardrails (like NVIDIA’s NeMo Guardrails). If Agent A sends malformed data, catch the error programmatically and format it into a rigid prompt to force a correction. For critical workflows like writing to a production database (or deleting production databases) - enforce a Human-in-the-Loop (HITL) pause. Let the system wait for human approval before executing destructive actions.

Lessons Learnt

• Treat agents like microservices: A2A communication should mirror microservice architecture. Define rigid, structured API contracts for every handoff and validate payloads before they reach the next node.

• Embrace MCP and emerging standards: Separate your A2C (tool use) from your A2A (agent coordination). Use standardized open-source protocols like MCP and A2A to offload the complexity of system integrations so your agents can focus on logic.

• Cap your feedback loops: Always implement hard limits on iterative loops. If agents go back-and-forth more than three times without success, throw an exception and escalate to a human or a deterministic fallback script.

• Persist the global goal: Inject the original user intent into the system prompt of every agent in the pipeline. Do not assume intent will survive passing through three different LLM nodes.

Multi-agent architectures are the future of complex AI applications, but they require rigorous distributed systems engineering. By enforcing strict protocols and shared state, you can stop the silent failures and build agentic systems that actually work in the real world.

See It In Action

I’ve talked about the common patterns and failure modes of multi-agent orchestration and how to design for them in this video. Have a look.

As always your comments and feedback are welcome. Please share your experience and thoughts.

Thanks,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

If you are reading yet another think-piece on “scaling autonomous agents” or “optimizing your basic RAG pipeline,” you are observing the trailing edge of the industry. We all know the standard playbook by now: deploy an LLM-as-a-judge, set up a vector database, and run basic semantic search. That is no longer a competitive advantage; it is table stakes.

To survive in production at scale today, engineering teams must stop treating generative models like brilliant, autonomous colleagues and start treating them like chaotic, highly expensive engine components.

Here are the three architectural blind spots that standard DevOps playbooks are ignoring - and how to fix them.

1. Stop Building Agents. Build State Machines.

The current industry obsession is giving LLMs autonomy - letting them chain tools, determine their own loops, and “think” their way out of problems. However, in an enterprise production environment, autonomy is just another word for liability.

You do not want an autonomous agent; you want a rigid, locked-down Finite State Machine (FSM).

Your software architecture should entirely dictate the exact path, the boundaries, and the execution graph. The LLM should only be utilized for the transition logic. Its sole job is to ingest messy, unstructured user input and output a deterministic decision: “Do we transition to State A or State B?”

By stripping the model of its agency and restricting it to routing and classification, your latency drops, your reliability scales, and crucially, your system becomes highly debuggable when an edge case inevitably breaks the flow.

2. Eradicate the “Politeness Tax”

If you audit your raw token logs, you will likely find that you are paying thousands of dollars a month - and sacrificing hundreds of milliseconds of latency per request - just to let your model clear its throat.

Every time a background model outputs, “Certainly! I’d be happy to extract that data for you. Here is the requested JSON:”, you are burning compute. At scale, politeness is an engineering flaw.

You cannot fix this with prompt engineering alone. You must enforce strict grammar constraints at the inference level. Do not politely ask the model to return JSON in the system prompt; force the API to accept { as the absolute only valid first token. Strip out all conversational abilities from your background processing models.

You do not need a polite assistant in your backend data pipeline; you need a ruthless text calculator.

3. Neutralize “Zombie Memory” in Semantic Caching

Semantic caching is universally recommended to reduce API costs. A user asks a question, you embed it, check if you have answered a mathematically similar query recently, and return the cached answer.

What nobody discusses is semantic cache rot. If you are caching answers about dynamic data, like your pricing tiers, live inventory, or active user permissions, the underlying reality will eventually change, but your vector cache remains static. When this happens, the cache intercepts the query and serves up a perfectly formatted, highly confident answer that is now entirely false. Your system isn’t hallucinating; it is remembering a dead reality.

To solve this, a simple Time-to-Live (TTL) expiration is insufficient. You must bind your vector cache invalidation directly to your database webhooks. If a product goes out of stock in your primary database, your system must aggressively and automatically flush the neighborhood of vectors in your cache that map to that specific product’s metadata.

The Takeaway

Moving AI from a compelling local demo to a hardened production environment requires a fundamental shift in engineering mindset. It is not about finding the perfect prompt or chasing the newest foundational model.

The best AI engineers do not try to find perfect model outputs . They build perfect architectural nets to catch the model when it inevitably behaves unpredictably.

What is your production AI blind spot?

We are all writing the playbook for production AI in real-time, and the best lessons come from the trenches, not the demo environments. Hit reply and tell me about the weirdest silent failure mode you have caught in production recently, the one that no standard DevOps tool saw coming.

If this issue helped you rethink your architecture, do me a favor: forward it to the engineer on your team who is currently trying to solve a systems problem with another paragraph of prompt engineering.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai