One of my goals with this newsletter is to make every issue genuinely useful for you. Rather than guessing what you’d like to read, I’ve put together a short survey to understand the topics, challenges, and formats that would be most valuable to you.

It only takes 30 seconds to complete, and your feedback will directly influence the future content of this newsletter. If there’s a topic you’ve been struggling with or would like me to cover, this is the best way to let me know.

👉 Take the survey here

Thank you, it genuinely helps me write better content for this community.

Over the past year, I’ve had countless conversations with teams building AI agents.

The discussion almost always starts the same way.

“Which model should we use?”

GPT-5? Claude? Gemini? An open-source model?

It’s a reasonable question. But it’s rarely the one that determines whether the project succeeds.

In production, AI agents usually don’t fail because the LLM isn’t intelligent enough.

They fail because the data infrastructure wasn’t designed for agents.

Model problem is an illusion, mostly

When an agent gives a poor answer, it’s tempting to blame the model.

In reality, the root cause is often somewhere else.

• The agent couldn’t find the right information.

• The retrieved context was outdated.

• The metadata was incomplete.

• Permissions prevented access to critical documents.

• Tool outputs were inconsistent.

• There was no reliable way to evaluate whether the response was correct.

Changing the model may improve the symptoms.

But it doesn’t fix the underlying system.

AI agents are data systems

Traditional software follows predefined logic.

LLM applications generate responses.

AI agents do something more demanding.

They continuously retrieve context, invoke tools, maintain memory, reason across multiple steps, and make decisions based on dynamic information.

That creates different engineering challenge.

The bottleneck is no longer model capability. The models are great.

It’s the quality of the surrounding data system.

Four infrastructure questions you should answer

Before choosing an LLM, I believe every architecture review should answer these four questions.

1. How does the agent acquire context?

• Where does information come from?

• Structured databases?

• Documents?

• Knowledge graphs?

• Streaming events?

• How is relevance determined?

• How is freshness maintained?

2. How does the agent remember?

   Memory isn’t just conversation history. You need to define:

• session memory

• long-term memory

• organisational knowledge

• user-specific context

• retention and expiry policies

Memory is a data architecture problem long before it’s an AI problem.

3. How does the agent interact with enterprise systems?

Most production agents spend more time calling APIs than generating text.

• Can tools be discovered?

• Are they authenticated?

• Are outputs standardised?

• Can failures be retried?

• How are side effects controlled?

These questions often determine reliability more than model quality.

5. How will success be measured?

This is the question I see missed most often.

Teams build agents first. Evaluation comes later.

Instead, define success before implementation.

• What does a correct response look like?

• What datasets will validate it?

• What metrics matter?

  • Task completion

  • Tool accuracy

  • Groundedness

  • Latency

  • Cost

  • User satisfaction

If you can’t measure quality, you can’t improve it.

Your role is changing

The next generation of engineers will be distinguished by designing systems that give any capable model the right information, at the right time, with the right controls.

Models will continue to improve.

The system around it remains the long-term differentiator.

That’s where architectural decisions create lasting advantage.

Want to go deeper?

This article only scratches the surface.

Designing data infrastructure for AI agents is a discipline in its own right, covering topics such as context engineering, memory architectures, retrieval strategies, evaluation frameworks, tool integration, governance, and observability. These are the decisions that determine whether an agent works reliably in production.

If you’re a software engineer, data engineer, platform engineer, or architect looking to build production-grade AI systems, I’ve partnered with Packt to deliver a hands-on workshop on Designing Data Infrastructure for AI Agents.

Rather than focusing on prompt engineering or model comparisons, we’ll work through the architectural patterns, design principles, and practical exercises needed to build agent systems that are scalable, reliable, and maintainable.

👉 [Register for the Packt Workshop]

I hope to see some of you there.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Six months ago, AgentBuild was a few hundred people. Today, it’s 1100+, spread across 73 countries - from Bangalore to Boston to Bradwell.

I want to index on proving value over increasing subscriber count.

Hence, I’m going to ask for your help.

First, the Growth Curve

A year ago, AgentBuild had roughly 300 followers. Today it’s 1,158 - nearly a 4x jump, with the steepest acceleration happening in just the last four months. Something clicked around April, right when I doubled down on decision traces, agent registries, and evaluation frameworks instead of chasing whatever AI news cycle was trending that week.

Where you’re all reading from:

• India: 37%

• United States: 24%

• United Kingdom: 7%

• Germany: 3%

• Canada: 2%

• Plus pockets across 68 other countries

That’s a genuinely global room of practitioners, which still blows my mind every time I look at it.

Where I stand today

My last 20 posts averaged an open rate between 22.5% and 30.3%, with a 30-day rolling average of 25.24%.

Let’s put that in context:

• General email marketing average: ~21-22% open rate

• Substack’s platform-wide headline number: often quoted at 35 - 37%

• Real-world compressed range for niche newsletters in 2025-2026: closer to 28-32%

So where does that put AgentBuild?

Comfortably ahead of generic marketing email, but trailing the Substack averages. I call it “solid, not spectacular.” And that’s fine.

My best-performing piece, “Why Solution Architects Are the Real Force Behind Enterprise AI Transformation,” hit a 30.28% open rate - proof that when the topic lands, the number moves. Which is exactly why I want your input on what topics actually land for you.

My most-read piece by a wide margin: “Why Agent-to-Agent Communication Fails - How to Design for Failure” pulled 897 views, nearly double my typical post.

Close behind: my two-part series on building an AI agent registry (586 and 476 views), and “The Evaluation Graph: Why Your AI Pipelines Are Lying to You” (466 views).

There’s a clear pattern here - you want tactical, failure-mode-focused content over broad conceptual overviews.

Take the survey, please

If I’m grading myself against the benchmarks: good, not great. I’m beating the average email newsletter by a healthy margin, but I’m not yet in “50%+ open rate, readers treat this like a must-open” territory that the best niche Substacks report.

There’s a room to close that gap and the fastest way to close it is writing more of what you actually want, instead of guessing.

Hence, I’ve put together a short survey to answer three things:

• what topics you want more of,

• what format works best for you,

• which of the three community features should I launch

👉 Take the survey here

Your answers will directly shape the next quarter of AgentBuild issues. This isn’t a formality - I’m genuinely building the next stretch of this newsletter around what you tell me. I don’t want to make assumptions.

What I’m Building Next

Based on the feedback I’ve already gathered, three things kept coming up as the most wanted next step for this community:

1. Ask Me Anything sessions - live, unscripted Q&A where you bring your hardest production AI problems and I answer in real time.

2. Virtual meetups - smaller group calls connecting practitioners across India, the US, UK, and everywhere else you’re reading from.

3. A reader community forum - a space to swap notes and debate frameworks between issues, not just consume them passively.

I’m committing to build all three.

The only question left is sequencing - which one should I do first?

Tell me in the survey above.

Thank you for being part of the community. I am focusing on delivering value to each of you. Feel free to reach out if you need any help.

Talk soon,
Sandi

👉 Take the survey here

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Palantir built a business on this word. Every catalogue vendor has quietly rebadged its glossary as a semantic layer. Features tagged with the word “Ontology“ are being released.

Underneath the marketing there’s a real engineering shift, and it’s worth separating the two.

Subscribe now

Why Ontology matters for enterprise AI?

An LLM reads your data. It doesn’t understand your business, and nothing in your stack is currently telling it the difference.

Ask three systems what a customer is. Sales counts anyone who signed. Support counts anyone with a login, which sweeps in two people who never bought. Finance only counts accounts that have paid an invoice. The gap between those definitions is real money, and the only place it’s written down is in the head of the employees.

So when someone asks the AI how many customers you have, retrieval pulls all three definitions into the same context and the model hands back one number. It won’t mention that it merged three incompatible populations to get there, and you’ve got nothing to check it against.

A glossary won’t save you, because a glossary is written for people to read, not for a pipeline to enforce. A tidier schema won’t either. A foreign key can say two tables are related; it can’t say what the relationship means. That missing layer is what an ontology holds, in a form the machine reads at query time rather than a form a human reads in a wiki.

It becomes even more serious in the write path. A chatbot that miscounts customers gives a wrong answer. An agent that miscounts and can act on it does something about it, mails the wrong segment or moves the wrong forecast. That’s roughly when the word “ontology“ started coming up in architecture reviews.

What is an Ontology?

The textbook definition is Gruber’s:

an ontology is a formal, explicit specification of a shared conceptualisation.

Let’s look at the four words:

• A conceptualisation is the abstract model of what exists in a domain. It’s already in everyone’s head, whether written down or not.

• Explicit means it’s written down.

• Formal means a machine can act on it, not just a person read it.

• Shared means it’s agreed and owned across a community, not one architect’s private diagram.

Mechanically, you write statements as triples: subject, predicate, object.

Here’s a fragment in Turtle:

:Customer    a  owl:Class .
:PayingCustomer  rdfs:subClassOf  :Customer .
:placed      a  owl:ObjectProperty ;
    rdfs:domain  :Customer ;
    rdfs:range   :Order .

The last block says something a foreign key cannot: placed only ever runs from a customer to an order. Assert that some account placed an order, and a reasoner infers the account is a customer. Nobody typed that fact. It follows.

Then you query the meaning instead of the tables:

sparql

SELECT ?customer WHERE {
  ?customer  a        :PayingCustomer ;
             :placed  ?order .
}

This returns paying customers across every source system feeding the graph, because the definition of “paying customer” lives in one place now instead of three. That’s the line between a schema and an ontology.

A schema constrains data for one application. An ontology models a domain for many, and it entails things you never wrote down.

How to actually build one

This is very new for most companies. There is no fromal playbook - but I can see a number of guidances coming out. There are many issues I see with ontology imlementation. Usually a modelling team would take nine months to produce two thousand classes, and nobody uses it. That’s costly.

Given what I have learnt so far, I can roughly put some thoughts on the implementation, already working ins ome enteprrises:

1. Start with competency questions. Write the twenty questions the ontology must answer, in business language, before you model anything. “Which counterparties have aggregate credit exposure above £50m across all subsidiaries?” If your model answers it, that part is done. If you can’t write the questions, you don’t have a use case, you have a vocabulary project.

2. Reuse before you build. FIBO already models counterparty, legal entity, and financial instrument, with a decade of EDM Council work behind it. SNOMED CT has done clinical terms. Your novel IP is the last 10% specific to your firm, not the definition of a legal entity. Reuses existing artefacts.

3. Treat it as software. Ontologies rot faster than code because nobody owns them. Put the files in Git. Write SHACL shapes as your tests and run them in CI, so a change that breaks a downstream contract fails the build rather than a quarterly report. Give every module a named business owner who signs off changes, and version it properly.

4. Ship thin vertical slices tied to one workflow. One agent, one domain, one set of competency questions, in production. The organisational fight you’re picking is agreement on what words mean, and that fight is much easier to win on a small surface with a working demo behind it.

The takeaway

The ontology conversation isn’t a semantic web revival. It’s what happens when systems start acting on their own and the industry discovers it never wrote down what anything means.

I went deep on this with Jessica Talisman, who has spent her career on taxonomy and ontology work at the sharp end of large organisations. We got into where taxonomies stop and ontologies start, what breaks at enterprise scale, and why most of the failures are organisational rather than technical. Spend sometime this weeked, watch it, and let me know how you found it.

Hope you enjoy it.
Happy Weekend.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.



More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

Subscribe now

If you are buidling with coding agents, you have surely faced this problem. You ask the coding agent to build a feature, it writes the code, writes the tests, and reports success. However, you still find bug was there the whole time. The tests didn’t catch it. This is a common problem and in this article I will share what works for me most of the time.

Why this happens

Think about what a test is actually for. A test is a second opinion. You write code that you think is right, and you write a test that checks whether it really is. The value comes from those two things being separate. The code is your attempt. The test is the check on your attempt. If they disagree, you’ve caught something.

Now here’s what goes wrong with AI.

When you ask an AI to write the code and the tests in one go, you lose the separation. This creates a correlated failure or model bias propagation. Because the LLM shares a “latent space” between the test and the implementation, it is statistically likely to produce tests that validate its own mistakes-the model simply predicts the most probable test code given the most probable implementation code it just generated. The code has the bug, and the test is written to expect the buggy behaviour.

Consider a simple shopping cart: you ask the AI to calculate the total price, including a 10% tax. The AI rushes the job and writes a calculation function that accidentally skips the tax entirely. Then, it immediately pivots to writing the test. Because the AI is still “thinking” about the buggy code it just wrote, it creates a test that expects the total without the tax.

Now, you have a buggy function that ignores tax, and a test that celebrates that exact result. The test doesn’t flag the error because it wasn’t designed to check for the correct business requirement, it was designed to match the code’s current, buggy behavior. Everything goes green, even though the feature is fundamentally broken.

The test didn’t fail you because it was a bad test. It failed you because it was written by the same thing that wrote the bug. It’s marking its own homework.

And it gets worse at scale. One developer writing careless tests writes a few. An AI writes dozens in seconds - all neat, all passing, all quietly checking for the wrong thing.

How I fix this problem

The fix is to put the separation. You want the tests written by something that has not seen the code and cannot change it. That’s the whole idea. Everything below is just how to set that up.

If you’re using Claude Code, you do this with something called a sub-agent. A sub-agent is just a helper with a specific job and its own limited set of permissions. You create a sub-agent by adding a small text file to your project or from the terminal on Claude code.

In this case, you have to make two sub-agents:

• one that only writes tests

• one that only writes code

The trick is in what each one is allowed to touch.

If you aren’t using an agentic framework like Claude Code, the core concept remains the same: force a “Contextual Firewall” between your test-writing process and your implementation process. You don’t need a specific tool; you need a workflow that treats them as separate entities.

Let’s see how to do this step-by-step.

Step 1 - Create the tester, and don’t let it write code

Add a file called test-author.md in a folder named .claude/agents/ in your project. Put this inside it:

---
name: test-author
tools: Read, Write
model: sonnet
---
Write tests based on the requirement I give you.
Do not look at the existing code first.
Write tests for what the feature is supposed to do.

The important line is tools: Read, Write. This lets the tester read the requirement and write a test file - but it does not give it permission to change any existing code.

Conversely, the Coder will need read access to these tests to understand the criteria it must pass, but restricting its write access is crucial to prevent it from “hacking” the tests to suit its own buggy implementation.

Step 2 - Create the coder, and don’t let it touch the tests

Add a second file, implementer.md, in the same folder:

---
name: implementer
tools: Read, Edit, Bash
model: opus
---
Make the failing tests pass by changing the code.
Never edit a test file. Never delete a test.
Keep fixing the code and re-running until the tests pass.

This one is the opposite. It can edit code and run the tests (Bash lets it run them), but it’s told never to touch a test file. So it can’t cheat by weakening a test that’s in its way. Its only route to green is to actually fix the code.

Note: using different models can help, but not always - especially within the same model family, where they tend to share the same blind spots. The real independence comes from isolation and tool limits, not the model choice.

Step 3 - Write the tests first, and read them yourself

This step matters the most.

Give the tester the requirement - in plain English, what the feature should do. Let it write the tests before any code exists. Then, perform a Senior Engineer Review.

Position the AI as a Junior Developer generating boilerplate; your job as the Senior Reviewer is to validate the “contract” of the feature. You don’t need to write the code, but you must ensure the tests are asking for the right thing. The AI does the typing; you own the architectural judgement.

Don’t try to automate this part. At least not for critical code.

In the shopping cart example, you’d check: is there a test confirming the total includes tax? If yes, you’re safe. If the code later forgets the tax, that test will fail, and the bug gets caught right there.

Step 4 - Let the coder make them pass

Now hand it to the coder. It writes the actual code, runs the tests, sees what fails, fixes it, runs again - until everything’s green.

But this time, green means something. The tests it’s passing were written by something that never saw the code and can’t be edited by the thing writing the code. The two are genuinely separate again. The second opinion is back.

Run through the cart example one more time. The tester, working only from your requirement, writes a test that says the total must include tax. The coder writes the function. If it forgets the tax, the test goes red, and the coder has to fix it. The bug that sailed through before now can’t get past step 4 - because the thing checking for it was never allowed to see the mistake in the first place.

Reminder

This fixes a specific, common problem: the AI quietly agreeing with its own mistakes. It’s a real fix and it’s worth doing.

What it doesn’t fix is you asking for the wrong thing. If the requirement you gave the tester was wrong, you’ll get a perfectly tested version of the wrong feature. The tests can only be as right as the requirement behind them. Checking that the requirement matches what the business actually needs is your job - you need to do it.

Takeaway

If you’re letting an AI write your code and your tests together, and trusting the green tick, be careful. That green doesn’t mean what you think it means. It means the code and the tests agree - and since the same AI wrote both, of course they agree.

Do these four things instead:

1. Make a tester that can write tests but can’t change code.

2. Make a coder that can change code but can’t touch tests.

3. Write the tests first and read them yourself.

4. Let the coder make them pass.

Separate the two, and the tests start protecting you again - which is what they were always supposed to do.

I love diving deep into these topics

Some weekends I spend time with teams talking about coding agents - from the basics right through to building production-grade applications with them. Last weekend I did this with a team at a software consultancy. It was fun, and we had such a valuable exchange of ideas.

This is the best time I get to spend - speaking to people who are genuinly interested in building real things with AI, not slops.

Thanks for being a part of this community.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

So the newsletter crossed 1,000 subscribers this week.

1,000 is nothing in newsletter world. People do that in a weekend with one lucky post. I know that. But those aren’t the thousand I got.

The thousand I got are people building AI inside banks, insurers, healthcare companies, startups, consultancies. People who reply to my emails saying “we hit exactly this last quarter.” That’s a different thousand. I’d take it over 50,000 randoms any day.

You give me space in your busy inbox - that matters a lot to me.

Thank you for that.

Subscribe now

Why I started the newsletter?

Everyone’s shouting about AI. Demos everywhere, announcements everywhere, everyone’s an expert on LinkedIn by Tuesday. But almost nobody talks about the boring part - what happens six months after the demo, when the thing’s in production, the data’s messy, the regulator’s asking questions, and someone senior wants to know why the numbers moved.

That part. The unglamorous part. That’s the whole newsletter. Because that’s where I’ve spent my career, and honestly, that’s where yours gets made too.

So nothing changes at 1,000. I’ll keep writing about the stuff that actually breaks - evaluation, data debt, decisions nobody wrote down. I’ll keep sharing the failures, mine included, because that’s where the lessons are. And I’ll keep reading every single reply, because half my best issues started as one of you saying “we’re seeing something weird and nobody’s writing about it.”

Keep sending me those. Seriously.

One more thing. I’m not going to take up space in your inbox just to celebrate - that’s not what you signed up for.

So here’s something useful. I just published a video on why most AI projects fail. It’s everything I’ve learnt working with enterprises over the past few years - the patterns I keep seeing, and what I think actually makes these projects succeed.

Watch it here.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

So when your agent starts giving wrong answers at scale don’t look for the failure in the model. It’s in the memory system you built around it. And if you don’t understand the memory types and how they behave differently under load, you’ll spend weeks debugging the wrong layer. I have seen this happen multiple times.

Why Memory Architecture Is the Real Work

Consider what happens in a real production deployment at a financial institution. An agent handles customer queries across thousands of sessions per day. It needs to know what the customer asked last week. It needs to know current product terms. It needs to know what it did three steps ago in the current task. It needs to know that when a customer says “the same account as before,” they mean their savings, not their current account.

These are four completely different information retrieval problems. Treating them as one (which is what happens when teams reach for a vector database and call it done) is where agents fall apart in production.

Each of these problems maps to a distinct memory type. Each type has different latency characteristics, different staleness risks, different failure modes, and different architectural requirements.

The Four Types of Agent Memory

1. In-Context Memory (Working Memory)

This is whatever sits inside the current context window - the active prompt, the conversation so far, the instructions, the retrieved chunks, the tool outputs. It’s the only memory the model can directly reason over.

• What it’s good for: The current task. Right now. This session.

• Constraint: Context windows are finite. GPT-4o gives you 128K tokens. Claude gives you 200K. They sound large until you’re injecting retrieved documents, conversation history, system instructions, and tool schemas simultaneously. You will always burn through context faster than you would expect.

• Failure pattern: Teams usually design for a 10-turn conversation in testing. Real users run 60-turn sessions. Context fills up, older turns get truncated, and the agent loses track of earlier constraints the user set. The agent doesn’t tell the user it’s forgotten. It just starts behaving inconsistently. And that’s dangerous.

2. Episodic Memory (What Happened)

This is the log of past interactions - previous conversations, completed tasks, decisions made, outcomes observed. It’s stored externally and retrieved when relevant.

• What it’s good for: Cross-session continuity. “Last time we spoke, you were working on the credit risk model.” Personalisation. Audit trails in regulated environments.

• Implementation: Conversation summaries or raw turns get embedded and stored in a vector database. At the start of each new session, a semantic search retrieves the most relevant past episodes and injects them into context.

• Failure Pattern: Episodic memory grows without bound. A user with 200 past sessions generates thousands of stored chunks. Retrieval latency climbs. Relevance degrades because older episodes are returned alongside recent ones. Teams don’t notice until p99 latency starts causing timeout errors in production.

3. Semantic Memory (What Is Known)

This is factual, reference knowledge - product documentation, policy documents, regulatory guidance, knowledge base articles. It doesn’t change per user. It’s the shared ground truth the agent reasons from.

• What it’s good for: Answering questions from authoritative sources. Think everything about the business. Grounding the agent in facts rather than hallucination. RAG (Retrieval Augmented Generation) is almost always semantic memory.

• Implementation: Documents are chunked, embedded, and indexed. Queries are embedded and matched via similarity search. Top-K chunks are injected into context.

• Failure Pattern: Source documents change, the index doesn’t (this is very common). Nobody set up a re-indexing trigger. The agent retrieves a policy document from eight months ago with full confidence, because from the vector database’s perspective it’s still the most semantically similar result. In a regulated environment, this isn’t a UX problem. It’s a compliance problem with vulnerabilities that can cause huge penalties.

4. Procedural Memory (How to Act)

This is the agent’s knowledge of how to behave - which tools to call, in what order, under what conditions. It’s not retrieved at runtime in the same way the other types are. It’s encoded in the system prompt, in tool schemas, in fine-tuned weights, or in explicit workflow definitions.

• What it’s good for: Consistent behaviour across all users and sessions. If you want the agent to always verify identity before accessing account data, that’s procedural memory.

• Failure Pattern: Procedural memory encoded only in system prompts is brittle. Long system prompts get partially ignored as context fills. Behaviour drift emerges - the agent follows the procedure at the start of a session but deviates by turn 40. Teams don’t catch this because they test short sessions.

How These Four Types Behave Differently at Scale

At low volume , say hundreds of sessions per day, all four types can be bolted onto a single vector database and it mostly works. At production scale, i.e., tens of thousands of sessions, millions of stored episodes, documents updating continuously, each type starts failing in a distinct way.

In-context memory degrades with session length. The longer the conversation, the more context is consumed, and the more the model has to work with a truncated window. At scale, you need a context management strategy: summarise older turns, compress tool outputs, prioritise what stays in window.

Episodic memory degrades with corpus size. Retrieval latency grows as the episode store fills. Relevance drops as more episodes compete for the same top-K slots. At scale, you need a tiered storage strategy: hot storage for recent episodes (last 30 days), cold storage for older ones, with a recency-weighted retrieval model rather than pure semantic similarity.

Semantic memory degrades with document churn. The faster your source documents update, the faster your index goes stale. At scale, you need event-driven re-indexing triggered by document changes, not scheduled batch re-indexing that runs weekly and misses the update that happened on Tuesday.

Procedural memory degrades with task complexity. Simple, short tasks follow procedures reliably. Long, multi-step tasks in complex domains show drift. At scale, you need procedures encoded as explicit workflow graphs - not just prose in a system prompt - with checkpoints that verify the agent is still on the correct path.

Architecture Choices and Optimisation Strategies

For in-context memory:
A common pattern is to use a sliding window with summarisation. Keep the last N turns verbatim, summarise everything older into a compressed context block. If you use Claude Code - you will notice it does that in long conversations. It compresses conversations at one point. (Yes, Claude Code is an agent).

LLM-generated summaries work well here. For long-running agents, implement a context budget - allocate token limits per component (system instructions, retrieved memory, conversation history, tool outputs) and enforce them at the orchestration layer, not as an afterthought.

For episodic memory:
Separate your recent episode store from your historical episode store. Redis or a fast key-value store handles recent sessions; your vector database handles historical retrieval. I love Databricks LakeBase architecture here - LakeBase can act as the transactional store while the historical data is stored in Delta Lake. (I know Databricks plug, but it is usful to know if you are already using Databricks).

Apply recency decay to your similarity scores - an episode from yesterday should score higher than a semantically identical episode from six months ago, because context drifts. Build a summarisation pipeline that condenses completed sessions before storage, rather than storing raw turns.

For semantic memory:
Your chunking strategy is your most important decision - more important than your choice of vector database. Chunks need to be semantically coherent, not just fixed-size token windows. 512 tokens with sentence boundary respect and document-level metadata outperforms 2048-token sliding windows without metadata. Build a document change detection pipeline that triggers re-embedding on update. Track embedding model versions - if you upgrade your embedding model, you need to re-embed your entire corpus, because old and new vectors aren’t comparable.

For procedural memory:
Move away from text prose system prompts for complex procedures. Define agent behaviour as a state machine or directed graph - explicit states, explicit transitions, explicit conditions. This gives you auditability (you can log which state the agent was in at each step), testability (you can write unit tests for state transitions), and recoverability (you can restart a failed task from a known state rather than from the beginning).

You know what’s common across all four?
Each memory type needs its own evaluation pipeline.

Not just evaluation of the agent’s final output - evaluation of the memory layer itself. Did episodic retrieval return the right sessions? Did semantic retrieval return current documents? Did the agent follow the correct procedure? Did context management preserve the right information?

They need to be measured separately. And at production scale, not doing this could prove expensive, and risk reputational damage.

Now, tell me what kind of memory failure patterns you observed in your implementations. Give me feedback - tell me how this article helps you. I want to hear more from you.

If you love listening to experts, this podcast with Denis Rothman, author of Context Engineering for Multi-Agent Systems is very relevant. Denis is a thoughtleader and he describes complex topic in simple human way. You will like it.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

The fix for this isn’t a better model log. It’s a different structure that sits alongside it - what I’ve been calling an evaluation graph. Every time the AI runs, it takes a hashed, timestamped snapshot of every governance artefact in force at that exact moment: the glossary term version, the policy version, the dataset certification. So six months later, when the definition has moved from version 2.3 to 2.4, you’re not reconstructing anything. You look it up. The snapshot tells you exactly which soup the kitchen made that day, and who signed off on the recipe.

Evaluation Graph vs Context Graph

People keep confusing this with a context graph, which is genuinely a different thing. What the difference?

The evaluation graph is the receipt that proves what was actually served. It records: today’s menu version, which supplier the tomatoes came from, which health certification was active, what time the dish left the kitchen. Six months later, if someone asks “was this dish safe and compliant on that day,” you don’t guess, you pull the receipt and know exactly which version of “today’s soup” this was, signed off by whom. That’s provenance. It’s not about why the waiter brought you soup. It’s about proving what was true in the kitchen at that exact moment.

The context graph is the waiter’s reasoning for why you got soup at all. Maybe you said “I’m not very hungry,” the waiter remembered you’re allergic to shellfish from last visit, saw the bisque was off the board today anyway, and recommended the soup as the lightest, safest option. That whole chain - your stated preference, the remembered allergy, the menu constraint, the final recommendation is the decision lineage. If you later ask “why did you suggest soup,” the context graph is what lets the restaurant reconstruct that reasoning instead of shrugging and saying “the waiter just did.”

So: same dish, two completely different questions.

• Evaluation graph: “Prove what was actually in the bowl, and that it met standards, on that specific day.” (Provenance, was the input governed correctly?)

• Context graph: “Explain why the waiter chose this for you specifically.” (Lineage, why did the agent decide what it decided?)

A restaurant or a bank can have perfect receipts and a clueless waiter. Or a brilliant waiter with no receipts to back up a regulator's question. You need both, and they're solving for different failure modes.

What this looks like technically

Strip away the restaurant for a moment. Here’s the conceptual shape of both graphs, and the one thing that ties them together.

A context graph is built from three kinds of nodes.

• Entities: the customers, products, policies, and people your business already has some version of in a knowledge graph.

• Decision events: each time an agent or a human makes a choice, that choice becomes its own node, carrying the inputs, the policy applied, any exception made, and the outcome.

• Temporal state: because the graph needs to know not just what’s true now, but what was true at the moment the decision happened. Sarah was a Director in May; she’s a VP today. The graph needs to know which one mattered when.

An evaluation graph is built differently, because it’s answering a different question. Its core node is a context snapshot taken at the exact moment a model runs, hashed and timestamped, pointing to the specific versions of every governance artefact in force at that instant: the glossary definition, the policy version, the dataset certification. Alongside that sits the run itself - the model version, the prompt template, the retrieval configuration, and a failure taxonomy that lets you diagnose, after the fact, whether a wrong answer was a model failure, a context failure, or a policy gap.

The two graphs aren’t separate systems sitting in isolation. They connect through one shared reference: the run, or the decision event itself. Every decision event in the context graph that involved a model call can carry a pointer to the evaluation graph’s snapshot for that exact run.

When someone asks “why did the agent decide this,” you walk the context graph. When someone asks “what was governing the system when it decided that,” you follow the same run ID into the evaluation graph and pull the snapshot. One graph for the reasoning, one graph for the proof, joined at the single point where the AI actually executed.

Practically, this doesn’t require building either graph from scratch on a green field. If you’re already running something like MLflow or Databricks AI Agent Evaluation, the run node is just a thin wrapper around the run ID your platform already generates. The context snapshot capture can be a pre-run hook - a lightweight call to your catalog and glossary APIs that resolves the current versions of everything relevant, hashes them, and writes the snapshot before the model executes. The heavier lift is the context graph side, because that one has to live inside the execution path - you can’t capture a decision event you weren’t watching for when it happened.

What I’d actually do about it

These concepts are new and pretty early for many organisations. My advise is to be congnizant and ask questions for every AI build that passes through the architecture team. So, before your next model goes anywhere near a regulated workflow, ask one question out loud in the project meeting: if a regulator asks us to reproduce this exact answer in six months, what’s allowed to change underneath us between now and then?

Then write down every governed artefact that answer depends on. Glossary terms. Policy versions. Dataset certifications. For each one, name who owns it and how often it moves. If you can’t answer that for every item on the list, you don’t have an evaluation graph yet - you have a model log dressed up as proof.

That’s not a backlog item to schedule for next quarter. It’s the difference between “we can rerun the query” and “we can actually explain the decision.”

Where to go next

I’d genuinely like to know what you’re seeing on your side. Has anyone asked your team to reproduce a decision and found the same gap I’m describing? Is this something your governance and AI teams have already wired together, or is it still sitting in the gap between two calendars like it was at the bank above?

Hit reply or drop a comment. Tell me about the patterns you’re seeing, they usually end up shaping what I write next.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

A quick note before you watch: this was recorded in April 2026, and the pace of change in this space means a few things have already moved on. Databricks has since shipped more platform features that make some of these pillars easier to implement than I describe in the talk. My own thinking keeps shifting too, as I talk to more customers and fold new patterns into the framework. I’ll keep sharing those updates here and on the YouTube channel as they land.

Here’s what it covers.

The talk is called “The Production AI Playbook” and the core argument is this: most AI projects fail for the same five reasons, and almost none of them are about the model.

I open with a pattern most of you will recognise: the Week 1 to 14 doom loop. Teams pick a model, build features, demo to leadership, ship, and watch it fall apart within weeks because nobody built the infrastructure underneath it. Gartner puts enterprise AI project failure above 40%. I’ve watched it happen first-hand, more than once.

From there I walk through five pillars, each one a dependency for the next:
Evaluation,
Observability,
Data Foundation,
Orchestration,
Governance.

Two pillars get particular depth. On evaluation, I show the actual judge prompt structure we use for LLM-as-judge scoring, including the fix for non-determinism: running each test case three times and flagging anything with high score variance before it ships. On orchestration, I cover the patterns that hold up in production against the ones that only survive in demos, including failure modes nobody puts on slides: context window bleed, cascading failures, and trust boundary violations between agents.

The spine of the talk is a real case study: a retail bank, 18,000 calls a month, a prior attempt that burned $85,000 over six months with nothing shipped. The second attempt took eight weeks, and the model wasn’t chosen until week seven. Everything before that was evaluation and infrastructure. The result: 87% accuracy, 62% call deflection, and a tripled API-call bug caught in two hours that would otherwise have cost $43,200 a year in wasted fees.

I also include a section on what I’d do differently. Three things that surprised us even after the framework held up: the test case library needs a named owner or it quietly rots, prompt version logs need to capture intent and not just the diff, and behavioural evals cost far more to run at scale than most teams budget for.

If you’re building or running production AI, this one will be useful.

Here’s in the link to the resources I talk about in the video.

I aslo did an online talk in the same conference on Multi-Agent Orchestration Patterns, more tehcnical deep dive into patterns and failures modes when you build agentic workflows.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

The model did its job. What failed was everything around the model. The context. And that’s the shift worth getting your head around: the intelligence is largely a solved problem now, and the bottleneck has moved to context.

Prukalpa Sankar, Founder and Co-CEO of Atlan, puts it bluntly: with AI, context might be everything, because “the intelligence is already here.” The model is the easy part. The hard, durable, defensible part is whether your agents understand how your business actually works. That understanding is what people are starting to call context intelligence.

What context intelligence actually is

It isn’t a bigger prompt or a better retrieval setup. It’s the infrastructure that gives an agent shared, governed, current knowledge of your organisation, plus a memory of the decisions it and its predecessors have made. Two ingredients: a map of how things relate, and a record of why things happened.

That map and that record are the context graph. A context graph is a living model of your business as a set of entities and the relationships between them, this customer, that contract, this SLA, that exception, joined to the decisions taken against them. Foundation Capital, in their Context Graphs essay by Jaya Gupta and Ashu Garg, made the sharpest version of the argument: the durable asset isn’t the data an agent reads, it’s the decision trace it leaves behind. What was gathered, what rule applied, why an action was allowed. Capture that, and precedent becomes something an agent can look up instead of guess at. The agent stops having data with no judgement and starts having judgement.

Why this is the accuracy story

Most enterprise agents fail on a trust gap, not a model gap. The SLA agent didn’t need a cleverer model. It needed to know which contract was current, that it was allowed to act on it, and what had been decided in similar cases before. None of that lives in the model. All of it lives in the context.

Without it, you get what Prukalpa calls context sprawl: every agent building its own private, partial view of the world, none of them agreeing on what “active customer” even means. Fifty agents, fifty versions of the truth, no shared map. Accuracy in that environment isn’t a model property. It’s an infrastructure property.

This is where the runtime and the infrastructure get confused. Anthropic’s work on context engineering covers the runtime half: context is finite, and the discipline is fitting in the fewest high-value tokens at the moment of inference, not the most. That’s real, but it assumes the right context already exists to be selected. Context intelligence is the layer below it, the one that decides what context exists, whether it can be trusted, and whether the agent may use it. One is what you put in the window. The other is what’s available to put there at all.

How to think about building it

You can read most failures here as three debts coming due.

• Data Debt: no single governed source of what’s true, so your agents disagree.

• Decision Debt: nobody captured why past actions were taken, so the context graph has no memory.

• Evaluation Debt: no framework to check whether the context an agent actually used was the right context. Work out which one is biting and you know where to start.

The build order matters more than the architecture diagram. The instinct is to spend two years plumbing every system into a perfect context layer before anything ships. Prukalpa’s advice is the opposite, and it’s right: bootstrap from the systems you already have, the CRM, the ERP, the BI definitions, get the context layer roughly 80% of the way there, and let the flywheel start turning. Every decision an agent makes then becomes institutional memory the next agent inherits. You don’t design the context graph up front. You grow it.

Which is why this isn’t really an AI problem at heart. It’s the next turn of data engineering: context as a governed product, with owners, versions, and tests, sitting between your data and your agents. The teams who treated data as a product a decade ago have a head start. The ones still treating context as something you cram into a prompt are about to learn the difference in production.

The open question, the one nobody at the table has a clean answer to yet, is who inside the enterprise actually owns this layer.

I got into exactly that with Prukalpa Sankar on the podcast. She’s been arguing for the context layer longer than almost anyone, and it’s the clearest thinking I’ve heard on where this is heading.

Also, read Prukalpa’s article - What an Enterprise Context Layer Actually Is

Enjoy your weekend.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Part 1 covered Schema and Ownership: the description and accountability layer of an Agent Registry. Unity Catalog as the agent registry. AWS IAM for per-agent identity. CloudTrail for audit trails. A stack that answers the questions an audit demands: what exists, who owns it, what it can do.

That is necessary, but it is not sufficient.

Schema and Ownership describe an agent. They do not constrain it. An agent with a well-documented schema and a clear owner can still make a consequential decision without human approval, access data it should not touch, or operate at an autonomy level the business never signed off on. That is where Risk Class and Autonomy Level come in and where the registry becomes a governance control rather than a filing system.

Component 3: Risk Class - Not All Agents Carry Equal Risk

Risk classification is the decision that determines what governance an agent receives. If you get it wrong and you either over-govern low-risk agents until teams route around the process, or under-govern high-risk agents until something goes wrong in production.

The four risk categories worth working with in a regulated environment:

1. Low Risk: bounded output, read-only, no consequential action. An agent that summarises documents or answers questions from a knowledge base.

2. Medium Risk: supervised action. The agent can write or update records, but a human reviews before execution.

3. High Risk: consequential action inside a tightly constrained boundary. The agent can execute without per-action approval, but only against approved tools, approved data paths, and fully audited systems.

4. Critical Risk: autonomous consequential action across multiple systems, with downstream effects that are difficult or impossible to reverse.

In Databricks, risk class is best treated as governed metadata attached to the agent assets you register. For example, on Unity Catalog models, model versions, and functions, all of which support tagging. That makes risk class visible, queryable, and reviewable across the lifecycle. If an agent gains new tools, new data access, or a broader execution boundary, the classification should be reassessed before the next promotion.

So, if you ar eon Databricks, use Unity Catalog to register the agent, its model versions, its functions, and its metadata. Use tags to record risk class. Use Unity Catalog privileges, managed authentication, and deployment-time permission checks to enforce which tools and data paths are actually reachable at runtime. If you need an explicit risk-policy engine that says, for example, a High Risk agent may call Tool A but never Tool B, that policy layer still sits outside Databricks - in an API gateway, middleware layer, or external authoriser.

That is the line between registry and policy engine. Databricks gives you the governed assets, the permissions model, and the audit surface. If you need deterministic policy decisions over agent behaviour itself, you compose that on top.

On the AWS-native path: Amazon Bedrock AgentCore Gateway with Cedar policies provides a similar enforcement pattern - deterministic allow/deny decisions on every tool call, with Lambda interceptors for dynamic validation. Cedar policies are authored in a declarative language and evaluated against principal, action, and resource with optional conditions over request context. Worth monitoring as it moves towards general availability.

Component 4: Autonomy Level - How Independently an Agent Operates

Risk class tells you what an agent is allowed to do. Autonomy level tells you how much independent judgement it is permitted to exercise in doing it. These are related but distinct. A High Risk agent can still operate at L1 with human approval, or at L3 with bounded multi-step execution. The risk class constrains the action space. The autonomy level constrains the operating model.

The five levels from the registry:

• L0 - Assistive only. Generates output for human review. No execution.

• L1 - Human-approved actions. The agent proposes; a human confirms before execution.

• L2 - Semi-autonomous workflows. Executes within defined boundaries without per-action approval.

• L3 - Goal-driven execution. Plans and executes multi-step tasks. Human oversight at checkpoints, not per action.

• L4 - Multi-agent autonomy. Coordinates with other agents, spawns sub-agents, orchestrates complex workflows with minimal human intervention.

In Databricks, autonomy level should be treated as an explicit registry attribute you manage in your own governance model, not as a built-in Unity Catalog field with native enforcement semantics. The platform gives you the places to record it, such as model and model-version metadata, but the meaning of L0 through L4 remains an operating policy you define and then enforce through workflow design, approvals, deployment controls, and runtime boundaries.

MLflow Tracing captures the full execution path - every tool call, every decision point, every input and output - linked to the agent version and autonomy level at the time of execution. In a regulatory investigation, that trace is the evidence.

So, autonomy level is declared in the registry, promotion is gated through MLflow deployment workflows, runtime execution is bounded by Databricks permissions and isolated tool execution, and trace evidence is captured through MLflow. The registry records the autonomy decision; the surrounding control plane makes it real.

Open source alternative: Microsoft released the Agent Governance Toolkit in April 2026 under MIT licence, an open source project that addresses runtime security governance for autonomous agents. It includes execution rings inspired by CPU privilege levels, kill switches for emergency agent termination, and circuit breakers. It maps directly to the OWASP Top 10 for Agentic Applications (2026) and is the most purpose-built open source option currently available for autonomy-level enforcement.

What the Registry Now Does

Across both parts, the four components work as a single control system.

Schema tells the registry what the agent is. Ownership tells it who is accountable. Risk Class tells it what level of control the agent requires, and that classification is carried as governed metadata across the agent’s registered assets. Autonomy Level tells it how much independent judgement the agent is permitted to exercise, and that decision is enforced through release workflows, runtime permissions, and execution boundaries rather than left implicit in code.

The audit request scenario from Part 1 - fourteen agents, nobody can answer, is no longer possible. Every agent has a schema record, an owner, a declared risk class, an explicit autonomy level, governed access to tools and data, and an execution trail that can be reconstructed through MLflow and platform audit logs. The registry is not a reporting artefact. It is the control plane that makes autonomous operation in a regulated environment defensible.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Talk soon,
Sandi.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

An organisation deploys 50 agents across 6 teams over the course of a year. Without any central register, wihtout any ownership mapping, wihtout any shared schema.

Then the audit request arrives.

They ask the following questions:

• Which agents are live?

• Who approved the one touching customer financial data?

• What has it been told in its system prompt?
  

Nobody could answer.

This is not a technology failure. The agents worked fine but the governance infrastructure was never built.

A Note on the choice of technology

Before we process, I want to make a note on my technology choices here. The architecture in this issue is Databricks-on-AWS. That is not a neutral choice, but it reflects where I spend most of my time. I work with Tier 1 UK financial institutions, and the patterns I see repeatedly across those engagements have shaped this stack.

Databricks gives me Unity Catalog for governance, lineage, and access control across the full data and AI estate. AWS gives me IAM for identity and CloudTrail for audit trails. These tools are production-stable, work in hybrid deployments, and hold up under compliance scrutiny. Where open source alternatives exist and are worth knowing about, I have noted them inline.

Where open source alternatives exist and are worth knowing about, I have noted them inline. But the primary recommendation here is the stack I have seen hold up under real compliance scrutiny.

What an Agent Registry Actually Is

An Agent Registry is the control layer that makes an agent estate governable. It is not a dashboard. It is not a catalog you update manually. It is infrastructure, the layer that sits beneath your agents and answers the questions an audit demands: what exists, who owns it, what it can do, and how much autonomy it operates with.

It has four components. This issue covers the first two: Schema and Ownership. Part 2, out on Wednesday, covers Risk Class and Autonomy Level - the enforcement layer.

I am splitting this deliberately.

• Schema and Ownership are about description and accountability - what an agent is and who controls it.

• Risk Class and Autonomy Level are about enforcement - what an agent is allowed to do and what stops it.

These are different engineering problems. Collapsing them into one piece does neither justice.

Component 1: Schema — How an Agent Describes Itself

Schema is the agent’s self-declaration. Without it, agents are opaque. Orchestration breaks. Integration fails silently. You cannot route work to an agent you cannot describe.

A schema record needs to capture: capabilities and skills, the APIs and tools the agent can call, input and output formats, memory and context handling behaviour, access permissions, and communication protocols.

Unity Catalog (UC) is the agent registry in this stack. It governs agent tools as registered, versioned functions - each one a securable object with access control, lineage tracking, and metadata surfaced through Catalog Explorer or the REST API. It supports attribute-based access control on tags, which means you can attach metadata directly to agent capability records and build access policies around them. If you think about it, UC becomes the same governance layer that already covers your data assets, your ML models, and your pipelines, just extended to agents without adding a separate system.

Unity Catalog also works across Databricks workspaces on AWS, Azure, and GCP. If your agent estate spans cloud environments, governance travels with it through the same control plane. That’s powerful.

AWS AgentCore Agent Registry is worth knowing about as a complementary discovery layer. It stores agent records across frameworks and clouds and supports semantic search and approval workflows - and is useful if your estate includes agents running entirely outside Databricks and you need a single cross-platform catalogue. It is a discoverability tool, not a governance layer. For most Databricks-on-AWS deployments, Unity Catalog covers the registry function without it.

Open source alternative: purpose-built open source agent registry tooling is immature right now. The practical path is a lightweight service catalogue like Backstage for discoverability combined with OPA for policy enforcement - neither of which was designed for agents, but both of which work today without significant custom engineering. I have found this GitHub Repo, I never tried it or even explored it - might you might want to have a look.

If you find one or know one - let me know in comments.

Component 2: Ownership — Who Controls the Agent

Ownership is where shadow AI comes from. An agent without an owner is an agent nobody is responsible for. In regulated environments, that is not an abstract risk.

An ownership record needs to capture: the team or business owner, accountability mapping, the access governance model, the escalation path, approval authority, and lifecycle responsibility - including who decommissions the agent and when.

On the identity side, AWS IAM is the enforcement mechanism. Each agent should operate under a dedicated IAM role with least-privilege permissions - no shared credentials, no roles that accumulate access over time. IAM Policy Autopilot, allows AI coding tools to generate baseline IAM policies directly from application code, reducing the gap between what an agent was built to do and the permissions it actually holds.

Accountability trails run through AWS CloudTrail. Every registry access and administrative action is logged. In AgentCore Agent Registry, CloudTrail integration is built in, meaning you have an auditable record of who approved an agent, when it was registered, and when its record was last modified.

Unity Catalog extends this further. Audit logs capture every agent action. Unity AI Gateway release introduced MCP server governance - controlling which agents can access which external systems and tracking how that data is used. Ownership in the registry is not just a field in a database. It maps directly to the IAM role, the Unity Catalog access policy, and the audit trail.

Open source alternative: Open Policy Agent (OPA) provides policy enforcement for ownership and access governance in environments not running on the Databricks or AWS managed stack. It is cloud-agnostic and widely used in regulated industries.

If you find good open-source stack please comment here. I am looking for them as well.

Coming Wednesday: Risk Class and Autonomy Level

Schema and Ownership tell you what an agent is and who is responsible for it.

That is necessary but not sufficient.

The harder question is what an agent is allowed to do and what stops it when it operates outside its boundaries.

Part 2 covers Risk Class: how risk classification moves from a metadata tag in Unity Catalog to a live runtime control via Unity AI Gateway and AgentCore Policy. I will also cover Autonomy Level, the scale from assistive-only (L0) to multi-agent autonomy (L4), and the kill switch infrastructure that makes higher autonomy levels safe enough to deploy in a regulated environment.

This is the part of the registry most organisations skip entirely. It is also the part that determines whether your governance is real or decorative.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

Mnay organisations I talk to gathers 50-100 AI use cases. They hold a big “priority workshop.” They vote on business value, strategic fit, and stakeholder excitement. They pick the top three, hand them to the engineering team, and wait for the magic to happen.

Six months later, the project is quietly shelved.

The reason? “The data infrastructure doesn’t support it.”

In my latest video, I talk about why this happens and more importantly, how to stop it using a framework I’ve been developing for production AI.

The “Human-to-Agent” Gap

The core problem is something I call the “Human vs. Agent” data standard.

As technical people, we often forget that data built for a human analyst is completely different from data built for an AI agent.

• Humans can reconcile messy schemas and ask a colleague for help.

• AI Agents cannot. They either fail, or worse, they fail silently.

If your infrastructure is built for BI dashboards, it simply won’t sustain a fully autonomous AI agent.

The 8-Dimension Assessment

To fix this, I’ve broken down AI readiness into 8 key dimensions across two groups:

1. Data Infrastructure: Consumers, Access Latency, Schema Rigor, and Metadata.

2. AI Operations: Error Handling, Memory/State, Evaluation, and Observability.

The goal isn’t to be “perfect” at all of them. The goal is to be honest. When you score your current setup (1 to 4) against what a use case actually requires, the “readiness” of your project stops being a debate and starts being a number.

Finding Your “Beachhead”

The most actionable part of this framework is finding your Beachhead Use Case.

A beachhead isn’t your most ambitious goal (like a fully autonomous loan negotiator). It’s the use case where:

1. The infrastructure gap is small (you can ship in weeks, not months).

2. The business value is standalone (it pays for itself immediately).

By starting here, you aren’t just “doing a pilot” - you’re building the foundation that makes the ambitious stuff possible later.

Want to run this assessment yourself?

I’ve put together a full walkthrough of these 8 dimensions and how to build your own “Gap Map.” If you’re tired of the “AI priority workshops” that lead nowhere, this might be the most useful 15 minutes of your week.

Check out the full breakdown here.

I’d love to hear your thoughts—drop a comment on the video and let me know which of the 8 dimensions is currently the biggest blocker in your org.

Have a great weekend,

Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

• MCP is the right abstraction for standardising tool access across multiple agents, but most teams are deploying it without the governance it needs

• Five failure modes keep appearing in regulated environments: hardcoded credentials, no authorisation layer between model and tool, invisible tool calls, server sprawl, and untracked data residency

• MCP is not always the right choice. Direct function calling, existing APIs, and async queues are better fits for several common patterns

• The fix isn’t complicated: treat MCP servers as infrastructure, instrument every tool call at the boundary, and never let the model be your policy engine

MCP is at that USB moment

Something shifted in 2024. Developers started asking, “can we connect an LLM to our tools?” and then wondering, “how do we do it without building a different integration for every model, every framework, every team?”

These are the questions MCP answers. The Model Context Protocol, originally developed by Anthropic and released in November 2024, and now gaining ground as a de facto standard.

By mid-2025, MCP had moved from research-adjacent to actively deployed. GitHub Copilot, Cursor, Claude, and a growing list of enterprise agent frameworks had either adopted or announced support. The server ecosystem - registries, SDKs in Python and TypeScript, community-contributed connectors for everything from PostgreSQL to Salesforce had expanded fast. Almost every other product has its MCP.

That speed should give you pause.

This is a pattern we’ve seen before. When USB was introduced in 1996, it solved a real problem: a dozen incompatible port types on the back of every PC. It was the right abstraction. And within a few years, “plug and play” had become a running joke because the driver ecosystem moved faster than the discipline around it. Devices connected. Systems crashed. Enterprise IT spent years cleaning up what consumer enthusiasm had shipped. While the protocol was fine, the deployment culture was not.

MCP is at that USB moment. The abstraction is right. The ecosystem is moving faster than the engineering rigour around it. Teams are shipping MCP servers in sprints, demoing them to CTOs, and having them in production eight weeks later. Six months after that, nobody can tell you what tools the agent is calling, the credentials haven’t been rotated since go-live, and there is no audit trail that would survive a compliance review.

The protocol didn’t fail them. The deployment pattern did.

This article is about the difference.

Tool Definition: A Contract with Non-Determinism

The fundamental difference between an MCP tool and a standard API is that it is a contract between a traditional, deterministic backend system and a non-deterministic LLM agent. Standard API engineering assumes a consumer will call a function exactly as documented. Agent tooling requires developers to accept that the model will interpret the description and choose the arguments. This non-deterministic usage is why building for agents requires a higher degree of protective rigor around entitlement, validation, and audit than standard integration patterns.

What MCP gets right?

MCP is solving a real problem, and the core of it is genuinely well-designed.

The integration tax is real, and MCP eliminates it

Before MCP, connecting an agent to a tool meant writing bespoke integration code. Every model had its own function-calling format. Every framework had its own abstraction layer. If you wanted to switch from one agent framework to another, you rebuilt your integrations. If you wanted the same capability accessible across multiple agents, you duplicated the logic and prayed for consistency.

Without a shared protocol, every team builds its own integration in the shape of its own constraints - its available libraries, its preferred auth pattern, its interpretation of what the downstream system needs. The result is integration sprawl that compounds with every new team that touches the same system. MCP breaks that coupling. A single MCP server exposes a set of tools, resources, and prompts under a standardised interface. Any MCP-compatible client regardless of which model or framework it uses, can discover and invoke those tools through the same protocol. Write once, expose everywhere.

This is what happened when REST became the default for web APIs in the early 2000s, displacing the chaos of SOAP, WSDL, and proprietary RPC formats. REST won because it was simple enough that teams could independently build to the same standard and have things actually work. MCP is attempting the same move at the agent-tool layer.

Runtime capability discovery changes the architecture

One of the underappreciated features of MCP is the tools/list endpoint. Think of it like DNS for tools - you don’t hardcode IP addresses into your application, you resolve them at runtime. An agent queries the server on startup, gets a schema-described list of available tools, and decides which to invoke based on the task. Your agent architecture can evolve without redeployment every time a tool is added or changed.

Where it genuinely shines

MCP is strongest when the problem is standardised access to well-defined internal systems. Four use cases stand out:

• Internal tool registries. A platform team builds and owns MCP servers for canonical internal capabilities - search, data retrieval, workflow triggers. Agent teams consume them without needing to understand the underlying integration. The boundary is clean, the ownership is clear, and the interface is versioned. This is the internal developer platform model applied to agent tooling.

• Governed data catalogue access. An agent needs to query dataset metadata, lineage, or schema. Exposing a data catalog (Unity Catalog, Alation, Collibra, DataHub) via MCP gives the agent a structured, permissioned interface without direct database access. The server enforces what the agent can see. This matters enormously in financial services, where an agent browsing raw schema can inadvertently surface data it has no business touching.

• Regulated workflow triggers. An agent initiates a downstream process - raises a ticket, submits a form, triggers a notification. MCP provides a typed, auditable interface for those triggers. The tool schema documents exactly what inputs are required; the server enforces them. The schema is the contract.

• Multi-agent orchestration. A supervisor agent delegates to specialist subagents, each with its own MCP server exposing its capabilities. The supervisor discovers what each subagent can do and orchestrates accordingly. This is where capability discovery really earns its keep; it makes composition between agents tractable without tight coupling.

• Tool Design for LLM Token Efficiency. The effectiveness of a tool is measured by its use within the LLM's context window. Prioritize tool interfaces that minimize the total volume of tokens consumed. For instance, prefer a search_datasets(query: str) tool to a generic list_all_datasets() tool. Furthermore, design tools with parameters for pagination and truncation to ensure the tool output - the data returned to the LLM is as concise and high-signal as possible. Refer to Anthropic’s blog: Writing effective tools for agents — with agents

A minimal MCP server for a governed data catalog wraps your catalog API behind a typed search_datasets tool enforcing domain filters, capping result limits, and keeping the agent away from raw schema access entirely. The tool definition is the contract; the server enforces it.

This is MCP doing what it’s designed for.

What risks MCP brings?

MCP moves fast to production. The failure modes tend to follow shortly after because the protocol makes it easy to ship something that works in a POC before the operational questions have been answered.

These are the five patterns you should understand.

Developers often forget “credential best practices”

Hardcoded credentials are not a new problem. What MCP changes is the rate at which new service boundaries get created. Spinning up an MCP server takes minutes - a few lines of Python, a decorator, done. That speed means teams are creating new integration points faster than their credential management habits have been built to handle. The result is more hardcoded credentials, in more places, with less visibility than traditional integration patterns would produce.

In a UK financial institution, this creates obligations that a missing credential policy would directly fail to meet under the Digital Operational Resilience Act (DORA). A credential with no owner and no rotation policy fails that bar.

DO NOT DO THIS: hardcoded credential in MCP server

DATABASE_URL = “postgresql://svc_agent:SuperSecret123@prod-db:5432/customers”

The credential is now wherever this server runs. If the server is containerised, the credential is in the image or the environment. If the image is pushed to a registry, it may be in the layer history. The blast radius of a compromise is the entire downstream system, not just the agent. This is the digital equivalent of writing your vault combination on a Post-it and sticking it to the outside of the vault.

Models usually have access to every tool in an MCP server - this is not least priviledge

MCP puts tool invocation decisions in the hands of the model. The model reads the tool schemas, decides which tool to call, and constructs the arguments. Nothing in the base protocol validates whether that decision was appropriate, whether the arguments are safe, or whether the calling agent had the entitlement to invoke that tool for that user in that context.

This is Saltzer and Schroeder’s principle of least privilege - articulated in their 1975 paper “The Protection of Information in Computer Systems” and still the foundation of access control design.

The principle states that every component should operate with only the permissions it actually needs. A model that has access to every tool in an MCP server, for every user, at all times, is a maximal privilege configuration. It is the opposite of least privilege.

In regulated industries, this matters concretely. An agent that can invoke a transfer_funds or update_credit_limit tool should not be making that invocation based solely on what the model infers from a user message. The trust chain is broken.

python

# Dangerous: model output routes directly to tool execution

async def run_agent(user_message: str, user_id: str):

    response = anthropic_client.messages.create(

        model=”claude-sonnet-4-20250514”,

        tools=mcp_tools,  # All tools. No entitlement check. No context.

        messages=[{”role”: “user”, “content”: user_message}]

    )

    for block in response.content:

        if block.type == “tool_use”:

            # The model decided. The server executes. Nothing in between.

            result = await mcp_session.call_tool(block.name, block.input)

This code grants the model access to all tools (tools=mcp_tools), establishing a maximal privilege configuration. The model’s tool\_use block is immediately trusted as the final decision, bypassing any policy or validation check. Execution proceeds directly to mcp\_session.call\_tool without confirming the user’s entitlement or context.

The model is a reasoning engine. It is not a policy engine. These are different things, and conflating them is how you end up with agents doing things nobody authorised them to do.

MCP invocations still require explicit instrumentation to be traced end‑to‑end

Any RPC to a separate process requires explicit instrumentation to appear in your trace - that is not unique to MCP. What makes MCP different is that the protocol is new enough that most observability platforms have no native integration for it yet. With a mature HTTP or gRPC stack, there is a reasonable chance your tracing library auto-instruments at the transport layer. With MCP, there is not. Teams adopting it now are on their own.

An MCP tool call is an RPC to a separate process. Without explicit instrumentation, that call disappears from your trace. You can see the model’s input and output. You cannot see which tool was called, with what arguments, what the server returned, how long it took, or whether it failed. This is the observability equivalent of a black box flight recorder that stops recording five minutes before the crash. You have most of the data. You’re missing exactly the part that matters.

For regulated deployments, this is an audit problem. The FCA’s Senior Managers and Certification Regime (SM&CR) creates personal accountability for outcomes. If an agent made a decision that affected a customer - a credit flag, a document retrieval, a workflow trigger - and that decision was influenced by a tool call, you need to reconstruct exactly what the tool returned. If the tool call isn’t in your trace, you cannot reconstruct it. “The model did it” is not an explanation that satisfies a regulator.

The fix lies in instrumentation at the MCP boundary, not inside the server. This is shown in the MCPGateway pattern below.

Every team builds one, no one owns them - there is no established governance for MCP servers

In a large organisation, the absence of a shared standard is a vacuum that teams fill independently - each making a locally rational decision that creates a globally irrational system.

The MCP equivalent: without governance, every team designs its own simple system. The risk team builds an MCP server for their data warehouse. The finance team builds a different MCP server for the same data warehouse with different auth. The platform team builds a third server that partially overlaps with both. The agent now sees twelve tools that do variations of the same thing. The people who built them have moved on. No deprecation path exists. This is not a hypothetical - it is the same pattern that played out with internal REST APIs at most large organisations that adopted microservices without a service catalogue, and it is already starting to repeat with MCP.

The structural cause is Hyrum’s Law: observed by Google engineer Hyrum Wright and now widely referenced in software engineering: “With a sufficient number of users of an API, it does not matter what you promise in the contract — all observable behaviours of your system will be depended on by somebody.”

Once a team starts using your MCP server, they will depend on its undocumented behaviours. Deprecating it without governance becomes painful very quickly.

Treating MCP servers as infrastructure from the first deployment - with an owner, a version, a changelog, and a deprecation policy is not bureaucracy. It is the thing that lets you move fast in two years without digging out from under your own sprawl.

MCP doesn’t answer the data residency question

MCP doesn't answer the data residency question, and neither should it, but because MCP makes it trivially easy to spin up a new server anywhere, teams are creating new data residency exposure points faster than they're tracking them.

This is the failure mode that almost never appears in ecosystem documentation and is the one most likely to cause a material incident in regulated industries.

When an agent calls an MCP tool, data flows in both directions: the arguments sent to the tool, and the response returned. In a regulated context, both can contain customer data, personally identifiable information, or material non-public information. The question of where that data flows - which process handles it, which logs capture it, which jurisdiction it transits through - is not answered by the protocol. That is not a criticism of MCP. It is simply a boundary you need to understand.

The MCP server is a process. That process can run anywhere. If it runs in a container in a region that is not approved for the data it is handling, you have a data residency violation before the tool even returns a result. If the tool call arguments are logged by intermediary infrastructure before reaching your server, you have a data handling question that needs a documented answer.

For UK firms post-Brexit, this intersects with UK GDPR, FCA data governance expectations, and potentially the location requirements of your outsourcing arrangements. The compliance question is not “does MCP support data residency?” - it does not operate at that layer - but “can you trace every byte of this tool call, confirm where it went, and show it never left an approved boundary?”

That answer requires mapping the full data flow before deploying any MCP server that touches regulated data. Not after the first incident.

Do you actually need MCP?

I see it commonly - every team jumps to the conclusion of using MCP very fast. MCP is infrastructure and infrastructure has a cost - operational overhead, governance burden, complexity. The Unix philosophy, articulated by Doug McIlroy in the early 1970s, puts it plainly: “Write programs that do one thing and do it well.” Before you introduce a protocol for interoperability, ask whether you actually need interoperability.

Here is a decision tree I like to use to help teams make that decision:

Let’s make each branch concrete.

• When direct function calling is the right answer

  If you have a single agent, a small number of tools, and no requirement for reuse across teams or frameworks, native function calling in the Anthropic SDK is simpler, cheaper, and easier to observe. No additional process boundary. No MCP server to maintain. No capability discovery overhead. Adding an MCP server here is the software equivalent of installing industrial plumbing to fill a kettle.
  

• When a well-governed existing API beats MCP

  If the downstream system already has a REST or gRPC API with proper authentication, rate limiting, observability, and documentation - a Salesforce API, an internal risk platform, a data catalogue with its own REST interface, wrapping it in an MCP server often adds a layer without adding value.
  

  The test is simple: does the MCP layer provide something the existing API does not? If the answer is capability discovery for agent consumption, standardised schema, or unified access across multiple systems, MCP earns its keep. If the answer is “it’s just a wrapper,” you’ve added a process boundary, a deployment artefact, and an operational dependency for no functional gain.
  

• When an async pattern is the right architecture

  MCP is synchronous request-response. The agent calls a tool, blocks, and waits. That is fine for fast, bounded operations. It is the wrong shape for long-running jobs (submit and poll, not block), audit-required fire-and-forget (a message queue with dead-letter handling gives you durability and replay that MCP cannot), and event-driven workflows where the agent should be consuming from a stream, not polling in a loop.

MCP solves a specific problem well. The mistake is treating it as the default integration pattern for anything agent-related, rather than the right answer to a specific architectural question.

One pattern that works

Let me show you an architecture pattern to address these failure modes above. First, let’s understand the few best practice that matter here. It a bit of recap, but worth a refresh.

1. MCP servers are infrastructure, not glue code. You need an owner, you need versioning, changelog, deprecation policy. It should be registered in your internal catalogue and deployed through the same pipeline as your other services.
   
   “Who owns this MCP server?” should have a human name attached to it.

2. Every tool call must be observable. Instrument at the boundary - in the layer between the orchestrator and the MCP session - not inside the server. The instrumentation wrapper is shared infrastructure, not something each team spends time re-implementing.

3. The model never touches credentials or entitlements directly. There is always an authorisation layer between the model’s invocation decision and the tool’s execution. This is not optional in a regulated environment. It is least privilege applied at the agent layer.

4. Credentials are runtime injection, not baked-in secrets. Retrieve credentials at server startup via your secrets manager (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault) - never at image build time, never from environment variables baked into a container. The credential is never in your code, never in your image layer history, and has a documented owner and rotation policy. Rotation is handled by the secrets manager; the server picks up new credentials on the next startup cycle.

5. Define quality before deployment using Evals. The observability focus in production must be preceded by rigorous quality assurance during development. And these ar enot unit tests - you need systematic measurement of the LLM's ability to use the tool correctly. Advocate for comprehensive evaluation tasks grounded in complex, real-world scenarios to measure tool efficacy. For debugging, run these evaluations programmatically, instructing agents to output their reasoning steps alongside the tool invocation. This practice helps developers probe exactly why an LLM selects or struggles with specific tools, ensuring the quality of the non-deterministic contract before it is exposed to regulated production environments.

Every tool call passes through the Guardrail Layer before it gets anywhere near the MCP server. That layer validates tokens, masks PII, fetches credentials from the Secrets Manager at runtime, and writes an immutable audit entry before execution starts. The MCP server receives a clean, credentialled, traced call. The response comes back, gets sanitised, and the trace span closes. The user gets a safe payload. Nothing touches the downstream system without a paper trail.

Conclusion

MCP is the right abstraction at the right layer. The standardisation problem it solves is real, and the use cases where it works well are genuinely valuable enterprise problems. The protocol is not the issue.

What history tells us from USB to REST to microservices is that good protocols get adopted faster than the discipline to deploy them safely.

That gap is where incidents come from. This is commonly described as the difference between a sharp knife and a blunt one: the sharp knife is more dangerous in the wrong hands, but it’s the right tool for someone who knows what they’re doing.

The teams that get this right usually ask the boring questions first - who owns this, where does the data go, what happens when this call fails at 2am - and build systems accordingly.

That’s not caution. That’s just engineering.

Using or building with MCP?

Don't let speed compromise security. Apply the governance and architectural rigor outlined here to your MCP servers now. Build your Authorization and Observability Layer first to ensure your tool calls are secure, auditable, and compliant from day one.

Tell me in comments whether this resonates, what other challenges are you facing, and where did MCP do the magic for you. I am eager to learn from your experience - so please comment, leave a feedback.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

But as teams push these systems into production, they are hitting a wall. Having five smart agents does not automatically equal one smart system.

The industry is quickly learning a hard lesson: orchestration is an understood problem, and communication reliability is the actual bottleneck . You can easily instantiate ten agents using frameworks like LangGraph, AutoGen, or CrewAI. But getting them to talk to each other reliably without hallucinating payloads, dropping context, or getting stuck in infinite loops is where the real engineering happens.

Let’s break down the first principles of agent communication, look at how they fail in production, and explore how modern production systems are solving these exact problems.

The First Principles of Agent Communication

At its core, getting agents to collaborate requires the same fundamentals as distributed computing, but with a chaotic twist: the “nodes” in this network are non-deterministic text engines.

1. Message Passing

Principle: Message passing is the transfer of information from one node to another. When Agent A finishes its job, it must hand off a payload to Agent B to trigger the next step.

• Failure Mode: The Hallucinated Payload.

• Example: A Data Extraction Agent is told to pull a user’s ID and pass it to a Database Agent. Instead of passing 12345, the agent passes, “Here is the user ID you requested: 12345.” The Database Agent expects an integer, receives a conversational string, and crashes.

  

• Why it happens: LLMs are fine-tuned to be helpful conversationalists, not strict state machines. Without hard constraints, they inject pleasantries and markdown formatting into their outputs, corrupting the message payload.

2. Protocols and Interfaces - H2A, A2C, A2A

Principle: Protocols dictate the rules of engagement. In modern agentic systems, we categorize these interfaces into three distinct buckets:

• H2A (Human-to-Agent): Conversational, unstructured, and forgiving (e.g., ChatGPT).

• A2C (Agent-to-Computer): Rigid and deterministic. The industry standard here is the Model Context Protocol (MCP). Introduced by Anthropic in late 2024 and now hosted under the Linux Foundation, MCP standardizes how agents securely connect to external tools, IDEs, and databases.

• A2A (Agent-to-Agent): Peer-to-peer communication between two non-deterministic models. This is historically the most fragmented layer, but the industry recently coalesced around the Agent-to-Agent (A2A) Protocol. Originally released by Google in April 2025 and unified with IBM’s Agent Communication Protocol (ACP), A2A is now a Linux Foundation open standard for cross-framework agent discovery and task delegation over HTTP and JSON-RPC.

• Failure Mode: Interface Confusion.

• Example: A developer uses MCP to perfectly connect a Research Agent to a PostgreSQL database (A2C). But when the Research Agent hands the data to a Writer Agent (A2A), the developer lets them communicate in conversational English. The Writer Agent misinterprets the unstructured text and hallucinates missing facts.

• Why it happens: Developers often treat A2A communication like H2A communication. Unless you enforce machine-readable protocols for peer-to-agent handoffs, conversational drift will inevitably break your architecture.

3. Shared Context and State

Principle: Agents need a shared understanding of the environment, current progress, and available data to collaborate effectively.

• Failure Mode: Context Desynchronization.

• Real-World Example: A Researcher Agent analyzes a 50-page PDF and passes a brief outline to a Writer Agent. The Writer tries to draft the article but fabricates details because it lacks access to the source material.

• Why it happens: Context windows are expensive. To save tokens and latency, builders often restrict the context passed downstream. This creates asymmetric information - Agent A knows something Agent B doesn’t, leading to poor decisions.

4. Intent Alignment

Principle: Every agent in the chain must understand the overarching goal of the user, not just its localized sub-task, to ensure the final output is cohesive.

• Failure Mode: The Telephone Game.

• Example: A user asks for a “brief, humorous summary of the latest AI news.” The Manager Agent passes the news to the Summarizer Agent but forgets to include the “humorous” instruction. The Summarizer writes a dry academic brief.

• Why it happens: Information decays across hops. When breaking a complex prompt into smaller agentic tasks, the nuance of the original user prompt is easily lost in translation.
  

  

5. Memory and Feedback Loops

Principle: When Agent B rejects Agent A’s work, Agent A needs memory of the failure and the ability to correct itself without repeating the exact same mistake.

• Failure Mode: The Infinite Death Spiral.

• Example: A Coding Agent writes a Python script. The Execution Agent runs it, encounters a SyntaxError, and passes the error back. The Coder apologizes, generates the exact same code, and sends it back. They repeat this loop 50 times until the API budget is drained.

• Why it happens: LLMs are highly sensitive to their immediate context. If the feedback isn’t explicit, or if the model’s internal weights heavily favor a flawed syntax pattern, it will deterministically generate the same wrong answer.

How To Solve This - Patterns That Are Working

Building reliable multi-agent systems requires shifting your mindset from “prompt engineering” to “protocol engineering.” Here is how top-tier engineering teams are building resilience into their agent networks.

1. Enforce Structured Outputs (a.k.a Strict Schemas)

Never let agents talk to each other in free-text prose if they are exchanging data. Treat agent communication exactly like an API. Use tools like OpenAI’s Structured Outputs, Pydantic, or standard Python libraries like instructor to enforce JSON schemas. If Agent A needs to pass an ID to Agent B, structurally guarantee that the output is only a valid JSON object matching your exact schema.

2. Adopt Standardized Protocols

Stop reinventing the wheel for tool use and communication. Implement MCP for all A2C interactions to securely connect your agents to external systems. For A2A interactions, adopt the A2A Protocol to standardize payload structures instead of injecting variables into conversational prompt templates. Leveraging these standards ensures your agents can operate reliably across different platforms and enterprise environments.

3. Centralize State (The “Blackboard” Pattern)

Instead of passing massive context back and forth between agents like a hot potato, use a centralized state mechanism. In graph-based frameworks like LangGraph, all agents read from and write to a single, shared state object or “blackboard.” This ensures no agent is operating on outdated or asymmetric information.

4. Implement Observability and Tracing

When a multi-agent system fails, it fails silently and weirdly. You cannot debug these systems with standard print statements. You need dedicated LLM observability platforms like LangSmith, Langfuse, or MLflow 3.0 to trace the exact input, output, and execution path of every single node. If the Telephone Game happens, you need to see exactly which agent dropped the context.

5. Defensive Programming: Retries and HITL

Expect agents to fail. Wrap inter-agent communication in standard retry logic with programmatic guardrails (like NVIDIA’s NeMo Guardrails). If Agent A sends malformed data, catch the error programmatically and format it into a rigid prompt to force a correction. For critical workflows like writing to a production database (or deleting production databases) - enforce a Human-in-the-Loop (HITL) pause. Let the system wait for human approval before executing destructive actions.

Lessons Learnt

• Treat agents like microservices: A2A communication should mirror microservice architecture. Define rigid, structured API contracts for every handoff and validate payloads before they reach the next node.

• Embrace MCP and emerging standards: Separate your A2C (tool use) from your A2A (agent coordination). Use standardized open-source protocols like MCP and A2A to offload the complexity of system integrations so your agents can focus on logic.

• Cap your feedback loops: Always implement hard limits on iterative loops. If agents go back-and-forth more than three times without success, throw an exception and escalate to a human or a deterministic fallback script.

• Persist the global goal: Inject the original user intent into the system prompt of every agent in the pipeline. Do not assume intent will survive passing through three different LLM nodes.

Multi-agent architectures are the future of complex AI applications, but they require rigorous distributed systems engineering. By enforcing strict protocols and shared state, you can stop the silent failures and build agentic systems that actually work in the real world.

See It In Action

I’ve talked about the common patterns and failure modes of multi-agent orchestration and how to design for them in this video. Have a look.

As always your comments and feedback are welcome. Please share your experience and thoughts.

Thanks,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

If you are reading yet another think-piece on “scaling autonomous agents” or “optimizing your basic RAG pipeline,” you are observing the trailing edge of the industry. We all know the standard playbook by now: deploy an LLM-as-a-judge, set up a vector database, and run basic semantic search. That is no longer a competitive advantage; it is table stakes.

To survive in production at scale today, engineering teams must stop treating generative models like brilliant, autonomous colleagues and start treating them like chaotic, highly expensive engine components.

Here are the three architectural blind spots that standard DevOps playbooks are ignoring - and how to fix them.

1. Stop Building Agents. Build State Machines.

The current industry obsession is giving LLMs autonomy - letting them chain tools, determine their own loops, and “think” their way out of problems. However, in an enterprise production environment, autonomy is just another word for liability.

You do not want an autonomous agent; you want a rigid, locked-down Finite State Machine (FSM).

Your software architecture should entirely dictate the exact path, the boundaries, and the execution graph. The LLM should only be utilized for the transition logic. Its sole job is to ingest messy, unstructured user input and output a deterministic decision: “Do we transition to State A or State B?”

By stripping the model of its agency and restricting it to routing and classification, your latency drops, your reliability scales, and crucially, your system becomes highly debuggable when an edge case inevitably breaks the flow.

2. Eradicate the “Politeness Tax”

If you audit your raw token logs, you will likely find that you are paying thousands of dollars a month - and sacrificing hundreds of milliseconds of latency per request - just to let your model clear its throat.

Every time a background model outputs, “Certainly! I’d be happy to extract that data for you. Here is the requested JSON:”, you are burning compute. At scale, politeness is an engineering flaw.

You cannot fix this with prompt engineering alone. You must enforce strict grammar constraints at the inference level. Do not politely ask the model to return JSON in the system prompt; force the API to accept { as the absolute only valid first token. Strip out all conversational abilities from your background processing models.

You do not need a polite assistant in your backend data pipeline; you need a ruthless text calculator.

3. Neutralize “Zombie Memory” in Semantic Caching

Semantic caching is universally recommended to reduce API costs. A user asks a question, you embed it, check if you have answered a mathematically similar query recently, and return the cached answer.

What nobody discusses is semantic cache rot. If you are caching answers about dynamic data, like your pricing tiers, live inventory, or active user permissions, the underlying reality will eventually change, but your vector cache remains static. When this happens, the cache intercepts the query and serves up a perfectly formatted, highly confident answer that is now entirely false. Your system isn’t hallucinating; it is remembering a dead reality.

To solve this, a simple Time-to-Live (TTL) expiration is insufficient. You must bind your vector cache invalidation directly to your database webhooks. If a product goes out of stock in your primary database, your system must aggressively and automatically flush the neighborhood of vectors in your cache that map to that specific product’s metadata.

The Takeaway

Moving AI from a compelling local demo to a hardened production environment requires a fundamental shift in engineering mindset. It is not about finding the perfect prompt or chasing the newest foundational model.

The best AI engineers do not try to find perfect model outputs . They build perfect architectural nets to catch the model when it inevitably behaves unpredictably.

What is your production AI blind spot?

We are all writing the playbook for production AI in real-time, and the best lessons come from the trenches, not the demo environments. Hit reply and tell me about the weirdest silent failure mode you have caught in production recently, the one that no standard DevOps tool saw coming.

If this issue helped you rethink your architecture, do me a favor: forward it to the engineer on your team who is currently trying to solve a systems problem with another paragraph of prompt engineering.

Talk soon,
Sandi.

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

A team deploys an AI system into production. It passes every evaluation they ran. Accuracy looked good. The stakeholder demo went well. The pilot was declared a success. Three months later, the system is quietly shelved because the outputs no longer make sense - or worse, they never did, and nobody caught it until real users started complaining.

When I dig into what went wrong, I often find the shape of the evaluation resulting in low quality agentic decisions.

The teams running linear eval pipelines - input goes in, score comes out - are measuring a snapshot of a moment. They are not measuring how their system behaves as context shifts, as data drifts, as agents hand off to other agents, as the real world does what the real world always does. They are measuring a straight line. Their system is a graph.

That mismatch is why so many AI evaluations feel thorough and turn out to be worthless.

Pipelines vs Graphs

The word ‘pipeline’ is everywhere in AI engineering. Data pipelines, inference pipelines, eval pipelines. We’ve adopted it as the default mental model for how AI systems work.

And for a lot of data engineering, it’s correct. Data flows in one direction. You extract, you transform, you load. A pipeline is a clean metaphor because data really does flow like water through a pipe.

But AI systems in production - especially multi-agent systems, RAG architectures, and anything that has to maintain context across multiple turns or tool calls - do not behave like pipelines. They behave like graphs. There are loops. There are conditional branches. There are nodes that depend on the state of other nodes that were resolved two steps earlier. Context that was established at step one can poison or distort the output at step seven.

When you evaluate a graph as if it were a pipeline, you get a false sense of confidence. You test the happy path. You test the input-output pair. You miss the edges. You miss the feedback loops. You miss the context that has been accumulating and silently corrupting your system’s reasoning.

I’ve started calling this context drift - the phenomenon where a system’s outputs because the context it’s operating in has shifted in ways your evaluations weren’t designed to detect. A pipeline eval can’t catch context drift. Only a graph-shaped evaluation can.

What is an Evaluation Graph?

The Evaluation Graph is not a tool or a framework you install. It’s a mental model - a different way of thinking about what you’re actually evaluating and when.

In a pipeline eval, you define a set of test cases, run your system against them, and score the outputs. Done. Repeatable. Clean.

In an evaluation graph, you map out the nodes of your system - the points where decisions are made, where context is retrieved, where agents hand off to each other, where state is read or written - and you evaluate at each node, not just at the final output.

Here is what that changes in practice.

First, you gain localised failure detection. When a pipeline eval fails, you know something went wrong. You don’t know where. When a graph eval fails, you know exactly which node broke down - was it the retrieval? The reranker? The summarisation step? The router that decided which agent to call? You can fix what’s actually broken instead of rerunning the whole system hoping for different results.

Second, you can evaluate context propagation. I have seen this skipped many times. It’s not enough to evaluate whether each node produces a good output given its input. You need to evaluate whether the context being passed between nodes is coherent, relevant, and not accumulating noise. I’ve seen systems where individual components all scored above 90% in isolation, but the system as a whole produced nonsense because each node was passing slightly degraded context to the next one. No pipeline eval would catch that.

Third, you can evaluate decision boundaries. Multi-agent systems have routing logic - conditions that decide which agent runs next, or whether to escalate, or whether to call a tool. These decision boundaries are often the most fragile part of a production AI system, and they’re almost never tested explicitly. In an evaluation graph, they are nodes. They get evaluated just like everything else.

How to Build One

Starting with the evaluation graph doesn’t require you to throw away your existing evals. It requires you to extend them in a specific direction.

The first step is decomposition. Draw out your system - literally, on a whiteboard or in a diagram - and identify every point where a meaningful decision is made or meaningful state changes. Each of those points is a node. Each connection between nodes is an edge. What you’re drawing is the evaluation graph. Most teams are surprised by how many nodes they find that they’ve never evaluated.

The second step is context mapping. For each edge in the graph, define what context is being passed from one node to the next. What does the downstream node need to function correctly? What could the upstream node pass that would corrupt the downstream output? These become your edge-level test cases - not just input-output pairs, but context-propagation scenarios.

The third step is failure mode enumeration. For each node, ask: what does this node look like when it’s failing quietly? Not failing loudly - that’s easy to catch. Quiet failures are the dangerous ones. A retrieval node that returns plausible but wrong documents. A router that sends requests to the wrong agent 15% of the time. A summarisation step that subtly omits the most important information. These failure modes need to be in your evaluation suite explicitly. If they’re not, you won’t find them until a user does.

The fourth step, and this is where graph-shaped evaluation really separates from pipeline evaluation is composing your node-level evals into end-to-end scenarios that test the interaction effects. Not just ‘does node A work’ and ‘does node B work’, but ‘when node A produces this class of output, does node B degrade in a predictable way’. The interactions between nodes are often where production AI systems fail.

This Needs a Shift in Mindset

The teams that build AI systems that hold up in production are building the most rigorous evaluation infrastructure. And rigorous evaluation infrastructure starts with a simple question: is my evaluation shaped like my system?

If your system is a graph and your evaluations are pipelines, you have a gap. That gap is where production failures live.

The evaluation graph is not a perfect solution - no evaluation framework is. Context still drifts in ways you won’t anticipate. Failure modes you didn’t enumerate will still appear. But it gets you structurally closer to what’s actually happening in your system, and that’s the difference between catching problems in staging and catching them after a customer has seen them.

This is one of the core concepts I’m currently working on. If it resonates with what you’re seeing in your own work, I’d genuinely like to hear about it. Hit reply and tell me what you see. The patterns you share inform what I write next.

Talk soon,
Sandi

👉 I wrote more about the Eval Graphs in my article on Atlan’s community substack.

Context & Chaos

Context Graphs as AI Evaluation Infrastructure

About the Author: Sandipan Bhaumik have spent almost 2 decades building Data & AI foundations. Now, through AgentBuild Weekly, he shares how builders and founders can move beyond AI hype to create Agentic systems that think, adapt, and truly work…

Read more

4 months ago · 11 likes · Sandipan Bhaumik

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.

More valuable content coming your way.

Share agentbuild.ai

It’s the person who gets pulled into the room when the demo worked brilliantly and the deployment didn’t. The person who has to figure out why a system that impressed everyone in the boardroom is now sitting in a security review queue with no clear owner, no evaluation criteria, and a go-live deadline nobody wants to move.

That person is usually a Solutions Architect.

And in this new world of AI transfomration architects are lacking the frameworks to match the responsibility they’ve been handed.

What the role has become

I’ve spent eighteen years in enterprise data and AI. The last several watching what happens when organizations decide to take AI seriously.

Here’s what I keep seeing: Solutions Architects are becoming the load-bearing wall of AI transformation programmes. By default.

They’re the ones who understand both the technology and the business context. They’re trusted enough to sit in executive sessions and technical ones. They have enough credibility to push back on vendor claims and enough pragmatism to know what actually ships.

So they get handed things. Big things.

Define the production readiness criteria. Assess whether the data infrastructure can support this use case. Figure out who owns the outcome when the model is wrong. Translate what the VP wants into something the engineering team can build. Get security and compliance aligned before the launch date nobody will move.

That’s not an implementation role. That’s an organizational diagnostic role. And most architects are not prepared for it.

The gap nobody names

The architects I see are not struggling because they can’t build. They can build. They’re struggling because the job has shifted from build to diagnose, and they don’t yet have the instruments for it.

When a doctor walks into a room, they’re not improvising. They have a diagnostic protocol. Repeatable questions. Known patterns. A framework that tells them what to look for and in what order, so they can tell the difference between something that needs immediate intervention and something that needs monitoring.

Right now, most architects walking into an AI programme are improvising. Drawing on instinct built from past projects. Pattern-matching against things they’ve seen before, hoping the pattern holds.

Sometimes it does. Often it doesn’t.

And when it doesn’t, the cost isn’t just the failed project. It’s the six months of organizational trust that went with it. The next AI initiative that’s three times harder to fund because this one didn’t ship. The architect who now has a complicated story to tell about why the thing they led didn’t work.

What real preparation looks like

I’ve been thinking for a long time about what it would mean to give architects the diagnostic tools they actually need. Something closer to a practitioner’s handbook for the organizational side of AI deployment. Not a vendor comparison or a tutorial on which framework to use.

The kind of resource that helps you walk into an early-stage AI programme and ask the right questions before anyone starts building. That gives you a structured way to identify where the real risk is - not the model risk, but the Data Debt sitting in pipelines that haven’t been touched in three years. The Decision Debt in an organization where nobody has agreed on who owns an AI error. The Evaluation Debt in a team that’s been running vibe checks and calling it validation.

The kind of resource that helps you have the conversation with the VP that reframes the whole initiative - not as a technology project, but as an organizational readiness problem that happens to have a technology solution.

That’s the conversation that changes outcomes. And most architects don’t have a framework for it yet.

Why I’m spending time on this

I’ve watched enough of these programmes - close enough to see the failure modes in detail - that the patterns are starting to feel predictable. Which means these are preventable.

I can walk into a kickoff meeting now and have a reasonable sense of what’s going to go wrong six months later. Not because I’m smarter than anyone in the room. Because I’ve seen it before. Enough times that it’s stopped feeling like bad luck and started feeling like a diagnostic problem with a known set of causes.

What I want to do - what I’m actively working on - is make that pattern recognition transferable. To give architects the frameworks that took me years of seeing things go wrong to develop, so they don’t have to learn the same lessons at the same cost.

That’s the work I’m orienting around. That’s the shape I want to give this community.

If you’re an architect who’s been handed one of these programmes - or knows you’re about to be - I’d genuinely like to hear what’s hard about it right now?

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

Let me explain.

Flight data recorders (FDR) are popularly know as the black box of the aircraft. Technically the black box contains the FDR and the Cockpit Voice Recorder (CVR). The FDR turns raw sensor traces into timelines that explain crashes, reveal root causes, and drive global aviation safety improvements. Before flight data recorders were mandated, when something went wrong, investigators worked from witness accounts, wreckage patterns, and whatever instruments happened to be installed at the time. The analysis was mostly incomplete, often contradictory, and it rarely led to systemic changes.

The aviation industry knew flying was becoming more consequential with more routes, more passengers, more complex airspace, but the infrastructure to understand why things failed hadn’t kept pace with the deployment of the systems themselves. It was invented in response to the recognition that consequential systems operating at scale need a structured record of their reasoning - not just their outcomes.

AI agents are at the same inflection point today. And the industry, especially in the regulated space is recognizing that.

What’s the gap right now?

An AI agent deployed in a production system today typically produces two things: an input log and an output. What it doesn’t produce is the reasoning chain between them in any structured, queryable, auditable way .

This gap has a name. It’s called Decision Debt, one of the three categories of debt that block AI from working in production. Decision Debt accumulates when you build and deploy AI systems before defining how decisions get made, recorded, and reviewed. It’s not a future problem. It’s accumulating now, in every agent deployment that ships without trace infrastructure.

A decision trace is the record of how an agent got from context to conclusion: what it knew, what it considered, what it weighted, what it discarded, and at what confidence level it committed to an action.

IMPORTANT: It’s not a log file. Logs capture events. Traces capture reasoning.

The distinction matters because when something goes wrong - and in any system operating at scale, you need to answer a different set of questions than a log can address.

Answering “what happened” is not enough, but “why did the agent conclude that, given what it had access to?” matters more.

This is a historical pattern

Aviation gets to the black box through painful iteration. Financial services gets to trade surveillance infrastructure the same way. Healthcare builds clinical decision support audit trails only after near-misses force the question.

The pattern across every regulated industry is identical: consequential system deploys, operates without adequate observability, incident occurs, retroactive audit reveals the trace infrastructure was never built, expensive fixes follow.

What’s different with AI agents is that we can see this pattern coming before the incidents accumulate. The decision trace problem is visible now, in advance, to anyone who has watched the previous cycles play out in adjacent domains.

Nuclear power operations built decision logging infrastructure into control room design before widespread deployment. And that’s not because regulators demanded it initially, but because the engineers understood that a system making consequential decisions in real time needed to be interrogable after the fact.

The Chernobyl investigation was partially possible because operator actions were timestamped and sequenced. The lessons extracted shaped reactor design globally.

The equivalent for AI agents isn’t complicated in principle.

It is, however, work that almost nobody has started.

What Decision Trace Infrastructure actually need?

The architecture for a decision trace system has five functional layers, and each one has a specific job. Here’s how they fit together.

1. Input Capture Service is where the trace begins - at the moment a request enters the system. Query, user identity, session context, and request metadata are captured here, backed by a metadata store (PostgreSQL can be a straightforward choice). This is the “who asked what, when, and from where” layer. Without it, you have no anchor for the rest of the trace.

2. State Retrieval and Context Snapshot captures the world as the agent saw it at decision time: which data versions were active, which policy definitions were in force, which catalog references were resolved. This layer pulls from a prerequisites datastore - Redis for low-latency state, S3 for snapshot durability. It’s also the layer that makes post-incident analysis possible. When you need to understand why the agent concluded what it did three months ago, you need to know what it knew at that moment - not what the system knows now.

   This layer is, in practical terms, where a context graph lives - even if most implementations don't call it that yet. A context graph is simply the structured representation of what the agent knew and how those things related to each other at decision time: data assets, policies, catalog nodes, versions, and their connections. The reason "context graph" is gaining traction as a term without a settled definition is precisely because this layer has been missing from most agent architectures. Once you build the snapshot layer properly, you have one.

3. Reasoning Chain and Decision Engine is the core trace layer. Chain-of-thought steps, intermediate logic, intermediate outputs - all captured as structured records. It is the path the agent took to reach the final answer. Every branch, every intermediate conclusion, every tool invocation that shaped the reasoning is an addressable record here.

4. Policy Binding Service records the guardrails, rules, and decision logic that were active during the reasoning process. This is what separates a decision trace from a debugging log. You’re not just capturing what the agent did, you’re capturing the constraints it was operating under. When a compliance team asks “was the agent following the policy that was in force on this date,” this layer answers that question directly.

5. Outcome and Action Capturing records the final response, the action taken, and critically - any redress or complaint data attached to that outcome. This closes the loop between the agent’s decision and its real-world consequence. It’s also the layer that feeds dispute resolution workflows when customers or regulators challenge an outcome.

All five layers feed into an Immutable Audit Record - timestamped, hashed, and written to an immutable trace store (S3, Delta Lake or a ledger database). The immutability is a must-have. It is the architectural guarantee that the record cannot be altered after the fact, which is what makes it defensible in a regulatory or legal context. The diagram you see specifies retention period, which aligns with financial services conduct requirements and is a reasonable baseline for any regulated environment.

From the trace store, a Trace Query Service and Data Lake make the records queryable at scale. This is the operational distinction you need to understand. A queryable trace lets you ask: “Show me every decision where the policy binding service applied rule X and the outcome was Y.” That’s the difference between evidence and insight.

The four downstream outputs from this architecture tell you exactly what it’s designed to serve: Redress and Dispute Resolution (when a decision is challenged), Audit Trail Reporting (when a regulator asks), Debugging and Root Cause Analysis (when something fails), and Improvement and ML Training (when you want to make the system better using real decision data).

No single layer here is novel in isolation. Input capture, immutable storage, policy versioning exist in adjacent systems already. What doesn’t exist yet, in any standardised form for AI agents, is this stack assembled as a coherent, purpose-built trace infrastructure. That’s the gap this architecture closes.

Why enterprise architects need to move on this now

This infrastructure is necessary to build the cleanest parth through regulatory scrutiny, incident response, and enterprise customer due diligence. The same principle made structured engineering logging standard practice in distributed systems. You cannot debug what you cannot observe. You cannot improve what you cannot measure. And you cannot defend in a board meeting, a regulatory inquiry, or a customer audit what you never recorded.

Decision traces are the observability layer for AI reasoning. It is the infrastructure equivalent of distributed tracing in microservices, now applied to systems that don’t just execute code, but form conclusions and take actions.

The good news is that this is buildable now, with current tooling, before the incidents force it. The question is whether engineering organisations treat it as foundational infrastructure from the first production deployment, or discover its absence after the fact.

Call To Action

If you are building AI agents for anything consequential, the time to design trace infrastructure is before the first production deployment. Start by mapping which decisions your agent makes that you could not currently explain, audit, or defend: that list is your build priority.

If this framing is useful, share it with the architect or engineering lead on your AI team - this is the conversation that needs to happen before the system goes live, not after.

Leave a feedback or comment. Share your opinion about this topic.

Thanks,
Sandi.

👉 You might also find my article published on Atlan’s community Substack useful:

Context & Chaos

Context Graphs as AI Evaluation Infrastructure

About the Author: Sandipan Bhaumik have spent almost 2 decades building Data & AI foundations. Now, through AgentBuild Weekly, he shares how builders and founders can move beyond AI hype to create Agentic systems that think, adapt, and truly work…

Read more

4 months ago · 11 likes · Sandipan Bhaumik

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

I owe you an explanation for going quiet last Saturday.

We took an Easter break as a family - properly offline, no laptop, in the English countryside. It was superb, the rolling green fields actually delivered on the promise. - and it was sunny ☀️

Right. I’m back. And there’s quite a lot to catch you up on.

I was at an AI engineering conference this week

And I gave a talk, it will be out soon. This is the first time it happened in Europe and I got to meet so many smart, talented founders and engineers. It was awesome experience. In-person events are irreplaceable.

I attended some fabulouse sessions on cutting-edge stuff on AI. And of course OpenClaw dominated the discussion.

Check out the conference here: https://www.ai.engineer/europe

I also made it to the online track of the conference

And the topic was something I’ve been working towards for a while - Multi-Agent Orchestration Patterns for Production.

The core argument: the field is moving fast, but most teams hit the same wall. They build multi-agent systems like they built single-agent systems. Same assumptions, same trust in the “it works in the demo” signal. And then production arrives, and nothing holds.

The talk walked through choreography vs orchestration, immutable state patterns, circuit breakers, and why distributed systems thinking is no longer optional if you’re building agents at any meaningful scale.

Check it out.

A piece I wrote just went live

This one has been in the works for a while, and I’m genuinely proud of it.

I wrote a guest article for Context & Chaos introducing two concepts I’ve been developing from my work with regulated enterprises: context drift and the evaluation graph.

When an AI system gives you an answer, that answer wasn’t produced in a vacuum. It was produced against a specific version of your world - a specific definition of what “active customer” meant that week, a specific policy that was in force that month, a specific dataset that may or may not still exist.

Think of it like this: imagine a doctor’s notes. It’s not enough to record what prescription they wrote. You also need to know what guidelines were current that day, what the patient’s history showed at that point, what the lab results said. Without that context, the notes are incomplete. Enterprise AI has the same problem. We’re recording the prescription. We’re not recording everything else that informed it.

This is original IP, and I think it’s going to become a recurring theme in how regulated industries think about AI governance.

Here is the article:

Context & Chaos

Context Graphs as AI Evaluation Infrastructure

About the Author: Sandipan Bhaumik have spent almost 2 decades building Data & AI foundations. Now, through AgentBuild Weekly, he shares how builders and founders can move beyond AI hype to create Agentic systems that think, adapt, and truly work…

Read more

4 months ago · 9 likes · Sandipan Bhaumik

New YouTube video: The AI Latency Stack

While you’re in content-consumption mode this weekend, I also want to point you to a video I put out recently on AI application latency - it keeps coming up in conversations and I wanted to have something concrete to point people to.

The short version: after your AI system ships to production, the model is almost never the problem. It’s a set of architectural decisions - streaming, database writes on the critical path, cold starts, context window bloat, prompt caching, sequential calls that should be parallel - that compound into something that makes users give up and go back to the manual process. The video walks through seven of these layers and how to address them, without swapping models or changing vendors.

Worth a watch if you’re anywhere near a production AI deployment right now.

That’s it for this week.

A lot happened in a short space of time, and I wanted to share it with you directly. As always - reply if anything resonates, or if you’re wrestling with something I touched on.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.

I’m lucky. My job puts me in front of a lot of engineering teams across a lot of organisations. Some are moving fast. Some are moving slow. And I get to see both. Not from a distance, up close, in the actual conversations where decisions get made.

What I’m watching is a quiet split happening inside engineering teams. And I think it matters for anyone thinking about where this profession is heading.

Two engineers. Same company. Same tools available. Same access to AI. Completely different outcomes.

One of them, when they hit a hard problem, opens a chat window and starts working through it out loud. They dump in the messy context. The half-baked question. The data that doesn’t quite make sense yet. They’re not looking for autocomplete. They’re looking for a way through.

The other one says “AI isn’t reliable enough for this.” And goes back to doing it the slow way.

I’ve watched this play out across banks, fintechs, and large regulated enterprises. And the gap between these two engineers is only getting wider.

What high agency actually looks like

I was working with a team recently trying to make sense of a large pile of unstructured documents. Audit logs, policy docs, historical reports. This kind of work that normally takes weeks of someone’s time.

One engineer on the team didn’t wait to be told how. She had no prior experience with the specific tooling. But she sat down, broke the problem into pieces, and used AI to work through each one. By end of day she had something working. Not perfect. But working.

She didn’t have a playbook. She made one.

That’s what high agency looks like in practice. Not waiting for a process document. Not waiting for someone to say it’s approved. When they hit a wall, the first instinct is to figure out what question to ask - not explain why the wall is there.

What the resistance sounds like

I want to be careful here. The engineers pushing back on AI are not lazy. Many of them are the most experienced people in the room.

But the resistance has a pattern.

“It hallucinates too much for our use case.”

“Security hasn’t signed it off yet.”

“The outputs aren’t consistent enough to trust.”

“This is hype, let it settle.”

Some of these are valid. I work in regulated environments. I understand the constraints.

But what I notice is this. The engineers saying these things have usually not given AI their hardest problem. They’ve given it easy tasks, watched it stumble, and concluded it isn’t ready. They’re evaluating a tool they haven’t really pushed.

The high agency engineers hit the same limitations. They just treat them as constraints to work around, not reasons to stop.

There’s something underneath the resistance

I think it goes deeper than technology skepticism.

A lot of experienced engineers have built their identity around already knowing the answer. They’re the person people come to. The one who’s seen this before.

AI is uncomfortable for that identity. Because the value is shifting. It’s moving away from already knowing - toward knowing how to ask. That’s a different skill. And it asks you to be a beginner again, at least partially.

The engineers I see thriving have a looser grip on what they already know. They’re curious before they’re skeptical.

They pick the tool up before they critique it.

One practical thing

Ask yourself honestly: when did you last give AI your genuinely hardest problem?

Not “summarise this document.” or “tidy up this function.”

The real hard thing. The one you’ve been circling because you don’t quite know where to start.

I’ve seen engineers use AI to compress weeks of analysis into a day. I’ve seen it catch patterns in production failures that a team had been chasing for months. I’ve seen it unlock a business conversation that had been stuck for a quarter - just by helping someone structure their thinking clearly enough to explain it.

None of that happened because the technology was perfect. It happened because someone decided to figure it out.

The job of an engineer is changing. I’m watching it happen. The ones adapting aren’t the most experienced or the most technical. They’re the ones most willing to stay curious.

That’s the only practical advice I have.

One question before you go - what's the most interesting thing you've seen an engineer do with AI that nobody is talking about yet?

Hit reply and tell me.

Talk soon,
Sandi

P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.

Ask your friends to join.
More valuable content coming your way.

Share agentbuild.ai

Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.