Why Enterprise AI Needs an Ontology
Ontology is the most talked about topic in Enterprise AI right now. Let's explore why it is important, what it is, and how to build one.
Six months ago, ontology was a word you heard from librarians and pharma researchers. Now it’s in every third architecture review I sit in. Teams that spent 2025 building RAG pipelines are arriving at the same problem now: “retrieval works, generation is fluent, and the answers are still wrong. WTF!”
Palantir built a business on this word. Every catalogue vendor has quietly rebadged its glossary as a semantic layer. Features tagged with the word “Ontology“ are being released.
Underneath the marketing there’s a real engineering shift, and it’s worth separating the two.
Why Ontology matters for enterprise AI?
An LLM reads your data. It doesn’t understand your business, and nothing in your stack is currently telling it the difference.
Ask three systems what a customer is. Sales counts anyone who signed. Support counts anyone with a login, which sweeps in two people who never bought. Finance only counts accounts that have paid an invoice. The gap between those definitions is real money, and the only place it’s written down is in the head of the employees.
So when someone asks the AI how many customers you have, retrieval pulls all three definitions into the same context and the model hands back one number. It won’t mention that it merged three incompatible populations to get there, and you’ve got nothing to check it against.
A glossary won’t save you, because a glossary is written for people to read, not for a pipeline to enforce. A tidier schema won’t either. A foreign key can say two tables are related; it can’t say what the relationship means. That missing layer is what an ontology holds, in a form the machine reads at query time rather than a form a human reads in a wiki.
It becomes even more serious in the write path. A chatbot that miscounts customers gives a wrong answer. An agent that miscounts and can act on it does something about it, mails the wrong segment or moves the wrong forecast. That’s roughly when the word “ontology“ started coming up in architecture reviews.
What is an Ontology?
The textbook definition is Gruber’s:
an ontology is a formal, explicit specification of a shared conceptualisation.
Let’s look at the four words:
A conceptualisation is the abstract model of what exists in a domain. It’s already in everyone’s head, whether written down or not.
Explicit means it’s written down.
Formal means a machine can act on it, not just a person read it.
Shared means it’s agreed and owned across a community, not one architect’s private diagram.
Mechanically, you write statements as triples: subject, predicate, object.
Here’s a fragment in Turtle:
:Customer a owl:Class .
:PayingCustomer rdfs:subClassOf :Customer .
:placed a owl:ObjectProperty ;
rdfs:domain :Customer ;
rdfs:range :Order .The last block says something a foreign key cannot: placed only ever runs from a customer to an order. Assert that some account placed an order, and a reasoner infers the account is a customer. Nobody typed that fact. It follows.
Then you query the meaning instead of the tables:
sparql
SELECT ?customer WHERE {
?customer a :PayingCustomer ;
:placed ?order .
}This returns paying customers across every source system feeding the graph, because the definition of “paying customer” lives in one place now instead of three. That’s the line between a schema and an ontology.
A schema constrains data for one application. An ontology models a domain for many, and it entails things you never wrote down.
How to actually build one
This is very new for most companies. There is no fromal playbook - but I can see a number of guidances coming out. There are many issues I see with ontology imlementation. Usually a modelling team would take nine months to produce two thousand classes, and nobody uses it. That’s costly.
Given what I have learnt so far, I can roughly put some thoughts on the implementation, already working ins ome enteprrises:
Start with competency questions. Write the twenty questions the ontology must answer, in business language, before you model anything. “Which counterparties have aggregate credit exposure above £50m across all subsidiaries?” If your model answers it, that part is done. If you can’t write the questions, you don’t have a use case, you have a vocabulary project.
Reuse before you build. FIBO already models counterparty, legal entity, and financial instrument, with a decade of EDM Council work behind it. SNOMED CT has done clinical terms. Your novel IP is the last 10% specific to your firm, not the definition of a legal entity. Reuses existing artefacts.
Treat it as software. Ontologies rot faster than code because nobody owns them. Put the files in Git. Write SHACL shapes as your tests and run them in CI, so a change that breaks a downstream contract fails the build rather than a quarterly report. Give every module a named business owner who signs off changes, and version it properly.
Ship thin vertical slices tied to one workflow. One agent, one domain, one set of competency questions, in production. The organisational fight you’re picking is agreement on what words mean, and that fight is much easier to win on a small surface with a working demo behind it.
The takeaway
The ontology conversation isn’t a semantic web revival. It’s what happens when systems start acting on their own and the industry discovers it never wrote down what anything means.
I went deep on this with Jessica Talisman, who has spent her career on taxonomy and ontology work at the sharp end of large organisations. We got into where taxonomies stop and ontologies start, what breaks at enterprise scale, and why most of the failures are organisational rather than technical. Spend sometime this weeked, watch it, and let me know how you found it.
Hope you enjoy it.
Happy Weekend.
Talk soon,
Sandi
P.S. If you’re new here - welcome 🎉. AgentBuild is a community of practitioners working through the real challenges of getting AI into production inside large organisations. Every week I share practical, grounded thinking from the people doing this work at the sharp end. The goal is never theory - it’s always: what can you use Monday morning.
Ask your friends to join.
More valuable content coming your way.
Thanks for reading agentbuild.ai! Subscribe for free to receive new posts and support my work.