Graphs Are the Knowledge Layer for Agentic AI

TL;DR

• The knowledge layer is a pattern, not a buzzword. Across thousands of Neo4j customers the same architecture keeps emerging: apps, agents, and tools talk to one knowledge layer that unifies operational and analytic sources, instead of wiring into each source directly.
• It is not a lift and shift. Knowledge, as distinct from data, means pulling signal out of noise and connecting it — and it can be built incrementally, starting small.
• Three graph shapes show up in production: a domain graph (semantics and ontology), a rules graph (procedural memory), and a semantic layer (business vocabulary mapped down to where the data physically lives) — exemplified by Walmart, Uber, and a major gaming company respectively.
• Why not the alternatives: foundation models are trained on the world's data, not yours; a bespoke small model is expensive, stale the moment it finishes training, and blows past your access controls; rows and columns answer fixed simple questions; vectors are opaque. Graphs handle both cross-silo connection and genuinely complex questions.
• The knowledge layer computes. Graph queries, graph algorithms, similarity search, GNNs, and visualization all run in this layer, yielding GraphRAG accuracy, multi-hop reasoning, decision traces, and memory.

Key Points

A pattern discovered in the field (~03:14)

Rathle opens by collapsing five fashionable terms — knowledge, context, semantics, memory, ontology — into one. They are separate topics on paper, but in enterprise deployments they converge on a single architectural tier, and that convergence is what he has watched happen across thousands of customers: apps, agents, and tools on top; a knowledge layer in the middle; operational sources on one side and analytic sources on the other.

Two qualifications matter. First, this is not a lift and shift — knowledge is precisely not everything about everything, it is signal pulled out of noise and then connected, and the connections are the point. Second, it is incremental: start small, build up.

Functionally, the layer stores structure, context, and meaning; gives agents a uniform memory and retrieval interface; and bridges human and machine understanding. That last point is where neurosymbolic AI, decades old, comes back: you cannot introspect a model or a vector database — they are unintelligible to a person — whereas a graph is legible to a human and executable by a machine.

Three graphs, three customers (~03:16–03:19)

• Walmart's People AI knowledge graph serves roughly 1.6 million employees navigating their career journeys. The graph holds people, the reporting structure and its history, how skills accumulated over time, project history, open roles, and role requirements. Put all of that in one graph and career recommendation reduces to connecting the dots — "solving for HR by degrees of Kevin Bacon." Crucially there is a model between the human and the graph, which buys optionality: the same question can be answered by feeding the model more context, or answered deterministically by querying the graph.
• Uber's rules graph: Uber operates in 15,000 cities with different offerings, different names for those offerings, different laws and regulations, plus Uber's own reasons for varying them. Those rules are both stored and executed in the graph.
• A major gaming company's semantic layer: conversational AI → API layer → multi-agent system → graph. Here the graph maps business terms, concepts, and their relationships down to physical and data infrastructure — effectively giving agents a map of where to go in the enterprise to find data.

What the three have in common — and the insight that drove each team to graphs — is that their data already arrived as a network (computer networks, social networks, communications, biology, ecology, transportation) or as a hierarchy and journey (Walmart's HR is several overlapping hierarchies; so are customer and patient journeys).

Why a new data model (~03:20–03:22)

Tables have been the default for decades, but two years ago the property graph data model became an ISO standard — the first time in nearly 40 years that ISO blessed a new data model in the database space.

Why isn't the existing toolkit enough? A foundation model is trained on the world's data, not yours. Training your own small model is expensive, stale the instant it finishes, and — more subtly — everything you trained into it becomes fair game for any agent answering any question for any purpose, which collides head-on with enterprise access controls; it still hallucinates, still lacks discernment, and is still a black box. Static rows and columns are great for fixed, simple questions and terrible when questions get intricate and unpredictable, which is exactly where graphs have shined for the decade-plus he has worked on them. Unstructured data is hard to mine; vectors are completely opaque and arguably a one- or two-trick pony; and organizational data sits in silos, so agents get one system's intelligence rather than the company's.

He cites Gartner's recommendation to data and analytics leaders as external confirmation: redesign the data and analytics architecture so the context layer becomes the central brain for AI agents delivering trusted intelligence.

What agents require of knowledge (~03:22–03:24)

Four requirements: (1) real-time, low-latency retrieval and write-back, so agents get a virtuous cycle of self-improvement — a lake or lakehouse alone cannot serve an interactive agent; (2) structured, semi-structured, and unstructured storage; (3) cross-silo integration that preserves the connections — lakes centralize everything without connecting it; (4) a RAG corpus (he'd argue a graph RAG corpus) plus data access controls for firewall, security, and privacy reasons.

Zooming into the layer itself: domain data connected as a graph (your own plus third-party reference data, with the option of running graph queries virtually against data lakes); ontology and semantic layer, carrying business meaning and its mapping to where data lives — a business-to-technology map; and memory. Conversational memory is just accumulated text and prompts, but much as the human brain consolidates memory during sleep, long-term memory answers "what matters and how does it connect" — and long-term memory, procedural memory, context, and decision traces are all best stored as graphs.

What the layer buys you — and the fact that it computes (~03:24–03:28)

The payoff list: context for better answers, connected silos, knowledge transparency, an option for deterministic processing, respected access controls, and — surprising even to him — markedly better handling of complex questions. Customer after customer reports that models translate a complex natural-language question into a graph query (GQL or Cypher) far better than into SQL, because a question that takes 200 lines of SQL may take five lines of graph query; the representation sits closer to human language.

For hard numbers he points at an independent study by the UK's NICD comparing GraphRAG to vector-only RAG: more than twice as many questions answered, roughly 80% better truthfulness, fewer hallucinations, and fewer tokens consumed.

The takeaway he most wants to leave: the knowledge layer is not a passive store — it performs computation. Graph querying (pattern matching and filtering), graph algorithms (clustering, link prediction), similarity search, GNNs (a topological kind of vector), and graph visualization all live here. The resulting advantages are GraphRAG accuracy, multi-hop reasoning for harder agent tasks, an enterprise semantic map, decision traces (a whole talk in itself, he notes), and memory.

Where to start? Breadth-first and top-down, treating the enterprise as a connected organism and mapping across the silos; or bottom-up, e.g. a product graph that immediately supports recommendations, pricing, and inventory, then connects to supplier data for supply-chain optimization, and likewise an employee graph for career journeys.

Quotes

"Knowledge, distinct from data, doesn't mean everything about everything. It means let's pull up the signal from the noise and let's connect it up." (~03:15)

The knowledge layer is not another name for the data lake; the editorial judgment and the connections are the value.

"It's almost like solving for HR by degrees of Kevin Bacon." (~03:17)

Career recommendation at Walmart is, structurally, path-finding on a graph.

"You're centralizing everything but you're not connecting it." (~03:23)

His one-line critique of lakes and lakehouses.