AI for Data and Data for AI

TL;DR

• The era of general-purpose data is over. Model providers have already swallowed most of it, but the data belonging to corporations, financial institutions, government agencies, medical centers and individuals will stay outside the models, and it is dynamic. One of GenAI's key use cases will be orchestrating and supporting access to and use of external data.
• "AI for data" is only the first half; "data for AI" is the point. Agents need an enormous range of data — design documents, tickets, correspondence about earlier generations of the software, trajectories, operational records, tool-use traces. The quality, structure and governance of your data determine what AI can actually do.
• "Retrieval works" is a myth. "Try to search your email and tell me whether it does work." Retrieval only fires when the query is lexically close enough to the target documents, which typically doesn't happen — hence the need for a semantic data layer across knowledge sources.
• Absolute numbers are more honest than leaderboards. Oracle's teams lead on Spider 2 and Archer, but Archer sits around 55% and Spider 2 in the low 70s — far from being able to access and use data properly.
• The tennis example lays it bare. A frontier model planned its own approach, pulled from structured, unstructured and web sources, reconciled conflicting information and produced a beautiful table — with two rows mis-sorted, Carlos Alcaraz missing entirely, and two players' prize money identical to the dollar. Prompted to add Alcaraz, it did — and Andrey Rublev vanished from the list while the numbers quietly shifted.
• The real conclusion isn't "models are bad" — it's that failure is invisible. Getting tennis wrong costs nothing; ask the same question about your company's financials and visibility of failure becomes the blocker to using these systems reliably and consistently.

Key Points

Changing gear: from models to data (~01:23–01:25)

He deliberately shifts topic to data — how we use it, how we access it. What excites him is GenAI's promise for data: moving from the messy world — many data types and representations, where people must learn the domain, learn how data is represented, transform between formats, and figure out how to fuse diverse data — to something far cleaner: working with data at a conceptual level, on our own terms, with our own metrics.

What adds to the excitement is a structural fact: a large amount of the world's data will remain outside language models. Models have swallowed a lot of data — which is why we have powerful agents today — but there is no more general-purpose data; most of it has already been absorbed by model providers. Yet there is a lot of data out there: corporations, financial institutions, government agencies, individuals, medical centers. This data will stay outside the models, and it will be dynamic.

So the question becomes: how do we deal with this external data? He believes one of GenAI's key use cases will be orchestrating and supporting access to and use of external data. True for human consumption, but just as important for agents. Think about coding agents: to build them, they need access to a great deal of data — from design documents to tickets to correspondence about what happened in previous generations of the software — and they need to do something with those knowledge sources.

Why decisions that depend on retrieval are so hard (~01:25–01:28)

Reliably supporting decisions that depend on retrieving and using data is extremely difficult — more difficult than we tend to think. Several reasons:

• Information retrieval itself. "There is a myth that information retrieval works — but try to search your email and tell me whether it does."
• Understanding information needs, understanding the stored data, and actually using the data are all extremely challenging in today's world.

Most of the time we frame this as AI for data — how AI can help us use data. "Ask what AI can do for you" — but what we really care about is data for AI, because building agents means agents accessing lots of different kinds of data.

So the relationship between AI and enterprise data is not one-dimensional. AI unlocks value from the data you already have, but the quality, structure and governance of your data determine what AI can actually do. Understanding both sides of that equation is the difference between AI that works and AI that may not.

The data layer underneath agents is crucially important and very difficult to build. Consider what agentic capabilities need: the many distinct types of documentation that exist, plus trajectory data, operational data, tool-use traces — everything your agents need to actually do what you want.

Where the difficulty lives (~01:28–01:31)

Core difficulties:

• Retrieval. It doesn't work unless the way you present your information is lexically close enough to the documents you care about — which typically doesn't happen. This necessitates a semantic data layer across knowledge sources: bring the representation of the data closer to how information is expressed, and make it so you don't have to care which data source it came from.
• Structure. NL-to-SQL, NL-to-other-formal-representations. Very difficult, partly because it also depends on retrieval.
• Multiple knowledge sources. Agents need access to many of them, and each requires its own expertise.
• Rich documents. More and more rich documents get stored — papers, figures, video, images — and the information inside them is not exposed to your retrieval unless you do something to expose it. Plus preprocessing your agents may or may not perform.
• Planning. How to access data, in what order, how to filter one source by another — very challenging (the tennis example exists to show this).
• Conflicting information. Think about your file system or your email: how much of it contradicts itself? And we want data-accessing agents to handle that well.
• Reasoning. In many cases beyond the capabilities of general-purpose models.

Underneath all of it sit governance, policy enforcement, reliability, consistency and auditability — unavoidable if you want to use large-scale data in a corporation or a medical center. Plus optimization: the first generation didn't care much about cost, but now we do — when to process data offline versus at runtime, how to cache computation, how to use query logs and history — all genuinely hard problems.

NL2SQL: top of the leaderboard, and what the absolute score says (~01:31)

He takes one slice in depth: NL2SQL. First a self-described shameless plug — some of Oracle's teams have competed in benchmarks like Spider 2 and Archer and done really well, leading the leaderboards.

But beyond the plug, look at the absolute results: Archer is around 55%, Spider 2 just in the low 70s — which means we are really far from being able to address the problem of accessing and using data properly.

The tennis example: a beautiful table with four things wrong (~01:31–01:35)

He follows tennis closely, so tennis is his running example. A few weeks ago he put a question to a top model: "Give me a list of the European male tennis players sorted by their tennis income. Only tennis income."

It's a genuinely complex question: the model must know who the players are, which countries they come from, whether those are in Europe, which tournaments they played in 2025, and how much they made at each. The model did show its work — which sources it went to, which tables, which pieces of text — and produced a nice table.

Before analyzing it, appreciate how cool this is. This is the promise: express an information need in natural language, and the model devises a plan, accesses many sources — structured, unstructured, web — uses parametric knowledge to know where countries are, reconciles a great deal of conflicting information, aggregates it, and summarizes it into a table. Beautiful.

Then look at the details:

1. Two entries are not sorted correctly. This was the most surprising mistake to him, because the model should have called a tool to do the sorting properly.
2. Someone is missing — the model forgot Carlos Alcaraz.
3. Two players' income is identical to the dollar. They made millions of dollars, and the figures match to the dollar. That cannot happen. Clearly a mistake.

Why? The model has to optimize access across knowledge sources. It can generate a list of players, filter to European countries, find which tournaments they played and how much they made; or it can start from the list of tournaments and look at who played and what they earned. There are far fewer tournaments than players, so maybe that's the right way — but tournament data is noisier. Doing it right really depends on understanding the data and the domain — possible in tennis, and in most cases simply impossible. So while agents do this faster than we can, they're prone to errors and high cost.

And then the step that matters most. He reminded the model it had forgotten Carlos Alcaraz; it added him — but look carefully and a few other things changed too. Andrey Rublev, number seven, completely disappeared from the new list. The money changed a little. Someone who had made over six million now makes less than six million. It is not consistent.

His actual point: this is tennis — who cares. And he knows the results, so he can look at it and decide whether it's correct. But if this were the same question about your company's financial analysis, or any other sensitive information? Then the notion of visibility of failure becomes really important. Without addressing it, we cannot use these kinds of systems reliably and consistently.

Closing (~01:35–01:36)

Real-world data is multimodal, temporal, multilingual, heterogeneous, and a great deal of work remains to unlock the information inside it. Understanding both sides of the equation — AI for human consumption over data, and data for agent consumption — is really the difference between AI that works and AI that disappoints.

Quotes

"There is a myth that information retrieval works, but you know, try to search your email and tell me whether it does work." (~01:26)

One line puncturing the RAG era's most common premise.

"Ask what AI can do for you — but really we care about data for AI." (~01:26)

The talk title's pun, and his position: the second half is what matters.

"While AI unlocks value from the data you already have, the quality, the structure, the governance of your data really determines what AI can actually do." (~01:27)

An enterprise's real AI ceiling is set by its data layer, not its model.

"This is tennis — really, who cares. … But if this was the same question that you ask about your financial analysis of your company, the notion of visibility of failure is really important." (~01:35)

The talk's actual thesis: the problem isn't that models err, it's that you can't see when they do.

"AI for human consumption over data, and data for agent consumption, is really the difference between AI that works and AI that disappoints." (~01:36)