Panel: Agentic AI Infrastructure & Platform

Framing: from chat to machine speed (~01:35)

Todd Graham set the scene: the ChatGPT moment made chat the primary interface — one request, one response, relatively short context, short bursts. Agents run at machine speed, sometimes for hours ("you've got 12 of them running code overnight for you"), with massive tool volume and token prefill. He asked who had set an agent running overnight and woken up to an auto-refilled token balance. Nearly every hand went up.

Topic 1: the single biggest failure point in agent infrastructure today (~01:36–01:39)

• Chuan Li: the answer is in the question. Put machine-speed traffic on a road designed for human drivers and what breaks first is road capacity, traffic lights, and speed limits. The opportunity is turning those choke points into checkpoints.
• Jonathan Cohen: the crux is the LLM component — surprisingly reliable, but also capable of surprising and unpredictable behavior. The opportunity is surrounding it with rule-based policy enforcement, more deterministic systems and memory systems that give you confidence and certainty about what it's actually doing.
• Saurabh Tiwary: the answer keeps changing and evolving. At the compute layer, remove as many bubbles as possible for maximum efficiency. At the infrastructure layer, requests arrive from all over the world, so routing has to smooth out spikes to keep compute efficient at low latency. Voice is becoming an interesting engagement medium with a very different profile from raw text, so managing heterogeneity of both use cases and compute environments is a problem. Sovereignty requirements are becoming much more important and in a way break the shared-cloud philosophy, adding significant engineering complexity. Layer identity and security on top, and these are becoming very big problems very quickly.
• Peter DeSantis: "My answer is all of it as well." It's a systems problem from the very top of the stack down through guardrails, infrastructure, power, and the chips running it — and you have to get it all right, because agents have enormous potential but without efficiency we only capture a fraction of it.

Follow-up: which is the biggest rock to move — where do you point someone who wants to build today? DeSantis: "I'm an optimist, so I don't worry. I look for opportunity, and I see opportunity at every level. You've heard about 16 problems and a hundred constraints — pick one, go deep, and have fun." (Graham, deadpan: "Maybe build a memory manufacturing plant tomorrow afternoon.")

Topic 2: how should hardware specialize for agent workloads vs. LLM workloads? (~01:39–01:42)

Jonathan Cohen carried this one:

• More context, and the ratio of what's cacheable versus not changes.
• Heterogeneity: an agent is an LLM surrounded by computer stuff, and a lot of that runs on CPUs, possibly multi-core. Plenty also runs on GPUs — but GPUs configured for different kinds of workloads, not neural network inference; a physics simulation, say.
• So a data center running agentic workloads is very heterogeneous and complicated, with many processing elements, many storage layers, and complicated network topologies.
• Layered on top: security, privacy, and sovereignty requirements.
• Frankly, the world still doesn't quite know the best way to do this, because agentic AI is still evolving. This has happened repeatedly in AI's history — we think we understand it, then a new thing shows up and the workloads look totally different and get more complicated.
• His conclusion: "Agentic AI workloads are significantly more complicated, heterogeneous, and expensive computationally than anything we've ever seen before."

Topic 3: fine-tuning or context engineering for workflow-specific enterprise agents? (~01:41–01:43)

Saurabh Tiwary laid out a ladder from simple to expensive:

1. Start with context engineering — the simplest option. Base LLMs already have enough reasoning power (Gemini and others), and with a long context window you can put in a lot of data for the model to reason over. The quality of reasoning over context has improved.
2. To actually perturb the model, start with LoRA fine-tuning — cheap, easy, and efficient and cost-effective on the inference side too.
3. Above that, full fine-tuning or post-training — here you perturb all of the model's weights, which changes the inference dynamics because you effectively have a brand-new copy of the model, making it much more expensive.
4. Very custom work happens only with a few very special partners.

The first two, in his view, are the primary paths.

Topic 4: does experiment tracking live inside the infrastructure or in a separate observability layer? (~01:43–01:45)

Chuan Li: it's definitely infrastructure, but different from the environment people train in.

• The environment is where the agent acts and learns; the experiment tracker is more like memory and measurement sitting beside it.
• His framing: the environment trains your first brain — learning goes into model weights, compressed, internalized, and capacity-bounded. The experiment tracker gives you a second brain — learning goes into artifacts, and it's unbounded.
• There's a path from the second brain back to the first: turn your system of record into a dataset, then fine-tune or run RL on it.
• Where to host it depends on your data containment policy — self-host if requirements are strict, use a managed service if not. But if you want to use it for training or fine-tuning, it should be close to your training.

Topic 5: where is margin durable in the agentic stack? (~01:45–01:48)

Graham's question: sitting between the model and the agent's outcome you have orchestration, sandboxing, runtime, memory, identity, eval — if you're greenfield today, do you go after memory or sandboxes? Where does the money actually get made?

• Peter DeSantis (interjecting): "At the hardware."
• Saurabh Tiwary: AI got adopted because of the quality of results and the quality of the developer/builder experience. The business opportunity is obviously massive, and — echoing DeSantis — there are tons of problems; pick one and solve it very deeply. If you deliver on your promise to the customer at high enough quality, the value gets extracted there. Take memory: it isn't about "working on memory," it's about whether you give customers a compelling experience that lets them actually deliver what memory is supposed to enable, rather than "I'm using memory and here are some prompts that got updated." That value exists across the stack — hardware, model, agentic, and application layers. And with almost every CXO at every major company thinking about AI, the opportunity is huge; it comes down to delivering value.
• Chuan Li: agrees on hardware, and adds vertical integration. Coming from a neocloud, he notes people usually assume cloud computing is a commodity — largely true, but cloud computing is very hard: land, power, data center buildout, HPC architecture, orchestration, software, and finance form one vertical stack. The agent layer — call it sandbox, call it harness, call it system of record — is a new layer to be integrated into that vertical. Whoever does a good job there is probably going to keep the margin.

Topic 6: neoclouds vs. hyperscalers (~01:48–01:50)

• Chuan Li, answering from the neocloud side: the market is big enough for multiple players, so he's happy to be a friendly peer to NVIDIA, Google, and AWS alike. A neocloud doesn't need to invent everything from scratch — its play is focus. Training and inference workloads never fit a general-purpose compute cloud, which is why so much innovation happened there: GPUs, fast chip-to-chip interconnect, specialized storage and orchestration like Kubernetes. What neoclouds added was making those options the default and offering them to the general public. That's how the training and inference cloud got built, and it's the same thing they now intend to do for the agent cloud.
• Peter DeSantis, returning to the word "durable": "I'm not sure the label neocloud is a particularly durable label." Demand for compute is huge; hyperscalers have delivered a lot of it, and there's real space for insurgent neoclouds — they've done impressive, interesting things. He doesn't think there will be 80 neoclouds, but he does expect some new businesses that look a lot more like hyperscalers over time. As an incumbent, the thing to guard against is being stuck doing things one way in a manner that disadvantages you — if you have an established cloud business, listen to your customers and respond. That's the perennial tension of big and small, incumbent and insurgent, with pros and cons on both sides.

Topic 7: what's missing from agent communication protocols beyond MCP? (~01:50–01:54)

• Jonathan Cohen: what needs working out is the hierarchy of agents — an agent spawning sub-agents spawning sub-agents, or spawning a team that works together. Emerging best practices around communication patterns imply a scope on some kind of shared memory: a team of agents might share a workspace that another agent has no access to. So there are new standards to be developed describing scoped shared storage systems, access, and message passing between agents. This differs from what we've built before, which was either services or human-scale: now an agent can, ad hoc, spawn 100,000 sub-agents, all needing to communicate in some complicated pattern. Those standards have to be invented, enforced, and implemented efficiently.
• Saurabh Tiwary, along the same lines: MCP gives you an interface for an agent to talk to a data source. What's also needed is multiple agents talking among themselves — shared memory plus communication — which is the direction of the agent-to-agent (A2A) protocol, which exists but needs more adoption. On the commerce side, agents need to start supporting payments and commerce, which opens a huge new opportunity; Google has ACP (agent commerce protocol). These need to be more fully absorbed into the ecosystem.
• Peter DeSantis: there are different ways to think about standards. De facto standards emerge where things become efficient, and the industry will trend there. But he'd hope we resist standardizing where we don't need to in the short term. Some places genuinely need it — security, low-level wire protocols where many things must talk together — but AI is pretty elastic, and agents will find efficient ways to collaborate; we'll then have to find the tools and systems that make them more efficient. "Hopefully we resist low-level standards where possible for a while. I think we want the innovation."

Topic 8: lightning round — what do you hope someone's building, and what would you steer them away from? (~01:54–01:57)

• Chuan Li: give someone a fish and you feed them for a day; teach them to fish and you feed them for life. Model weights and the solution currently in your hand are the fish — don't hold on to them, the value depreciates. What compounds over the long term is how you get there: the tools, the environment, the system of record.
• Jonathan Cohen: honestly, he can't name a thing someone should do that nobody is doing — every single thing he can think of, there's a company doing today, seven of them vertically and horizontally. And that's great: agentic AI is clearly the new frontier of AI, with tremendous room to improve and innovate from security to storage to vertically integrated domains.
• Graham's editorial addition: if you're building something, assume you're not the only person with the idea — you need to really understand who you're competing with, what color the ocean is, and how you'll differentiate when you go in.
• Peter DeSantis, echoing his keynote: in these frenetic times there's a risk of looking at the progress being made and thinking a problem is solved. The other side of that coin is just as dangerous — watching how the models are evolving and what the large labs are doing, you might conclude there's no space there at all. Resist that, because what looks like a solved problem today will be unsolved again. "I know that's important to you as a VC, but I deeply believe this as a technologist: this is very early days."
• Saurabh Tiwary, for people building end products or full agentic solutions: an existing business process has often evolved over many years into a multi-stage pipeline, and the common move is to sprinkle AI into each piece of that pipeline — at which point the benefit you extract from AI is heavily diminished, because you're thresholded by the limitations of every stage. His trivial example: license plate identification for parking — take image, remove background, and so on down the pipeline, sprinkling AI on each step. Instead: look at what AI's core capabilities are, always stay on the cutting edge (capabilities keep changing — don't be static), revisit the entire business process from the ground up, ask what it would look like if you built it AI-native, and build that. That's what delivers significant value.

Quotes

"The thing that break first are like road capacity, traffic light and speed limit. The opportunity there is how to turn those choke point into a checkpoint." — Chuan Li (~01:36)

Put machine-speed traffic on a road built for humans and the road fails before the cars do.

"Agentic AI workloads are significantly more complicated, heterogeneous, and expensive computationally than anything we've ever seen before." — Jonathan Cohen (~01:41)

The panel's consensus sentence.

"An agent can in an ad hoc way spawn 100,000 sub-agents, all of which need to communicate in some complicated pattern." — Jonathan Cohen (~01:52)

Why service-shaped and human-scale standards both run out.

"I'm not sure the label neocloud is a particularly durable label." — Peter DeSantis (~01:49)

He credits the newcomers' work while predicting the category itself dissolves.

"Hopefully we resist low-level standards where possible for a while. I think we want the innovation." — Peter DeSantis (~01:54)

The clearest disagreement on stage: standardize now, or let it grow first.

"This is very early days." — Peter DeSantis (~01:57)

Back to the day-one framing of his keynote.