Is Kubernetes Good for Agents? Infrastructure Solutions for Agent-Shaped Problems

TL;DR

• Why agents fit Kubernetes badly: agent workloads are bursty (short spurts, then idle for minutes, hours, or weeks), generally untrusted (so they must run in sandboxes), necessarily single-tenant (you can't share a sandbox without defeating its purpose, which kills amortization of overheads), and human-in-the-loop (so they're very sensitive to perceived latency). Kubernetes was built for long-running, horizontally scalable things whose costs amortize.
• Only two patterns exist today: (1) Kubernetes runtime classes — the default OCI container, plus gVisor and microVM classes — with one sandbox pod per user session. Pod startup takes seconds, agent startup can add 10–15 seconds or more, so operators keep pods idling anywhere from tens of seconds to tens of minutes, burning resources nobody else can use. (2) DIY: one giant per-machine workload containing a manager, with a bespoke control plane scheduling into a mega sandbox — efficient, but you reinvent large parts of Kubernetes and it's hard for smaller companies to operate.
• Agent Substrate: Google's open-source project, started earlier this year, takes the best of both — suspend/resume plus cloud storage to multiplex agents onto pre-warmed runners and aggressively suspend them, driving 5–10 seconds to 10 minutes of idle down to a second or less. Targets: billions of sessions under management, thousands to tens of thousands of activations per second, and wake-up times in the low three-digit milliseconds. Still pre-production, aiming for a workable state by the end of the fall.

Key Points

What makes agents infrastructure-shaped differently (~00:45:30–00:46:40)

Hockin looks at agentic AI bottom-up, as an infrastructure problem. Customers arrive saying "we have these agents, we need to run a lot of them, and we're not getting the efficiency." Four properties matter:

1. Bursty — very short spurts, then idle for minutes, hours, days, or weeks. Handled naively, that's a lot of idle resources.
2. Untrusted — so they run in sandboxes, and new sandboxing technologies keep appearing.
3. Single-tenant — you can't share a sandbox without defeating the purpose, which means losing a ton of optimization opportunities, specifically amortization of overheads.
4. Human-in-the-loop — making them very sensitive to perceived latency.

Kubernetes, by contrast, was built for long-running, horizontally scalable workloads whose costs amortize. So asking whether it's the right platform for agents is a fair question.

The two patterns in the wild (~00:46:40–00:49:50)

Pattern one: Kubernetes runtime classes. A runtime class defines a kind of sandbox — the default being a Docker-style OCI container, with classes for gVisor and microVMs, and it's an extension point, so there's room for research. Some event (a user starting a chat session) requires a new agent instance, which means a dedicated single-tenant sandbox pod.

Starting a pod is not the thing Kubernetes does fastest — it's seconds, which isn't fast enough — and depending on what the agent does at startup (loading runtime environments, browsers), it can take 10 or 15 seconds more. Once the pod is ready the user chats, one turn completes, and then what? Since pod startup isn't free, people wait for another event. How long? It depends on the agent and the startup cost, and it's different for everyone; Hockin has seen tens of seconds at the low end to tens of minutes at the high end. Through all of that the agent holds resources nobody else can use. The next turn might land at 9:59, restarting the whole cycle; eventually a timeout declares it idle and the pod is discarded or its state saved and terminated.

And this is happening for every instance of every agent for every user on the system. Warm pools and suspend/resume tricks exist, but Kubernetes doesn't support them natively, so you're working around the system — hard to justify at scale.

Pattern two: DIY on Kubernetes. A giant workload per machine, containing a manager application; a bespoke control plane schedules into that mega sandbox and the manager handles lifecycle. Everything is opaque to Kubernetes, and you end up reinventing large parts of it. Very efficient, fairly complicated to build, and difficult to operate — especially for startups.

Agent Substrate (~00:49:50–00:55:00)

Twelve years into Kubernetes, Hockin holds "a deeply held belief that when it comes to infrastructure, open always wins." So earlier this year they set out to build something taking the best of both models — leaning on Kubernetes where possible, since nearly everyone has some in their fleet, and building new components where necessary. He's explicit that there's no earth-shaking research here: they took the best ideas from their customers and users and glued them into something everyone can use.

Agent Substrate is that project, and it's open source. The model: no idle resources. Use suspend/resume plus cloud storage to multiplex agents onto pre-warmed runners and aggressively suspend and resume them, driving 5–10 seconds to 10 minutes of idle down to a single second or less.

Goals, from a Google-scale vantage point: billions of sessions under management, thousands to tens of thousands of activations per second, and wake-up time in the low three-digit milliseconds.

Vocabulary: an actor is the stand-in term for an agent, sandbox, or agent-like thing. An actor template is the cookie cutter to the actor's cookie — it carries details like which sandbox technology (gVisor or microVM), and a single substrate can hold many different templates. A worker is usually a pod in a Kubernetes cluster and is the thing that actually consumes resources; it waits for an actor assignment, runs it, and can run many in serial, with many workers per cluster. atelet is the per-node manager for all workers on that node (needed for networking reasons), and atenet is their "enlightened proxy," which triggers wake-ups on receipt of traffic.

How it fits together: start with any modern Kubernetes cluster — kind is supported, so it runs on a laptop. The cluster admin deploys an Agent Substrate instance: API server and storage layer, a controller that synchronizes data in and out of Kubernetes, one or more worker pools (which create deployments, which create pods), atelets to manage the workers, and the atenet proxy that moves traffic from outside to inside. The substrate admin then creates actor templates; the controller spins each one up, takes a golden snapshot, and stores it for later.

At the substrate layer (Hockin warned that some of this is aspirational and still being built), a higher-order system creates an actor from a template — note that creating an actor doesn't run it. In a chat app, the front end does its authorization work and creates the actor. The user sends a prompt, which routes to the atenet proxy; the proxy wakes the actor and assigns it to a worker via the router; since this actor is new, it wakes from the golden snapshot; the proxy forwards the message to the now-running actor. The whole thing happens in a couple hundred milliseconds, and they're still optimizing.

As soon as the turn is done the actor is probably idle, so they pause it: take the snapshot, shuffle the data off, and the worker becomes unassigned and available to another actor — all at the speed of a single API operation or agent transaction. The next prompt restarts the cycle; when messages stop, the data goes to cloud storage and the actor sleeps for the night.

From the top: a Kubernetes cluster can be one node on a laptop or 200,000 nodes in the cloud, each node runs multiple workers, and each agent is idle 99.999% of the time — which is how the arithmetic reaches billions.

Status and ask (~00:55:00–00:55:30)

Agent Substrate is open source and still pre-production grade; they're working hard to have it in a workable state by the end of the fall. The most useful feedback they can get, he said, is experience, requirements, and context about what people are actually doing with agents.

Quotes

"Kubernetes was built for things that run for a long time and tend to be scale out, horizontally scalable, and you can amortize your costs." (~00:46:29)

The whole mismatch in one sentence.

"I have a deeply held belief that when it comes to infrastructure, open always wins." (~00:49:56)

Twelve years of Kubernetes, applied as the rationale for open-sourcing Agent Substrate.

"Each of those agents are idle 99.999% of the time, so you can do the math and see we're driving up to those billions of numbers." (~00:54:50)

The economics behind the entire suspend/resume design.