Trustworthy Agents in the Real World: Physical Autonomy Lessons for the Agentic AI Era

TL;DR

• Waymo is an agentic AI company. Thousands of agents run very long-horizon tasks to completion in the real world for real paying customers, fully autonomously, across 11 cities. The goal: autonomy at scale, and becoming "the world's most trusted driver."
• Trust is more than safety — it's consistency, repeatability, predictability, which are exactly the factors behind trusting any agent.
• Reliability is a game of nines. Going 99% → 99.9% → 99.999% typically demands a different solution each time: you don't buy a nine by improving the system a bit or swapping in a better model, you buy it by redesigning the system for that reliability level.
• Level 3 is a weird beast. Mixed human/agent authority is bad for cognitive load, safety, and productivity — and L2 and L4 are different species; you cannot graduate from one to the other. Both lessons replay exactly in coding agents.
• Step-wise optimization drifts (the DAgger problem). Minimizing per-step error in a multi-step system produces correlated errors and lands you somewhere else entirely. Hence long-horizon planning, RL, and different objective functions.
• Waymo World Model, built on Google DeepMind's Genie 3 and adapted to Waymo's cameras and lidar, delivers realism, controllability, and generalization for closed-loop long-horizon evaluation.
• The closing question to virtual-agent builders: what is your world model?

Key Points

Waymo as an agentic AI company (~00:29–00:31)

He opens by claiming the frame outright: "Waymo is an agentic AI company — except our agents are big beautiful hunks of metal that have four wheels and are driving around in your neighborhood." Thousands of agents, very long-horizon tasks run to completion, real paying customers, fully autonomous, 11 cities today with plans for many more in the US and abroad.

The mission is autonomy at scale and being the world's most trusted driver — and he insists trust is not just safety: consistency, repeatability, predictability, "a lot of the same factors in the trust that we place in agentic agents in general."

His overall thesis: physical AI and agentic AI rhyme heavily. "If I talk to my roboticist friends and they look at what's happening in agentic AI today, they look at it with a smile, because all of the hard problems we've been experiencing for the past 20 years in robotics are exemplified in agentic AI." The shared problem: build a system with enough affordances to do interesting things in the real world that you can also trust, and that evolves in the real world safely, stably, and consistently.

Lens 1 — Industrial automation: the game of nines (~00:32–00:33)

Industrial automation is the art of stringing components that mostly work but occasionally fail into a line that delivers very high reliability, safety, and autonomy over long horizons at scale. His one lesson from it: it's all a game of nines. At small scale 99% repeatability is survivable; at scale you need 99.9%, then 99.999%. Every nine typically requires a different solution — and the crucial part: you don't earn a nine by improving the system a little or reducing the error rate. You redesign the system for the reliability level you want.

Lens 2 — Levels of autonomy: L3 is a weird beast (~00:33–00:35)

Driving has five levels, from no automation through driver assistance up to Waymo's L4. Coding agents have the same gradation, from hand-writing every line to "you just YOLO it and let the agent write everything."

Two transferred lessons:

1. Level 3 is a weird beast and an unpleasant place to be — bad for cognitive load, safety, and productivity. The same showed up in agentic AI when people moved from code completion to agents: "sometimes the human is in charge, sometimes the agent is in charge, and this interface between the two is very uncomfortable."
2. L2 and L4 are different beasts. Different design decisions, different approaches; you cannot incrementally graduate from L2 to L4 — you have to rethink the system dramatically. Likewise, today's agentic systems are not a scaled-up version of the code-completion systems you used to use.

Lens 3 — Dynamical systems: closed loop needs a different reward (~00:35–00:37)

Agentic systems are, in his framing, extremely nonlinear dynamical systems. The math from control theory may not transfer directly, but the concepts do: observability, controllability, stability, and realizability — the last being "can you actually do the task."

The lesson: optimizing for closed-loop behavior (long horizon) requires a very different reward than optimizing open-loop. In robotics this is the DAgger problem: minimize error at every step of a multi-step system and you end up making highly correlated errors, so the system lands somewhere completely different from where you wanted it. "Optimizing one step at a time hasn't really worked." That's the reason for long-horizon planning, for RL, and for treating long-horizon tasks with different objective functions.

Their approach: still optimize one step at a time, but imagine the future — dream up what the long horizon would look like under a specific action. That is where world models enter.

The Waymo World Model (~00:37–00:43)

Waymo built its own — named, with self-deprecating flourish, the Waymo World Model ("we're very, very creative"). It enables closed-loop evaluation and simulation of longer-horizon behavior at high fidelity.

Realism. They started from a strong video generative model, Google DeepMind's Genie 3, which "has some notion of physics, but it's got all the notions of physics — it can handle Minecraft physics as well as real-world physics." Waymo wants real-world physics and real-world visuals only, so they fine-tuned and adapted the model to Waymo's use cases: all the cameras on the car, plus lidar generative capabilities grafted on. He showed a 30-second fully imagined rollout.

Controllability. "You want to be able to imagine counterfactual events — dream up what is possible if you take any action." Instead of turning right, turn left or go straight, and still get high-fidelity rendering. Control operates through language (same scene in the morning or at night, cloudy or foggy, changed appearance), through driving actions, and at fine grain — conditioning the whole model on every car, pedestrian, and piece of lane geometry, so you can ask what happens if a car moves one lane over.

Generalization. "If we only simulate the kind of data we see in the real world today, it has some usefulness — but a world model really shines when you extrapolate to scenarios you've never observed, or potentially never want to observe."

The elephant example is the clearest version of this. Waymo has never seen an elephant on the road, so: train naively on Waymo data and you get no elephant at all; tweak for generalization and world knowledge and it tries very hard and fails in funny ways — a truck that looks like an elephant, because it knows trucks well and has some notion of elephant appearance but can't reconcile them; do it right and you get a full elephant on the road, with the lidar rendering it plausibly despite never having seen one.

Other extrapolations: driving through a raging fire (never experienced, not implausible), snow on the Golden Gate Bridge, and someone in costume walking past — which is very realistic, since "every Halloween we will see people in costume walking around in various stages of inebriation."

That last one carries the safety point: the system must register that the thing on the road is a human being, a very vulnerable person — not "a random moving blob." "A very significant amount of safety derives from us understanding the semantics of the scene, and we need to be able to validate that."

Closing (~00:43)

"Physical AI is agentic AI taken to the next frontier. Lots of lessons from robotics, from automation, from autonomy can be applied to non-physical agents as well." Then the question he leaves with the room: what is your world model? Do you have a high-fidelity simulation of your environment — because that is the key to long-horizon planning and to building resilience and long-range autonomy into your system.

Quotes

"Waymo is an agentic AI company — except our agents are, you know, big beautiful hunks of metal that have four wheels and are driving around in your neighborhood." (~00:29)

The framing that licenses every analogy in the rest of the talk.

"Every nine that you earn is not earned by just improving your system a little bit, getting a better model or reducing the error rate. You have to redesign your system for the level of reliability that you want." (~00:33)

The sharpest rebuttal to "we'll just tune the model a bit more before shipping."

"Level three is a weird beast. Level three is not a very pleasant place to be. It's very bad for cognitive load. It's very bad for safety. It's very bad for productivity." (~00:34)

The half-automated handoff hurts identically in driving and in coding agents.

"I want to ask the audience that is working on virtual agents: what is your world model?" (~00:43)

The talk ends on this question rather than a conclusion.