Making Autonomy Autonomous: Toward Mental Models for Discovery and Intuition

TL;DR

• The setup: physical AI lives in the open world, edge cases keep arriving after deployment, teams run continuous discovery-and-development loops to fix them — and every fix must ship with traceable release evidence.
• Three nested loops: the execution loop (data, simulation, DevOps → a verifiable substrate), the discovery loop (grounded ideation, verification, traceability), and the intent loop (tacit expertise, mental models, trusted attribution, co-evolving partnerships).
• The demonstration: from a single line of developer input — generate ideas to improve 3DGS for pedestrians in autonomous driving, test on this GPU family, validate on real benchmarks, compare to baselines, publish a paper — a team of agents ran the whole pipeline. Three of four submissions to a CVPR 2026 workshop were accepted (the rejection had equation formatting problems).
• The point isn't a paper-writing system but a system doing traceable, verifiable scientific discovery: every candidate method mutates on top of a verified reproduction, and every claim traces back to a real log, a real line of code, a real GPU run.

Key Points

The problem: endless edge cases, but every fix needs evidence (~00:55–00:57)

Physical AI lives in the open world. Edge cases arise endlessly after deployment, teams go into continuous discovery and development loops to fix them, and all of it must be fixed with traceable release evidence.

Meanwhile the physical world is rapidly becoming executable: CI/CD frameworks run loops continuously, agents deploy code, simulation bridges the gap to data. All of this points to a future of self-improvement.

The friction is real, though: physical stacks are complex and require real-world experimentation and real-world knowledge, and human expertise lives beyond static data — in scientific ideas, expert workflows, and physical interaction with systems. So experimentation and discovery in physical workflows demand a lot of compute, a lot of data, and a lot of human talent.

His three themes: making autonomy autonomous in collaboration with humans; discovery frameworks grounded in execution; and reflecting human intent in agentic workflows so people keep participating in AI-native economies.

The demonstration: one line in, a paper out (~00:57–00:59)

The scenario: an autonomous driving team observes a pedestrian perception issue → hands it to the simulation team → pedestrians are thin structures, so better 3D reconstruction is needed → the team proposes a new Gaussian splatting method → it's validated on real data → they write a paper or publish a spec sheet.

"Now what if I told you that this whole process is something that is run by a team of agents?" The only developer input is one instruction: generate ideas to improve 3DGS for pedestrians in autonomous driving, test on a particular family of GPUs, validate on real benchmarks, compare to baselines, and publish a paper.

The agents' roles carry explicit requirements: produce research that is verifiable, deploy code that is executable, ground ideas in the physical world, and follow a discovery process that is traceable and verifiable.

The flow: input arrives → agents define the task to be solved (here, improving NVIDIA's OmniRe) → they form hypotheses about dimensions where improvement is possible → specialist agents launch a set of experiments → all experiments are validated or falsified in an Elo-rated tournament → the highest-Elo idea survives as a new method, implemented on real GPU hardware so the gains are measurable → writer and reviewer agents produce a paper.

How good is it? They submitted several of these papers, with the organizers' permission, to a CVPR 2026 workshop on physical AI — "we're not trying to game the system." Three of four were accepted, one with high praise; the rejected one had formatting problems with its equations, "so we believe the reviewer, in this case."

The framing matters to him: this is not a system for ideating and writing papers, it's a system performing traceable and verifiable scientific discovery.

The verifiable substrate (~00:59–01:02)

Everything rests on a verifiable substrate, built from three commitments:

1. Grounded ideation — a context-free grammar constrains the ideas the system can generate.
2. Mutation on verified reproductions — every candidate method mutates on top of a verified reproduction, so everything is measurable, validatable, falsifiable.
3. Traceability — every claim traces to a real log, a real line of code, a real GPU run.

How do you build it? By reverse-engineering the process: given a knowledge repository and papers, can you follow the scientific process backwards to produce the code that wrote the paper? His example uses a paper that has never been publicly implemented ("so it's not like Claude or GPT knows about this paper"): analyze the paper → determine the new ideas and the factors that produce improvements over baselines → generate the code → physically verify it against real data → suggest improvements → validate on real benchmarks against real data.

For a full application like autonomous driving, the execution loops that build these substrates already exist from years of prior work:

• Data agents analyze edge cases given a toolbox of computer vision and machine learning tools.
• Simulation agents take user input ("get a car to change lanes, cut in or cut out"), reconstruct the 3D background and all dynamic agents in the scene, deploy diffusion models to generate novel reactive behaviors and challenging edge cases, and use further diffusion models for photorealistic rendering.
• Development agents take ODD specs plus the data and simulation tools and run AI model training and validation.

The human role: the intent loop (~01:02–01:04)

"Once we have these autonomous systems that are driving autonomy, what does it mean for the role of the human?"

He sees new loops opening up as data moves from static to interactive. We begin harnessing tacit expertise, which gives a handle on the lived experience behind expert behavior. Lived experience drives mental models, which let us reason about not just how experts act but why they act.

Intents drawn from those mental models then drive world models that no longer work from aggregated preferences alone but carry traceable contributions. Attribution mechanisms enable trusted adoption, which enables continuous human participation in the AI loop.

Building the mental models: ingest all traces from the discovery pipelines, convert them into evidences via claim-backed verification, distill evidences into beliefs, and attach attribution to every use of every belief.

Two payoffs. On coding tasks: more tasks solved, solved better — and importantly fewer user interventions, fewer memory tokens, fewer tokens consumed, which points at better user-system alignment. On physical tasks like Gaussian splatting: mental models can adopt different personas — a quality persona and a speed persona, say — which emerge from the discovery loop and can be steered controllably.

His summary of the architecture: execution loops built on data, simulation, and DevOps form the verifiable substrate; discovery loops run on top with grounded ideation, verification, and traceability; and intent loops add tacit expertise, mental models, trusted attribution, and eventually co-evolving partnerships.

Quotes

"Now what if I told you that this whole process is something that is run by a team of agents?" (~00:57)

The pivot from human research to agent research.

"Every claim that is made by the system is something that we can trace to a real log or a real line of code, a real GPU run." (~01:00)

The line that separates automated science from automated hallucination.

"Mental models … allow us to reason about not just how experts act but why experts act." (~01:03)

Tacit expertise is valuable for the reasons behind the actions, not the actions.