Harness Engineering: How to Build Software When Humans Steer and Agents Execute

TL;DR

• Capability overhang: even holding the model and its containing harness constant, models are far more capable than they are able to side-effect into the world. What's missing isn't intelligence — it's that they don't know what local and global good looks like to their operators, and lack the context for full autonomy inside the organizations they're deployed in.
• He hasn't written code since June 2025, and neither has anyone on his teams — writing code is "not a permitted activity." The only thing humans are permitted to do is get the agent to do the parts of their job that need doing.
• Once implementation is abundant, three things stay scarce: human time, the model's context window, and attention on both sides. So track your own time ruthlessly, identify what you keep doing, and engineer your way out of doing it — that self-improvement loop is how agent autonomy grows.

Key Points

An opening correction, and the capability overhang (~02:49–02:50)

The host introduced him as a member of technical staff at OpenAI leading a project called Dark Factory. His first line onstage was a correction: "as of two weeks ago I am at Google" — with a disclaimer that the talk was his own view, not Google's.

He then defined harness engineering: even keeping the model and its containing harness constant, we are in a capability overhang — the models are far more capable and far more intelligent than they are able to side-effect into the world today. The gap isn't intelligence:

• They don't know what local and global good looks like to their operators.
• They don't have the context to have full autonomy within the organizations where they're deployed.

So the human role is stewardship — giving agents the tools, context, guardrails, coaching, and trust necessary to fulfill the fullness of the job we want them to do.

The bet: humans aren't allowed to write code (~02:50–02:52)

His reason for being onstage is a call he made more than a year ago: the earliest reasoning models were already capable of doing his full job. He put money where his mouth was — as of June 2025 he stopped doing his job, hasn't written any code since, and neither has anyone on his teams. It's simply not a permitted activity. The only thing humans are permitted to do is get the agent to do the parts of their job that need doing.

In June/July 2025 the models were much weaker and the proposition much harder — he couldn't get a model to read Slack and respond to pages on his behalf; tool use, capability, and complex orchestration weren't there. The method was to drill down: double-click into the task, and double-click, and double-click, until you bottom out on something the model can do. Reassembling those capabilities back up the stack permanently accrues value to the agent's ability to reliably and safely side-effect into the world. And when teams of people collaborate on a single agent this way, you get the best parts of everyone.

That's what he means when he says the way we build software has changed: you can take an agent at any capability level and solve problems with how it executes by writing code — because code is free to produce now. Models spike very highly at producing it.

But in a world where implementation is abundant, a few scarce areas still need investment so the agent coheres over long timelines. "It is not the case today, even with models as advanced as they are, that you can say 'make me a billion dollar business' and end up with something coherent at the other end. They are still struggling to operate vending machines." Human expertise exists to constrain the regions of latent and physical space we permit the machine to enter, keeping the system tracking in the right direction over time. He generalizes the problem as: what does it mean to continuously evolve an artifact — a codebase, a Word document, or a Confluence-sized wiki of an organization's knowledge?

Three near-term scarce resources (~02:53–02:55)

1. Human time. This is foundational to how we've built organizations so far: platform teams and central dashboards exist to concentrate a scarce pool of human labor into high-leverage things that empower an organization. The same constraint holds with agents. - His most quotable operating rule: humans interacting with agents must be incredibly ruthless about tracking their time and identifying what they find themselves doing — going back and forth one-on-one to put a plan together, reviewing code, rejecting slop — and then figure out ways to stop doing it. - That self-improvement loop is what increases the autonomy of the agent side of the system, permitting more complex, more autonomous, and more parallel work over time.
2. Model context window. A foundational limitation of what it means to put a model together. Context windows may grow and auto-compaction may improve, but coherence over long-horizon work within a single trajectory remains something you have to keep in mind.
3. Attention — human and model. Both will be constrained; that's a necessary fact of the world. So the way we structure tasks and work has to account for humans and models being able to focus rather than scattering attention across many competing concerns.

A concrete heuristic: "if I find I need to intervene more than three times with an agent, I'm probably going to have a bad time." Sometimes he'll deliberately go through the bad time anyway — to learn where the agent failed to account for what he needed — and then back-propagate that into the environment he provisions for the agent, so the next time he rerolls the task it pulls the right bits of context exactly when needed instead of scattering its attention.

Harness engineering is onboarding (~02:56)

He closed on the analogy the whole talk keeps returning to: discussions of good parallel autonomous agentic systems keep landing back on how to build good organizations, because the principles for empowering humans in a complex organization apply to empowering agents.

Think about onboarding: you're hiring generally capable humans, but they don't necessarily know what good looks like for you in this context.

So — surfacing the collections of non-functional requirements that go into making good local work for you is the name of the game of harness engineering; and curating context, in the background and via tools, to constrain how the agent works is what chasing a well-constructed agent harness means.

Quotes

"The models are far more capable and far more intelligent than they are able to side effect into the world today." (~02:49)

The definition of capability overhang: the bottleneck is side-effecting into the world, not intelligence.

"I haven't written any code since then, and neither has anyone on my teams. It's just not a permitted activity." (~02:50)

Not a recommendation — a report on a policy that's been running for over a year.

"Code is free to produce now." (~02:52)

The premise underneath everything else he argues.

"You can say 'make me a billion dollar business' and you will not end up with something coherent at the other end. They are still struggling to operate vending machines." (~02:52)

An honest calibration on long-horizon coherence, and why human constraint is still load-bearing.

"They must be incredibly ruthless by tracking their time … identify what they are doing and then figure out ways to make it so they don't do that." (~02:53)

The most actionable rule in harness engineering.

"If I find I need to intervene more than three times with an agent, I'm probably going to have a bad time." (~02:55)

A useful stop signal: past three interventions, the problem is the environment, not this prompt.

"You're hiring generally capable humans, but they don't necessarily know what good looks like for you in this context." (~02:56)

Agent harnesses and employee onboarding are the same problem.