Using Agents to Build Production AI Systems: Lessons, Pitfalls, and Limits

TL;DR

• Demos and production are different games. Production cares about maintainability, controllability, and completeness, and those constraints change how you can use agents at all. The honest metric: of 313 bug fixes authored by their agent, only 73 were accepted and landed on main — more than 100 were rejected by humans as incomplete, too reward-hacky, or too superficial.
• Split responsibilities, and move the reviewer earlier. Their multi-agent flow separates planning (plan generator + plan reviewer) from implementation (coder + reviewer + QA) so each sub-agent stays focused and spends context efficiently. Adding a plan reviewer alone cut convergence from 175 iterations to 35 at the same quality — a large token saving.
• Don't build a heavy harness. On the training side they started with a deep-agent-style workflow and deliberately retreated to "general coding agent + very lightweight harness," because the general coding agent's own evolution kept absorbing what they had hand-built. Staying light is how you keep riding the foundation model.

Key Points

Three inference-side case studies (~00:03–00:10)

The testbed is NVIDIA's large-model inference framework (heard as "the 10M" in the auto-captions; almost certainly TensorRT-LLM — see To Verify), used by many customers and behind key benchmark submissions such as MLPerf and SemiAnalysis. It is genuinely product-grade software, which raises the bar for agents considerably.

Bug-fixing autobot (~00:03–00:06). The workflow is two-phase. The agent phase runs bug analysis → reproduction → (once reproducible) agent fix → agent review → agent verification → mark the agent phase done. Then the human phase: a careful review to confirm the fix is not reward hacking, not hacky, and complete enough, with feedback fed back so the agent keeps iterating until it converges.

The dashboard numbers (~00:05:46) are the talk's most quotable result: of roughly 800 bugs, 313 have agent-produced fixes, but only 73 were accepted and merged to main, with over 100 rejected outright. Jun Yang treats this ratio as concrete evidence of the gap between production requirements and current agent capability — and as the number the team is actively trying to move.

Model bring-up (~00:06–00:09). New models arrive almost monthly, sometimes weekly, so they built an agent-based bring-up tool. Users declare the model definition and acceptance criteria in a task YAML via a CLI; the agent flow runs from there, and a human can interrupt mid-execution, supply feedback, and re-trigger to steer the agent back on course.

The flow is a textbook multi-agent split: a plan phase with a plan generator and a plan reviewer, and an implementation phase with coder, reviewer, and QA. Responsibilities were divided on purpose so each sub-agent stays narrow, uses its context window well, and shares common state through shared files.

Their Gemma example (~00:07:52) plots iterations 0–175 against a quality score. The planner kicks off execution, the coder and reviewer trade many rounds, a passing review hands off to QA for evaluation, QA sends results back to the coder, and the four-way interaction eventually converges. Adding the plan reviewer (~00:08:47) reached the same quality in just 35 iterations — their single most effective token-efficiency change.

Performance optimization (~00:09–00:10). After bring-up, another agent loop tunes performance. Starting from a weak baseline, iterations two and three each land a batch of optimizations and iteration four converges and stops. The interesting piece is the cookbook: the team's accumulated optimization know-how, written up in a structured form the agent can query — host-performance recipes in one iteration, CUDA graph recipes in the next. Notably, several completed optimizations were rejected by the workflow itself because the measured gain was too marginal.

Training side: from deep agents to a thin harness (~00:10–00:11)

Training shares some properties with inference but differs enough to warrant its own workflow. They began with a deep-agent-based flow and deliberately moved to a very lightweight harness on top of a general coding agent, because general coding agents had evolved to subsume what they had built. Light and adaptive is what lets them inherit improvements from general agents and foundation models for free.

Two additions are specific to training: a kernel arena that reuses historical kernels to make kernel generation easier, and a learning feedback loop where the agent does root-cause analysis on past runs, a human reviews it, and the result is fed back to improve the training agent. They also built an evaluation toolkit so that adding a new skill or sub-agent doesn't silently regress the workflow — though he was candid that this "is not ideal; we are still trying to figure out the best way to defend the agent system evolution."

The demonstration used FLUX, a vision transformer model (~00:11:20) that already had a strong human-tuned baseline. The agent ran over 120 experiments — some OOM'd, some ran but were rejected — and the surviving optimizations delivered roughly 20% performance improvement on top of that strong baseline.

Four lessons (~00:11:59–00:13:20)

1. Agent intelligence is fundamentally unlike human intelligence — it is a probabilistic process by nature, and every design decision should account for that to exploit the strengths and avoid the weaknesses.
2. Don't build a heavy harness. Keep the customization workflow as lightweight as possible so you can keep leveraging future evolution of general coding agents and foundation models.
3. Full autonomy vs. human-plus-agent is a real trade-off. You want to automate as much as possible, yet combining a human with the agent is frequently far more efficient in both token cost and iteration speed. The related tension is centralization: production instincts say centralize logic for reuse, but because agents evolve so fast, a human plus a locally customized agent sometimes beats the centralized agent. There is no single silver bullet.
4. Balance the coding agent against the lightweight harness so you can move quickly as foundation models and general coding agents improve.

Quotes

"Among the 313 back fix from agent, only 73 of them have been accepted and landed to our main branch." (~00:05:56)

In production software, "agent finished it" and "a human will accept it" are a factor of four apart.

"Rather we should try to make sure the customization workflow should be as lightweight as possible so we can be able to leverage the future evolution of the general coding agent and foundation model." (~00:12:20)

The heavier your harness, the more of it gets invalidated by the next general-purpose agent release.

"Human plus local custom agent can sometimes work more efficiently than the centralized agent." (~00:13:03)

Centralization is the software-engineering instinct; at current agent velocity, it doesn't always hold.