Agentic Modeling via Internalizing Agent Harnesses

TL;DR

• Three definitions up front: agentic modeling (turning a talker into a doer), agent harness (the hardcoded module layers wrapped around the model), and internalization (distilling the harness's capabilities into the model itself).
• Agentic modeling needs three pieces: the harness (information / execution / feedback layers), the environment (an RL environment where agents act and emit trajectories), and the trainer (mid- and post-training on those trajectories).
• A new data flywheel: harnesses generate trajectories → trajectories train models that are better at doing → better models let us build better harnesses. "Harness is like the training data to agentic modeling, just like the internet data to model pre-training."
• The open-research bottleneck is the environment, not the harness: community infrastructure has focused on improving harnesses, but nobody can run hundreds of thousands of agents in parallel to produce training data. Orchard exists to close that gap.
• One project per layer: Orchard for environments (built on the Kubernetes-native Orchard Env), EvoLib for the harness (test-time learning via an evolving knowledge library rather than parameter updates), and GFlowRL for the trainer (policy learning as distribution matching rather than reward maximization).
• Cost efficiency is the training-side keyword: not just cheap serving, but sample efficiency — training a capable model without a mountain of samples.

Key Points

Defining the three terms (~00:02)

• Agentic modeling transforms large-scale language models into autonomous agents capable of complex tasks through interaction with an environment. The stereotype of an LLM is a talker: it seems to know everything, but ask it to do something and things fall apart. Agentic modeling is the process of turning that talker into a doer.
• Agent harness: since the model itself can't do much, you couple it with a harness — "in some sense, a harness is a bunch of code" that handles what current models can't. What we call an AI agent is usually the combination of one or more language models plus the harness.
• Internalization: distilling agentic capabilities out of the harness and into the model — memory, context management, tool use, planning — typically via supervised learning or RL during mid- and post-training.

The three components of agentic modeling (~00:04–00:08)

1. The agent / harness. Gao's lab does not build frontier models — too resource-intensive. Microsoft's dedicated MAI team builds in-house models (a dozen or so released recently), contracts with OpenAI and others supply frontier models, and Azure AI Foundry hosts hundreds to thousands of open-source models. The lab's focus is the enabling technology for harnesses, which are layers of hardcoded modules rather than models: - Information layer: memory and context management, plus the tools and skills the model can call. - Execution layer: prompt builder, action parser, decomposition modules, error-recovery mechanisms. - Feedback layer — the one he calls most important: it collects every trajectory the agents generate so those trajectories can refine the underlying model through post-training. This is the crux of agentic modeling.
2. The environment (an RL environment, or RLE): where agents interact with the external world, perform tasks, and emit trajectories.
3. The trainer: uses those trajectories to imbue agentic capability through post- and mid-training. The key methods are RL and distillation variants (on-policy distillation, reverse-KL algorithms). Cost efficiency matters on both ends — cheap serving, but above all sample-efficient training.

The new paradigm: harnesses are the new training data (~00:08)

Put together, this is a new paradigm of AI modeling in which the harness plays the role internet data played for pre-training. As the industry builds new agents and harnesses every single day, a data flywheel emerges: distilling agentic capability from harnesses yields models that are more capable of doing things; those models help us build better agents; better agents generate more sophisticated trajectory data by solving harder tasks; and the loop continues as harness and model advance together.

MSR's ongoing agentic research maps onto exactly those three boxes: green (modeling technology for harness engineering — memory management, test-time learning), blue (environment services hosting many RL environments in parallel, the source of training data), and yellow (the trainer).

Project 1: Orchard — an open ecosystem for agentic modeling environments (~00:11–00:15)

• The gap: open research on agentic modeling is constrained by infrastructure. In the early days of pre-training, an internet connection was all you needed for data. Agentic modeling is different — you need an environment that can run hundreds of thousands of agents simultaneously to produce trajectory data at scale. Community infrastructure so far has mostly gone into improving harnesses, not into environments that host many different agents in parallel.
• Orchard is an open-source ecosystem for scalable agentic modeling. At its core is Orchard Env, a Kubernetes-native environment service offering reusable primitives for sandbox lifecycle management.
• His analogy, improvised on stage: Orchard Env is an operating-system runtime for AI agents — an orchestration and virtualization layer managing container images inside sandboxes and exposing a clean interface. "It's like the organizers of this conference. The operating system in this context is the building — it provides all the facilities. If you view each workshop as an agent, the organizers did a great job putting each workshop in a particular room; that room is the RL environment, the pod. And afterwards the organizers collect all the results and share them back. Orchard Env is the organizer of this whole conference."
• Three demonstration recipes: Orchard-SWE (software engineering), Orchard-GUI (navigation), and Orchard-Claw (personal assistant).
• Result: Orchard-SWE distilled roughly 100k trajectories and reached 73% on SWE-bench Verified, beating many open-source models despite the model's small size.
• He said the paper, code, and data would be released "sometime next week" (published as arXiv 2605.15040).

Project 2: EvoLib — test-time learning with an evolving library (~00:15–00:16)

Memory matters, but we usually treat it as a store of raw experience — and raw experience isn't learning. "I collected a lot of experience previously; I have no idea how to use this experience to solve new tasks. But human learning is different. I never recall all the details of my previous experience. I only remember what matters — the failures I try to avoid, the new skills I learned once and hopefully can apply multiple times in the future." EvoLib is designed to simulate that: accumulate, reuse, and evolve knowledge across a sequence of tasks without touching parameters.

Project 3: GFlowRL — policy learning as distribution matching (~00:16–00:17)

Gao's hope is that this becomes a genuine alternative to the dominant RL methods (GRPO, PPO). The key difference: policy learning is defined as distribution matching rather than reward maximization. Concretely, the algorithm samples solutions in proportion to the reward they were assigned instead of collapsing onto the single highest-reward mode. That unlocks far greater solution diversity and, he argues, turns out to be more cost-effective to train.

He ran out of time here and closed by inviting questions offline.

Quotes

"The portrait of the large scale language model is mainly a talker. It seems that the model knows everything, but it's really difficult if you ask it to do something for you. So agentic modeling is the process where we want to turn this talker to a doer." (~00:02)

The premise of the whole talk.

"Harness is like the training data to agentic modeling, just like the internet data to model pre-training." (~00:08)

The load-bearing analogy: building a harness is manufacturing training data.

"Human learning is different. I never recall all the details of my previous experience. I only remember what matters." (~00:16)

The motivation for EvoLib: memory should be distilled skills and lessons, not a warehouse of transcripts.