The Open Agentic Stack: Building the Future of AI Systems with Open Source, Open Standards and Composability

Opening: Matt White — why openness matters more in the agent era (~02:58:30–03:04:45)

Former Global CTO of AI at the Linux Foundation, previously executive director and CTO of the PyTorch Foundation, now a visiting scholar at Columbia. No slides — "it's just me rambling" — but this is the frame for everything after.

One: capability is surging exactly as openness comes under real pressure. Models reason across long contexts, write and execute code, use tools and operate software, coordinate increasingly complex workflows, solve complex mathematical proofs, generate extensive codebases, and power systems that take actions in production networks with real-world impact. And at precisely this moment, restricting or even banning access to open-weight models is no longer hypothetical — it's being actively debated in Washington and elsewhere, alongside proposals to control the frontier through chips, compute, distillation, safety testing, and model release processes.

He concedes the point directly: the safety, security, and national security concerns behind those discussions are real and legitimate, and deserve serious technically informed responses. But the risk on the other side must be stated with equal clarity: blunt restrictions concentrate capability, infrastructure, and decision-making into very few institutions, turning AI from a broadly accessible general-purpose technology into something the rest of society is only permitted to rent from a few labs. "That cannot be the foundation on which we build the future."

Two: openness is both how you get competition and part of how you get safety. It's how researchers inspect systems, developers adapt them, students learn, enterprises keep sovereignty over their data and stack, and countries build AI capability without permanent dependence on a single vendor. It enables reproducible evaluation, transparent interfaces, independent scrutiny, shared security tooling, and the ability to find and fix problems. But openness does not mean ungoverned — not careless, unsafe, or a free-for-all. It means systems can be inspected, evaluated, adapted, and improved; interfaces are interoperable, implementations contestable, and no single organization controls every layer or decides who may participate.

Three: agents raise the stakes because an agent is a stack. Model, harness, invocable tools, environment, guardrails, the RL systems through which it improves, the evaluation infrastructure that measures it, and the compute underneath. It may depend on APIs, credentials, sandboxes, memory, inference systems, orchestration frameworks, and multiple forms of specialized hardware — crossing organizational, cloud, and geographic boundaries while completing a single task. Close and vertically control any one layer and the ecosystem becomes less portable, less competitive, and more fragile.

So open models alone are not enough. We also need open environments so agentic tasks can be reproduced and shared, open RL frameworks so more people can improve models and agents, open compute layers so workloads aren't captive to one cloud or vendor, and open standards so the components interoperate without permission from a central platform.

The open agentic stack is not one project, framework, or model — it's a shared architecture built from open source software, open standards, and composable components. The workshop walks four foundational layers: environments, models, reinforcement learning, compute. Different technical challenges, one shared principle: the future of AI should be modular rather than monolithic, interoperable rather than captive, and open to participation rather than controlled by permission — an architectural, economic, and strategic requirement, not just a philosophical preference.

Talk 1: Ben Burtenshaw (Hugging Face) — Open Source Agentic RL Environments (~03:05:45–03:18:20)

Thesis: RL environments are the best way to democratize AI, mainly because they're easy to build — they're just applications, the kind of thing we write daily — so everyone can get involved.

What an RL environment is. The simplest framing: a world an agent acts within. Chess is the canonical case — the board is the world, the pieces are state, the rules are the actions, the score is the reward. Scale that up to software engineering: the task is a GitHub issue, the work is the pull request, the score is the test suite and CI. That's how papers like SWE-smith built software engineering environments. Strip the code out of the PR, have an agent generate it, use the test suite for reward, and you have an environment to train a software engineering agent. Carry that archetype to email triage, project management, and so on — find the task, the reward, and the state, and build them into applications.

The ecosystem: a capability cycle. He updates the old MATTER cycle (model, annotate, train, test, evaluate, revise) into discover → benchmark → represent → train. You elicit a capability with a harness or prompt and see the model can do a thing, maybe unreliably. You evaluate it with a separate evaluation harness that measures how reliable it is, and iterate on that as a community. You represent it, first as a dataset and eventually as an environment — this is where domain expertise enters; companies like Mercor go hire domain experts and build RL environments around a capability, partially based on the benchmark but ideally with no leakage between the two. Then the trainers come in — another software stack, another community.

The upshot: environments are the means by which capabilities move from domain experts into models. Once a capability is in the weights, everyone moves to the next one. Because so many communities meet at the environment, that layer has to be open — "imagine if datasets were in a closed format; it would be a particularly horrible situation."

The current problem is fragmentation. Environments live in papers and on GitHub. The recent Kimi K3 paper came with thousands of environments; the MiniMax paper had around ten thousand; even the first DeepSeek paper had roughly a thousand. All implemented differently, shared differently, and many of them sold rather than readily available the way open-weight models are. It reminds him of being an MLE in 2017: find the implementation on GitHub, then chase the weights down on Google Drive — nothing exposed and interoperable the way model checkpoints are today.

The problem compounds at training time. A training process is a framework (TRL, Unsloth) updating a model based on its actions, but in between you need an agent harness to perform the task, interact with the environment, and use its tools. Every edge there is a layer that needs an interface — a harness can't hardcode an expected environment definition, and you need to work with multiple harnesses (he names Codex, Claude Code, and others).

What OpenEnv does about it.

• Environment hubs: about 4,000 environments on Hugging Face today, in a standard format you can plug into most major training frameworks. Others exist (Prime Intellect) with more coming; they're sharing their spec with other orgs so anyone building a hub can use the same format for downloading and sharing.
• A Docker-like CLI: openenv init scaffolds a hello-world environment that just returns a generic score; from there you write the business logic yourself, or generate it with a coding agent. You can import environments from libraries like verifiers, and push, pull, and fork on the hub.
• Coming next: discover and validate. Finding an environment for your use case is hard today; hub search would give you a range of community environments — which creates the next problem: which one is better? which will actually help my use case? Hence a new validate command that checks the spec is valid and trains a small model to see whether it lifts performance on a given benchmark. Together those two commands give an autonomous loop for integrating environments into post-training.

Three ways to use an environment, easiest first:

1. Evaluation — what he suggests most people start with. Represent a typical workload as an environment, run a model (weights or just an API) over a set of tasks with a reward function, get an aggregate score.
2. Reinforcement learning — the core use. Roll a policy out over the environment, compute rewards per rollout, use something like GRPO to take the group mean and update weights, iterate.
3. Distillation — here you can't distill from strings or tokens; you need log probabilities. Roll out the student in the environment, take its log probs; have the teacher roll out over the same tokens, take its log probs; update the student on the difference (reverse KL). Variants: self-distillation (best student example versus the average), or giving the student privileged information or a hint — in the PR setup, hand the agent the actual solution up front and update weights on the difference between its bad run and its good run.

How he expects this to work: leaderboards. Environments currently live in "a bit of a shadow realm," rarely discussed in the community around model releases. If people build and try their own environments, put them on leaderboards, and figure out which ones actually contribute to model success, more people will get involved.

Talk 2: Daniel Han Chen (Unsloth) — Making Open Models Not Suck (~03:19:38–03:35:35)

Who Unsloth is. One of the largest organizations on Hugging Face, roughly 400 million cumulative downloads (a number he says is already outdated), the third largest model distributor in the world, shipping both the largest and smallest models. But they don't just distribute — they fix bugs in open source models, collaborating with the major labs and hardware providers to fix issues before the model reaches you (he cites OpenAI's gpt-oss, Llama, Gemma, Mistral). "If you're using open source models on your laptop, they were most likely fixed by us in some way."

Quantization right now. Kimi K3 shipped Monday — the best open-weight model anyone can download, 1.5 TB as released. They quantized it to 1 bit, bringing it to about 600 GB. The hard part: Kimi K3 is natively 4-bit, so 4-bit → 1-bit doesn't save as much. Their result: 76–78% of accuracy retained for an 82% size reduction — shrinking a model by 82% doesn't make it 82% dumber, only about 16%. Separately, DeepSeek V4 Flash dropped the previous day (0731) at an odd hour in an odd time zone; they quantized it too, and it now needs 90 GB of VRAM or RAM. His verdict: the best model for its size — Kimi K3 is much better but enormous, while V4 Flash actually fits on your machine.

The METR curve and the pre-reasoning plateau. Models now complete tasks that take humans 16 hours (at 50% probability; about 6 hours at 80%). He plotted Mythos / Fable as a red line and GPT-5.6 Sol — the latter with cheating examples removed. Including cheating cases, GPT-5.6 Sol reaches 270 hours, which he says isn't a good example; removing them puts it around 16.

His central observation is the flat stretch in the curve: roughly 2023–2024 was an approximate plateau (GPT-4 to GPT-4o showed little progress), because reinforcement learning and reasoning hadn't been invented yet. Fit two separate trends: - Gray line (pre-reasoning): capability doubling every 7 months. Had OpenAI never shipped o1-preview and told everyone reasoning was possible, models would have plateaued by now. - Green line (with RL and reasoning): doubling time shrinks to 3.5 months. "Opus 5 came out a few days ago — wait 3.5 months and you'll get something better."

The caveat he stresses is that this holds only if the trend continues. Will the green line keep climbing, or do we fall back to the gray one and have to invent something new again? "That's a question we'll see answered in our lifetimes — all you need to do is wait 3.5 to 7 months."

Have open models caught up? He cites Artificial Analysis' open-versus-closed curves: open models are now very close. But his favorite single benchmark is WeirdML (search "weird ML"), because it's genuinely hard to game. By WeirdML: Fable 5 is very good and ginormously expensive; for cost/accuracy balance he'd recommend Opus 5 High; among open models Kimi K3 is extremely powerful and on the leaderboard.

He then plots closed minus open as a gap curve. About a year and a half ago there's a stretch he calls the open source drought — the period after o1-preview when open models fell behind. Once DeepSeek published how to do GRPO with R1, open models snapped back to trend. His estimate: without DeepSeek publishing GRPO, open models would be 16 months behind closed; now it's about 2 months. A blog post extrapolates that by December 2026 there'll be an open model as good as a closed one — all trend extrapolation, so "who knows; wait a few months and we shall see."

Are collapsing token prices reasonable? OpenAI cut GPT Luna prices by 80% a few days ago. Shocking-sounding, but he thinks not — and he ran the numbers before the talk:

• Per SemiAnalysis' InferenceX benchmarks of vLLM / SGLang and other inference engines on NVIDIA and AMD GPUs: DeepSeek V4 Pro does about 13k tokens/s on a single GB300 → 47 million tokens per hour at saturation.
• A GB300 costs roughly $5 per GPU-hour.
• Divide: DeepSeek V4 Pro's floor price is about 11 cents per million tokens. DeepSeek charges 44 cents per million input and 87 cents per million output — so nobody is shedding money running these models.
• By the same logic, assuming GPT-5.6 Luna is the same size as DeepSeek V4 (he's explicit it's most likely not), OpenAI still clears roughly $1 per million output at the new price.
• The contrast: Kimi K3 is a new, not-yet-optimized architecture at about 5k tokens/s per GPU (~2.6× slower than DeepSeek), giving a floor of about 30 cents per million tokens — while it charges $15 per million output. That means Moonshot and other inference providers are taking about $14.17 of your API bill. "If they wanted to reduce price, they could go all the way to 30 cents." Conclusion: prices are still somewhat inflated, and DeepSeek is the only one charging near floor.

The other option: run locally. He likes Arena's Pareto efficiency plot (less so the Arena score itself): many open source models sit on the Pareto frontier. For web development (HTML/front-end), Opus 5 Max is currently best with Kimi K3 just behind; GLM 5.2 and others are on the plot too. If you don't want to pay API costs, run locally and the cost drops to electricity plus some front-loaded setup.

Which is Unsloth's business: dynamic quantization down to very small bit widths. GLM 5.2 at 1 bit is his showcase — a one-shot prompt, no bug-fixing iterations, holding up well against full-precision closed models. They publish perplexity and KL divergence benchmarks to show 1-bit works. The goal is to push further: Kimi at 600 GB still probably doesn't fit on your computer, so they're aiming below 1 bit — Kimi at 250 GB, for example.

Last point: the ARC-AGI harness settings. A few days ago OpenAI noted the ARC-AGI benchmark is somewhat unfair to GPT, because the eval used a rolling truncation of 175k tokens with compaction disabled and preserve thinking disabled. His blunt takeaway: just enable those two flags and OpenAI's models climb to 40% accuracy. You don't need to do anything else. (He then ran out of time and skipped the remaining slides.)

Talk 3: Shang Yang (RadixArk) — Miles: An Intro to Enterprise Facing RL with Miles (~03:36:48–03:45:35)

The speaker is a research resident at RadixArk and a PhD student at MIT, presenting Miles on behalf of the team.

What Miles is: an open source RL engine for frontier models, production-facing, built for speed and stability — "a stable, efficient, and reproducible reinforcement learning system at scale," minimizing the effort to run an RL job on modern language models across different environments and reward designs.

Three layers:

1. Rollout — SGLang as the rollout engine, specialized for high-throughput generation and highly compatible with today's agentic systems, environments, and infra.
2. Orchestration (middle) — organizes the rollout engine and the backend training systems as a whole so the system runs efficiently.
3. Training — supports Megatron and FSDP, and you can customize your own training engine on your own compute.

Data flow: the rollout engine generates trajectories by interacting with an external environment or agent framework (SGLang provides the generation layer; the environment itself can be heavily customized per customer requirements) → the middle layer turns trajectories into rewards and loss functions, i.e. training signals → Megatron or FSDP runs forward/backward, computes loss, updates weights → weights sync back to the inference engines, and the loop closes.

Why SGLang as the rollout backend: - Co-evolved with RL workloads, including partial rollout support and dedicated rollout control, so specifying your workload is easy. - High-performance router design — well-designed caching and load balancing for high-throughput generation. - Native asynchronous generation with the training engine — an important feature for modern RL algorithms. - Fast-moving, with optimizations such as speculative decoding with Miles, usable during the RL rollout stage.

How Miles changes the RL development workflow: - Customizable — different workloads via pluggable function passes; defining your own RL workload, reward function, and update method is easy. - Modular and non-invasive — switching training backends (FSDP, Megatron) is easy, with more backends coming.

System design details: - Multi-agentic rollout, with token-in-token-out (TITO) support so the same multi-turn generation isn't tokenized and detokenized repeatedly — saving cost and making inference more stable. - R3 (routing replay): when training MoE models with RL, the same request can be routed to different experts across runs. R3 makes this fully reproducible. - Low precision: end-to-end MXFP8 support plus per-token NVFP4 quantization for the rollout stage, greatly reducing inference latency and increasing throughput — all verified in their latest blog against real training workloads. - LoRA support for resource-constrained settings, training large models on fewer GPUs.

Support matrix and track record: - Day-zero support for the newest models: Kimi K3, Thinking Machines Lab's Ling (captions render "inkling" — to verify), DeepSeek V4, Nemotron Ultra, and others. - Wide precision and hardware coverage; fine-tuning recipes beyond SFT and RL, including on-policy distillation. - Verified on RL training workloads across more than 10,000 GPUs. For Kimi K2.6, generation throughput was sped up to 12.5k output tokens per minute per GPU. - Stability work: R3, TITO, and precision-aligned training to reduce training–inference mismatch; plus fault-tolerant recovery and cluster backends including Ray on Kubernetes, Slurm, and bare metal.

Recent releases: - Day-zero support for Kimi K3 (nearly 3 trillion parameters), with fine-tuning on math problems lifting accuracy from around 40% to 76%. - Ling (Thinking Machines Lab, trillion-parameter class): both full-parameter and LoRA fine-tuning, with LoRA adapter synchronization optimized (merging the LoRA rank into the SGLang inference engine) from nearly 50 seconds to under 3 seconds. - A recent mixed-precision training blog verifying that MXFP8 or NVFP4 maintain model accuracy and training stability while substantially accelerating rollout and inference.

Talk 4: Romil Bhardwaj (SkyPilot) — AI Needs an Open Compute Layer (~03:46:13–03:55:58)

The case: AI needs an open compute layer — specifically, something that sits between your workloads and the actual GPUs they run on.

Why AI infra is hard, told through OpenAI: an exclusive Azure GPU agreement in 2016; a $10 billion Oracle deal in 2024 for more compute; another $11 billion with CoreWeave just a year later. And it still isn't enough — Sam Altman is on record asking for more GPUs and telling people to call if they can secure any. His conclusion: if the most capitalized company on the planet can't secure enough GPUs, or get them in a single cluster, you probably can't either. The GPU crunch is very real.

What a typical AI organization actually has: some NeoCloud reservation (a two- or three-year deal signed when the hyperscalers had no capacity); some hyperscaler on-demand instances to supplement it; some on-prem clusters for regulatory reasons or to keep data in-house; another cluster in Europe because there are customers in Europe; and a large CPU cluster for running environments and sandboxes. Each was a sensible decision at the time; the result is a large collection of clusters you're now responsible for managing.

RL makes it worse — it turns this into a scheduling problem. An RL run is one job composed of subcomponents with very different resource profiles: the trainer needs lots of VRAM and beefy GPUs; the rollout server generating trajectories needs less VRAM but a lot of instances; and for something like code generation, the sandboxes need a ton of parallel CPUs with very fast startup.

Why not use existing solutions? - Slurm: two decades old, from HPC, and in his view among the best advanced scheduling and quota/priority systems out there. But it doesn't support container isolation (you can bolt on pyroot and similar layers, but it wasn't designed for it), and it was never built for serving — it's for batch jobs. - Kubernetes: about ten years old, popular for microservices, very elastic and cloud-native. But a super steep learning curve ("hand Kubernetes to a researcher and they'll just go ask their agent to make it work") and it lacks AI-specific primitives, notably gang scheduling, again bolted on via schedulers like Volcano.

More importantly, both are single-cluster managers. Neither solves "I have 10 different clusters running — how do I manage them?" What's missing isn't a better scheduler; it's something to manage all the schedulers and platforms you already have.

The goal isn't to replace Kubernetes or Slurm — they're good at what they do — but to bridge the gap between workloads (training, serving, sandboxes) and compute fragmented across Kubernetes, Slurm, and clouds. An open unified compute layer should optimize availability, figure out the best place to run a job, handle execution, and do it without the workload needing to know what's under the hood.

That's SkyPilot: fully open source, use AI compute wherever you have it, bring your own framework — Ray, PyTorch, OpenEnv, Unsloth, your own. You submit "I need eight B300s to run this" and SkyPilot orchestrates: it looks at your Slurm clusters, Kubernetes clusters, and cloud VMs, provisions, runs the job, and returns logs and results. The project started a few buildings away in Soda Hall, rooted in Sky Computing Lab research, with an active community and adoption at some of the largest companies.

What it looks like in practice: - The simplest path is sky launch: state the infra and the GPUs you need. Switching from Kubernetes to AWS is one flag, and so is switching GPU type — your application doesn't need to know what's running underneath. - For complex work there's a declarative abstraction called job groups: "a rollout server with these resources running these commands; spin up a thousand sandboxes; put the rollout server on H100s and the trainer on B200s" — then SkyPilot orchestrates the whole workload, with service discovery and lifecycle management included. One job failing doesn't fail the run — SkyPilot restarts just that job.

Serving benefits too: you gain capacity across all your clusters instead of launching one KServe instance per cluster, and when a failure hits, SkyPilot seamlessly moves replicas to a region where you still have capacity. Routing all workloads through one layer also enables something interesting: running inference and training on the same compute. When inference requests spike, SkyPilot dynamically reallocates from training to inference — preempting training jobs while your checkpointing logic preserves progress — then scales inference back down after the spike and resumes training.

Bring-your-own, emphatically: you don't hand SkyPilot a credit card to buy compute. You bring your own compute from your own provider and your own data; everything runs on your own premises. Closing echo of the opening: we don't need to replace Kubernetes or Slurm — we need a unified layer bridging workloads and all those clusters.