Unified Orchestration System for Verifier-Free Evolution

TL;DR

• The problem is that verifiers are expensive. Existing discovery frameworks generally require an external verifier, but in domains like the physical sciences evaluation is both costly and time-consuming. Their question: what is the upper bound of an evolution system with no verification in the loop — and can it also be cheaper?
• Single-model evolution degrades itself. Running evolution with one model often makes performance worse, because a single model carries only so much entropy and creativity. The fix is to make the evolution loop multi-model, using model heterogeneity itself as the source of diversity.
• Routing by difficulty buys diversity and cost efficiency simultaneously. Using token log-probabilities or solution diversity as the fitness function, hard solutions go to expensive models and solutions easy enough to evaluate and combine go to a cheap one. Result: on ARC-AGI, 10 evolution steps with a strong model reach 97.5% at roughly $7/task; mixing in a cheap model retains the same accuracy in just 2 evolution steps at about $5.9/task.

Key Points

Four components of the discovery loop, and the verifier cost problem (~00:41:53–00:44:20)

Ben Athiwaratkun presented Squeeze Evolve, a unified multi-model orchestration system for verifier-free evolution, aimed at scientific discovery.

He opened by decomposing the discovery loop — a decomposition taken from SkyDiscover — into four distinct components:

1. Context builder — supplies problem context and injects ideas, guidance, and reflections on previous mistakes and successes into the prompt.
2. Solution generator — an LLM with access to environments such as code execution or web lookup.
3. Evaluator — provides scores, logs, feedback, and any necessary artifacts.
4. Solution selector — picks sample solutions from the candidates and their metadata to hand back to the context builder for the next loop.

The catch: prior discovery frameworks generally require an external evaluator or verifier, and in fields like the physical sciences evaluators are costly and time-consuming. Hence the research question: what is the upper bound of an evolution system without verification in the loop, and can cost come down too?

Why multiple models: the degradation problem (~00:44:20–00:46:00)

The other characteristic failure of evolution systems is degradation of solutions (~00:44:31). His summary was blunt: using a single model to run evolution often degrades performance, because there is only so much entropy and only so much creativeness in a single model. Their answer: use multiple models to attack the diversity problem directly.

Mechanically, multiple solutions and generations are sampled from previous and current generations and assigned probabilistic fitness scores. The fitness function uses token log-probabilities or the diversity of the solutions themselves (~00:45:12).

The routing rule is simple:

• Difficult solutions → expensive models, to raise diversity.
• Solutions judged easy enough to evaluate and combine → a cheaper model (~00:45:47).

That single split increases diversity and cost effectiveness at the same time, which is the design idea at the heart of the method.

Results: fewer steps, same accuracy, lower cost (~00:46:00–00:47:58)

The headline numbers (~00:46:11): the full pipeline with 10 evolution steps using a strong model (heard in the captions as Gemini 3.1 Pro) reaches 97.5% accuracy on ARC-AGI at roughly $7 per task. Mixing in a cheap model (heard as Gemini 3.0 Flash), they retain the same accuracy with only 2 evolution steps at about $5.9 per task.

A second observation concerns vision tasks: not using the vision component for the aggregator stage significantly improves cost effectiveness. The red curve on his chart shows a strong multimodal model used throughout — performance climbs high, but cumulative dollars per problem climb proportionally. Under Squeeze Evolve, combining a powerful model with a cheaper tier cuts cost substantially, and in some heterogeneous configurations they outperform the single-model scenario outright.

He closed with two adoption notes: the work has been integrated into NVIDIA Dynamo (with a shout-out to NVIDIA), and a Claude Code plugin is available.

Quotes

"If we use a single model to perform evolution, it is often the case degrading the performance, because there's only so much entropy and only so much creativeness in a single model." (~00:44:35)

Diversity isn't something you tune into a single model; you buy it with heterogeneity.

"In the scenario where we use a cheap model in the mix … we're able to retain the same accuracy with only two evolution steps." (~00:46:25)

Mixing in the cheap model didn't just cut cost — it cut the number of evolution steps needed.

"In the heterogeneous case we're able to outperform the single model scenario." (~00:47:25)

The heterogeneous mix isn't only cheaper; sometimes it beats the strongest single model.