Are Small LLMs Ready for Coding Agents?

TL;DR

• A personal itch: on a transatlantic flight he realized how unproductive an AI scientist is without a coding agent while offline, so he built his own local coding agent to find out whether a local LLM on a MacBook can handle realistic agentic coding tasks. It's now open source.
• Three design principles: (1) minimize the context handed to a small LLM, because small local models degrade badly with long context; (2) take one very small action at a time and verify it immediately; (3) minimize reasoning, using it only in recovery loops when something goes wrong.
• The finding: small models do most of the work, but fail at verifying their own work and closing the observe-then-fix loop. Building an app feature, the agent passes most tests but not all — the same story across Qwen 3.6 and Gemma 4.
• Positioning: far lighter than the Codex or Claude Code code bases. He compared against the most minimal harness people use today and against OpenHands, and his is smaller than both, aimed squarely at very small local models. Still work in progress.

Key Points

The origin: a few offline hours over the Atlantic (~00:37–00:38)

A personal story. A few months ago on a transatlantic flight he realized that as an AI scientist he becomes deeply inefficient without a coding agent when disconnected from the internet. So he set out to build his own local coding agent and see whether a local LLM running on his MacBook could handle realistic agentic coding work. He open-sourced it recently; the repo also carries the slides for this talk.

Three design principles: small steps, immediate verification, minimal reasoning (~00:38–00:40)

The agent "does one small thing at a time," built around three ideas:

1. Minimize the context passed to the small LLM — small local models are not good when you hand them a lot of context.
2. Take very small actions and verify each one immediately.
3. Minimize reasoning, invoking it only in recovery loops when things go wrong.

His worked example: give a small model (Qwen 3.6 here) a hello-world-level task — write a small program, compile it, run it. It failed, because it compiled into a temporary directory but needed to execute from a different one. Small models struggle to notice that class of problem, which is exactly why changes must be tiny and verified one at a time.

The harness itself is deliberately spare: a minimal loop in a single file, once around 1,000 lines and now closer to 2,000. The strategy is to avoid big planning steps, take smaller ones, and only backtrack when something breaks.

Evaluation: it can write code, it can't check itself (~00:41–00:42)

He ran a few hundred tests:

• Smoke tests across two model families, Gemma 4 and Qwen: everything works except one Qwen model. This was a sanity check on usability.
• Realistic tasks: building a feature for an app. The agents do most of the work, but the problem is that they fail to verify their work and lack the full observe-what-happened-then-go-back-and-fix loop. The result: most tests pass, but not all. Same story across Qwen 3.6 and Gemma 4.
• An honest caveat: for the record, these runs were on OpenRouter, not locally — running them locally would take ages, and his MacBook has limited memory.

Positioning and comparisons (~00:42–00:43)

His conclusion: a small agent running locally with a minimal harness shows you don't need a large code base like Codex or Claude Code to work with a very small LLM. The comparisons he called out:

• The most minimal harness people currently use (transcribed as "PI harness"), which he notes is much more efficient than Codex or Claude Code.
• OpenHands.
• His is smaller than both, targeted at very small local models.

He was explicit that it's still work in progress with plenty of open issues, and asked the room for feedback.