Omnigent - A Meta Harness for AI Agents

TL;DR

• The problem isn't adoption, it's operations. Databricks has 3,000+ engineers on different AI tools and hit five walls: the space is crowded (harness × model is a combinatorial explosion, with new entrants weekly), fragmented (switching harnesses means rewriting agents, skills, tools, policies), personal (agents and memory live on one laptop, invisible to everyone else), hard to govern (no enterprise spend caps or tool access rules), and tethered (close the laptop, work stops).
• The answer is a new open layer above the harness — a meta harness — called Omnigent, fully open source. The architecture is deliberately simple: a server where sessions live (history, policies, MCP servers, skills, custom agents) and a runner where the agent works (laptop, dev box, Kubernetes pod, cloud sandbox), with one common API that every surface speaks.
• Three capabilities: composition (a dozen-plus harnesses with no migration; have one model implement and another review in the same session), collaboration (shareable, co-drivable sessions with Google-Docs-style comments on code, markdown, HTML, and PDFs, injected into the agent via "address all"), and control (policies that see the whole session and can allow / deny / require human approval, plus budget checkpoints and smart model routing).
• The security model is worth noting: all LLM calls happen on your laptop — the server makes none; it only orchestrates hosts. Omnibox provides a built-in OS-level sandbox, and credentials are injected at a proxy so agents never see real secrets.
• Community numbers: week seven since open-sourcing — ~8,000 stars, ~3,000 merged PRs, 360+ community members who have merged a PR, with roadmap discussions, PR reviews, and design docs all public on GitHub.

Discussion Topics

Why a layer above the harness (~01:31–01:35)

The opening argument: Claude Code is an excellent harness shipping features every week, but it isn't the only one, and building harnesses is easy — plenty of people have. The real problem shows up afterward: adopting agents is easy, operating them is not.

Databricks saw this firsthand across 3,000+ engineers using different AI tools. Five problems: the space is crowded (harnesses times models is an infinite combination, with new ones every week); it's fragmented (each harness locks you into its own configuration, so switching means rewriting custom agents, skills, tools, policies, and harness-specific config); it's personal (agents and memory live on your laptop; nobody else can see, steer, or contribute to your sessions); it's hard to govern (individuals can set local policies, but enterprise-level spend caps and access rules for the tools behind agents are difficult); and it's tethered (tied to your laptop — Aravind's image is walking between meeting rooms with your laptop half open and a finger in the middle to keep the session alive).

Their internal anecdotes make it concrete: sending CLAUDE.md files over Slack so a colleague's agent could pick up where yours left off; private worktrees where they committed pseudo-code just to collaborate; and reconfiguring every MCP server to talk to internal systems the moment they wanted to try Codex.

The conclusion: what's needed is a new open layer above the harness level — a meta harness.

Architecture (~01:35–01:36)

Two halves. The server is where sessions live: session history, policies, MCP servers, skills, and custom agents. The runner is where the agent actually does its work: a laptop, a dev box, a Kubernetes pod, or a cloud sandbox. Between them sits one common API that every surface speaks — terminal, web UI, native desktop app.

Because the server sits in the middle, a session is no longer trapped in one terminal on one machine: close the laptop, the agent keeps working, pick it back up from your phone. That one architectural choice buys composition, collaboration, and control.

Composition (~01:36–01:37)

Omnigent speaks with over a dozen harnesses out of the box — Claude Code, Codex, Cursor, OpenCode, Pi, and others. Three things matter. No migration: Omnigent picks up the setup already on your machine, so subscriptions, MCP servers, and skills come along ready to go. Orchestration across harnesses: if Claude Code has a capability tomorrow that Codex doesn't, you don't open a second terminal and copy context — you ask for it from the same session, and you can have one model implement while another reviews. Extensibility: a harness plugin SDK lets anyone ship an integration without touching the core package — Atlassian built their own harness, Rovo, and integrated it without touching Omnigent at all.

Custom agents are just YAML files: a system prompt, a model, some tools. The interesting part is that other agents can be added as tools, and those agents can be built on different harnesses. Two ship as examples: Polly, a multi-agent coding orchestrator that fans tasks out across harnesses in parallel, and Debbie, a debating agent that runs the same prompt across multiple harnesses so you can compare. Because agents are plain YAML they're portable — share with your team or publish server-wide, with the same policies and sandboxing applying throughout.

Control (~01:37–01:40)

Policies in most tools look at one action in isolation: is this command allowed, yes or no? In Omnigent, policies see the whole session — the data that has come in and the action being requested, together. Enforcement is granular: allow, deny, or ask for human approval. A library of built-ins ships with the tool, and policies are plain Python, so adding your own is easy — Red Hat engineers wrote policies to detect agents stuck in loops and stop them.

The same machinery watches cost. Budgets give you soft checkpoints: with a $100 limit, at $20 it prompts you to confirm you want to keep going. Dhruv's favorite detail is that this need not be a hard limit — it can be configured as a downgrade gate, so after $20 on frontier models you start routing to cheaper, smaller, open-source ones. There are per-session versions and per-user daily caps enforceable server-wide. Smart routing adds an auto option with a lightweight classifier that picks the harness and model for you — a frontier model for a big design task, something fast and cheap for a small fix. (Dhruv admits he's "completely guilty of picking the biggest model at the highest effort level for even the smallest task.")

Collaboration (~01:40–01:41)

The feature Dhruv personally uses most. Any session can be shared read-only or in full edit mode, and someone can co-drive with you.

The pattern they keep seeing mirrors on-call: someone spends an hour building context with an agent, hits a blocker, and needs a teammate. Instead of pasting history or a final response into Slack, you pull the teammate into the session with all the context already in place. You can also comment on the artifacts the agents produce — code, markdown, HTML, PDFs — almost like Google Docs editing but for agentic work. Once comments pile up, hitting "address all" injects them into a prompt for the agent.

Runtime and sandboxes (~01:41–01:42)

On the cloud side, Omnigent connects to sandbox providers through one interface: Modal, Daytona, E2B, plain Kubernetes, and a Databricks sandbox that runs the agent inside your Databricks workspace. Cloud sandboxes are what give the always-on story — kick off a session from your laptop, close it, check in from your phone. Providers are plugins too, so an existing dev box setup can be brought in without needing PR reviews from the Omnigent team.

For defense in depth there's Omnibox, their built-in OS sandbox, native to Linux, macOS, and Windows. Credentials are injected at a proxy, so agents never see real credentials or raw secrets. This applies inside cloud sandboxes as well, so even on Daytona or Modal there's an OS-level sandbox and the blast radius stays small.

Live demo highlights (~01:43–02:02)

The demo subject is a note-taking app repo "Claude made in 30 minutes," with GitHub issues filed against it (add dark mode, pin important notes).

Instead of running claude, you run omni claude: it starts a runner (the laptop), connects to the configured server, and opens the familiar Claude terminal plus an Omnigent URL. In the UI (desktop app or web) there's a host picker, a folder and worktree selector, and a harness picker covering Claude Code, OpenCode, Codex, Cursor, and custom agents like Polly and Debbie. The same session can be driven from three synced surfaces — the chat UI, the web terminal, and your actual local terminal — so you're still using Claude Code, just with a much better GUI over your sessions and projects.

omni setup configures harnesses (four in the demo: Claude, Codex, Cursor, OpenCode). Polly was then pointed at the repo's GitHub issues and told to use Claude, Codex, OpenCode, and Cursor sub-agents. The result: Claude SDK as the top-level driver orchestrating five sub-agents, each with its own inspectable terminal — Codex on the edit-notes PR, Cursor on live search, OpenCode running Kimi via OpenRouter.

The embedded browser opened the app on localhost, showed dark and light mode, and let him comment directly on the UI ("I don't like this, let's change it to a toggle"), which the agent picked up. He then shared the session with Dhruv with edit and approve access; Dhruv opened it under "shared with me" on his own machine, browsed the changed files, left comments, and hit "address all" — sending a message to the agent running live on Aravind's laptop. No GitHub, no Slack.

Smart routing with auto: "What is the capital of France?" was classified as simple and routed to GPT-5 Nano; "explain this complete codebase and give me a diagram" was classified as complex and routed to Opus. For policies, he added a 10-cent session cost budget mid-session and the next call was denied by policy once the budget was hit; the UI also offers tool-call limits per session, blocking specific skills, enforcing a sandbox, and denying PII, settable at session or server level.

The automations panel schedules recurring tasks on any host or sandbox — a prompt, a frequency, and a host (or whichever host is online). Theirs include a morning PR sweep that reviews PRs and leaves comments, and a company news digest. Finally, the Databricks sandbox: it takes a GitHub repo, comes preconfigured with GitHub credentials, and runs entirely off the laptop, so you can close the lid and continue with the same agent from your phone.

Q&A highlights (~02:02–02:12)

Company-wide policy enforcement? Yes — an admin configuring the server can set server-wide policies that apply to every user and every session, on any host connected to that server.

Self-improvement and harness optimization? They're learning from Pi's approach to extensibility — harnesses, sandboxes, and smart routing should all accept your own plugins. He's candid that they're not at Pi's level of extensibility yet, but that's the gold star.

Scale inside Databricks? 3,000+ users, everyone uses it daily, thousands of sessions per day. "As soon as it goes down, people are pinging us faster than PagerDuty."

Brains/hands separation? They run the harness on the same host or sandbox as the commands, rather than splitting the LLM loop onto a server. Safety comes from Omnibox's OS-level protections plus a tool hook on every harness that runs your policies before the harness executes. No harness-level optimizations yet.

Context and compaction over long-running sessions? Compaction is left entirely to each harness's native mechanism — a Claude session compacts with Claude's algorithm, Codex with its own (and Codex compacts more frequently). Since Polly is built on Claude in the demo, the top-level agent compacts with Claude's logic, but you can swap Polly's main harness to Codex. Omnigent adds no compaction or clearing of its own. The server does store conversations, so if an agent dies they rehydrate the session from their database. The questioner pressed on the real risk — repeated compaction causes context rot, and you may not be able to tell that the code produced has issues, which sits outside the harness's control. Both sides agreed to continue offline.

Data privacy boundaries? The clearest answer of the session: all model calls happen on your local laptop; the server makes no LLM calls. Credentials are configured locally via omni setup. The server's job is to orchestrate hosts — the laptop opens a WebSocket to the server, and the server knows which pod it's in and which laptop to dial. Where the server itself runs is up to you: any multi-tenant architecture or your own infrastructure.

Can policies restrict which models a harness may use? Yes.

Custom agents with deterministic logic, or agents calling agents? Custom agents are YAML, exactly as Polly is; wrap deterministic workflows as tools like you would with any other agent. There's also a REST API for Omnigent you can call from your own deterministic workloads.

Quotes

"Adopting agents is pretty easy, but operating them is not." (~01:32)

The problem statement for the whole workshop.

"Each harness locks you in into its own code, and switching harnesses means rewriting your custom agents, your skills, your tools, your policies." (~01:32)

The concrete cost of fragmentation.

"Policies in most tools look at one action in isolation … In Omnigent, policies see the whole session." (~01:38)

Moving governance from a single action to the whole session is their most substantive design difference.

"The server does not make any LLM calls. It's all on laptop." (~02:10)

The answer to the question enterprises care about most.