Beyond Harnesses – Platform Solutions for Agent Reliability, Security, and Efficiency

TL;DR

• A third platform wave: the first wave separated hardware from software and gave us high-level languages; the second was cloud computing and distributed applications across data centers. Each produced an operating system — a layer of abstraction separating applications from infrastructure, plus primitives applications can rely on for resiliency, scalability, and efficiency. AI applications are the most non-deterministic and unreliable component ever introduced into enterprise architectures, and will need a third such foundation.
• Two families of challenges: structural — agents have an open-ended instruction set chosen and ordered at run time, which breaks zero-trust security (it depends on knowing interaction patterns at configuration time), traditional recovery (compensation and rollback become intractable), and pre-deployment testability; plus instructions and data aren't separated in context, and agents communicate by exchanging context, so execution isolation and blast-radius control are lost. Semantic — agent goals can't be fully specified ("should do" is hard, "shouldn't do" is harder), there are no formal methods for expressing or validating them, and models are poor at reporting their own status: there are no reliable error codes, so success, failure, and liveliness are all hard to determine — and what you can't diagnose you can't recover from.
• The approach: don't replace the platform. Add an interception layer on top of existing cloud infrastructure that integrates with all three styles of agent (self-built, well-known harnesses controllable via hooks/plugins, and opaque black boxes) and offers a uniform way to observe and modify every interaction agents have with the outside world. Security — specifically zero trust — came first; semantic-layer control follows. The project is Rossoctl.
• The right application pattern is serverless: separate the agent loop as a stateless component from durable session storage holding context and from an execution tier of diverse sandboxes — better resiliency, accuracy, performance, and scalability, with measured cost savings.

Key Points

Why a third operating system (~00:33:26–00:34:50)

Steinder frames three waves of application platform evolution: hardware/software separation and the first high-level languages; cloud computing and globally distributed applications; and now AI. Each of the first two produced an operating system — Unix and POSIX semantics for the first, Kubernetes and its ecosystem for the second — where "operating system" means both a layer of abstraction between application and infrastructure and a set of primitives applications rely on for resiliency, scalability, and efficiency.

AI applications can certainly run on the existing foundation. The problem is that the existing foundation does almost nothing to help with their novel challenges.

Structural challenges: the open-ended instruction set (~00:34:52–00:36:10)

"Instruction set" here means the operations an agent can execute against the external world. That set is open-ended, and agents decide which instructions to use and in what order at run time — which breaks several assumptions existing platforms make:

• Zero-trust security depends on understanding interaction patterns between applications a priori, at configuration time.
• Recovery techniques like compensation and rollback become intractable to implement.
• These applications are not fully testable pre-deployment.

Two more structural problems compound this. Instructions and data are not separated in the context agents work off — which is why so many novel security threats are showing up around agents. And agents communicate by exchanging context, so execution separation between agents is lost, making the blast radius of resiliency and security issues very hard to contain.

Semantic challenges: no reliable error codes (~00:36:15–00:37:30)

Here Steinder explicitly picks up the thread from Ion Stoica's preceding talk. Agent goals cannot be fully specified: expressing what an agent should do in narrative form is hard, specifying what it shouldn't do is harder still, and there are no formal methods for expressing or validating those goals.

Worse, the models are bad at reporting their own status — there are essentially no reliable error codes. So determining whether an agent is working correctly, whether it is succeeding, detecting its failures, and assessing liveliness (is it making progress and converging, or hanging?) are all hard. And if you can't determine what the problem is, you can't recover from it.

There is also a semantic gap between how an agent thinks and plans versus the schema-based, constantly changing nature of real-world interfaces — a steady source of new errors.

How it's solved today, and why it will consolidate (~00:37:37–00:39:05)

Today all of this is handled bespoke, in the application layer. The industry has produced a lot of frameworks and harnesses; every few months a new powerful agent or harness arrives as a proof of existence that a solution is possible. But that creates heavy fragmentation, and harnesses are genuinely hard to build — they demand a lot of expertise.

Her read: we're clearly still in the experimentation phase, and every prior wave looked the same. Multiple versions of Unix at the start; many container runtimes, many orchestration platforms, and divergent ecosystems at the start of the cloud-native era. Eventually the industry identified common patterns, standardized, and consolidated — onto Unix and POSIX, then onto Kubernetes. She expects the same in the AI era.

Their approach: an interception layer over three kinds of agents (~00:39:08–00:41:30)

They started from the agents actually running in their own organization, which fall into three buckets: agents they implemented themselves on some framework or SDK (full control); agents built on well-known harnesses (controllable via hooks and plugins); and black-box agents (controllable only by observing from outside).

So they are building a layer of interception that integrates with all three styles and provides a uniform way to observe and modify every interaction those agents have with the external world, with control functions layered on top of that abstraction.

Security came first — specifically zero trust, implemented as a multi-tier, multi-layer permission system: establish identity; use that identity for delegation flows with authorization; then policy-based access; and finally intent-based access, evaluating whether what the agent is doing actually aligns with the user's objectives.

They have since moved into the semantic layer, asking whether — in a business-logic-independent way, outside the agent — they can control and manage the context agents use, manage the correctness of tool calls, and do data-flow analysis to understand and control how data moves. This is the Rossoctl project; their web pages carry deeper benchmark and experiment results, and a colleague (Maya) presented a related poster at the summit.

Crucially, they are not replacing the existing platform. Everything is built on existing cloud infrastructure, leveraging and extending existing standards: OAuth 2, SPIFFE for identity, known policy languages, orchestration on Kubernetes. The gateway-based approach was originally built on Envoy proxy and is moving to a more efficient Rust-based proxy from the Praxis project.

Results so far: cost reduction from context compaction, even with state-of-the-art agents; consistent improvements in tool-calling accuracy that translate into agent quality; and transparent permissioning for agents.

The application pattern: serverless (~00:42:21–00:43:30)

The other half of platform evolution is application patterns, and Steinder argues the right pattern for agents is serverless: the agent loop as a stateless component, separated from durable session storage where context is managed, and from an execution tier provided by a diverse set of sandboxes. That yields better resiliency, accuracy, performance, and scalability.

Measured benefits: substantial infrastructure cost savings, particularly for model-bound agents; flexible sandbox allocation, since not every agent needs the most expensive sandbox (and those get expensive); and sandbox reuse for agent types where security policy permits it.

She closed with an invitation: she believes this new platform wave is coming, they've started the journey, and she wants to hear from people who agree, disagree, or are working in the same direction.

Quotes

"AI applications are without any doubt the most non-deterministic and unreliable component that has ever been introduced in enterprise architectures." (~00:34:20)

The premise of the whole talk: the job isn't to fix the AI, it's to design infrastructure around a component this unreliable.

"It's very difficult to express what agents should do … but it's even harder to specify what agent shouldn't do." (~00:36:26)

A direct echo of Ion Stoica's requirement gap — omissions versus exclusions.

"We essentially do not have any reliable error codes." (~00:36:45)

One sentence for why agent observability and automated recovery are so hard.