From Multi-Agent Patterns to Reliable Orchestration

TL;DR

• Patterns are just wiring; the real problem is the delegation decision. Handoff (control transfers to another agent), agents-as-tools (the orchestrator delegates and waits for a result), and routing (a classifier dispatches each turn to a specialist) all reduce to the same runtime choice, whatever each enterprise calls it.
• Agent delegation is not tool selection. Tool selection is typically evaluated against one ground-truth label; in agent delegation overlap is normal — several agents can legitimately complete the same task — and the criteria include cost, latency, and the user experience you want to deliver.
• Even implemented as a tool call, agents are not tools: a tool call is bounded, agent delegation is unbounded. Delegating starts an autonomous loop that may reason, ask for clarification, get stuck, resume later, or never return cleanly.
• So evaluation can't just ask whether the final task succeeded: it also has to weigh whether the path was reasonable, plus cost, latency, and experience — and benchmarks should admit multiple acceptable choices with full runtime context attached.

Key Points

Grounding the problem in an in-car voice assistant (~01:15–01:16)

His background is agent architecture — system-level architecture for customer-facing agentic products — and GUI agents (computer-use agents) that operate screens the way users do. (He points to a talk he gave last year at the AI Engineer conference on why an enterprise would consider GUI agents at all.)

The running example is a multi-agent in-car voice assistant with three agents: a navigation agent for routing, a car control agent for windows, seats, and climate, and a GUI agent that drives applications through the screen. Above them sits an orchestrator that coordinates execution and sometimes combines results.

His thesis: the hard part of orchestration is not the patterns or how you shuttle state between agents — it's making the delegation and coordination choices well, repeatedly, at runtime, under constantly changing context. The pattern zoo looks large — handoff transfers control to another agent; agents-as-tools has the orchestrator delegate to a specialist and wait; routing dispatches each turn via a classifier — and every enterprise names them differently, but underneath it's the same thing: a choice.

Why this isn't ordinary tool selection (~01:17–01:18)

First, overlap is normal and ground truth isn't singular. Tool selection is usually scored against a single correct label. But for "who should handle this work in this context," multiple agents often can — the GUI agent in particular overlaps nearly everything — and the criteria extend to cost, latency, and the experience you want to give the person in the driver's or passenger's seat.

His example: the user says "play some jazz." With an empty screen, the media API may be the best route; but if a song or playlist is already on screen, tapping it is an equally valid choice. Contrast that with "navigate me to Munich," which unambiguously belongs to the navigation agent. A reliable system shouldn't force these cases into one canonical label — it should use runtime context and adapt.

Second, delegation is unbounded. Even when implemented as a tool call, agents are not tools: a tool call is bounded; agent delegation is unbounded. Delegating starts an autonomous loop that may reason, ask for clarification, get stuck, resume later, or never return cleanly. So orchestration isn't just selecting a capability — it's deciding when it's safe and useful to let another loop take over effectively unbounded control.

Consequences for evaluation and org design (~01:19–01:20)

Because multiple valid trajectories exist, a benchmark shouldn't only ask whether the final task succeeded. It has to account for whether the path was reasonable, plus cost, latency, and intended user experience; datasets need the full runtime context attached, and the metric depends on the situation — sometimes you optimize latency, sometimes human-machine collaboration.

For enterprises, his suggestion is organizational: each team that owns an agent builds its own benchmark, then teams combine those benchmarks into a shared orchestration and delegation benchmark that also covers collaboration and coordination patterns.

Closing: patterns are just the means of connecting pieces. What matters is making orchestration paths and delegation decisions measurable, context-aware, and reliable — so keep asking who handles this work, what path should they take together, and how do we know it was good?

Quotes

"A tool call is bounded and the agent delegation is unbounded. So basically delegating to an agent can start an autonomous loop." (~01:18)

The sharpest formulation of why agents aren't tools: the question is whether control comes back.

"The patterns are just the means to connect the pieces of the multi-agent system. What matters is making orchestration paths and delegation decisions measurable, context-aware, and reliable." (~01:20)

The thesis of the talk in one sentence.