From Agent Demos to Production: How Postman Is Building Reliable AI Agent Infrastructure

TL;DR

• The microservices lesson repeats itself with agents. The pile of specialized agents from their hackathon produced compounding cost and compounding confusion — the handoffs and testing between agents were the real problem. It was a decomposition problem, not something one more agent could fix.
• The more deterministic the outcome, the more reliable the agent. Their win-loss agent could say "we lost this deal because our price was too high," and a human could reason backwards to check it. Ask the same agent how to generate leads and it went off the rails, producing output nobody could validate. Automate the questions whose answers can be verified in reverse.
• Knowledge is not strategy. Asked which persona to sell to, the agent said "procurement" every single time — not technically wrong, but it would get you laughed out of the customer's room. Strategic content has to be supplied by the teams that own it: account plans from sales, solution plays from field engineering.

Key Points

Why enterprise sales, and the three problems the hackathon exposed (~00:13:39–00:17:45)

Postman already ran agent-first development, but wanted runtime agents that make decisions inside the business. Prototypes generated real excitement from the CEO and the board — and with it the harder questions: how do I measure this, manage this, and make sure what we ship stays maintainable? That is where ROI models and runbooks entered the picture.

They picked enterprise sales because the process is well defined (from identifying an opportunity to renewing the deal) yet spans many teams — and those teams struggle to hand off context to each other. Underneath sits a swarm of systems whose data relationships are hard to untangle. The net effect is an opaque process that leaks context at every step.

So they ran a hackathon (~00:15:52). In 30 hours engineers produced a mass of agents — drawn out, a tangle of systems, connections, and agents. The one that stood out was a win-loss agent, and from it came three core problems:

1. The more deterministic the outcome, the better the agent performed (~00:16). The win-loss agent could explain why an account was lost, and a human could work backwards from that reasoning and sanity-check it. Ask it "what's the best way to generate leads?" and it drifted, producing results nobody could judge as right or wrong. Separately, a more complex model reasoned better — but understanding its results got correspondingly harder.
2. It is a compounding problem (~00:17:13). Anyone who has built microservices recognizes the shape: A2A, agent communities, and then compounding cost plus compounding confusion, with handoffs and testing between agents becoming the dominant difficulty.
3. It is a decomposition problem (~00:17:31). They kept building individual agents without noticing that all of them belonged to the same domain (he name-checked Martin Fowler here). The interfaces between the systems and the handoff of context were the real challenge.

The four-layer solution (~00:18:00–00:20:45)

Layer 1 — the monolith agent (~00:18:00). The foundation is a single agent that understands the whole sales process: Salesforce, Gong calls, transcripts. It answers genuinely interesting questions and makes a great co-pilot — but in gray areas it starts making things up, and as a generalist facing specialized questions it simply has too much information to give the right answer.

Layer 2 — decomposition, two pieces (~00:18:42). - A hallucination layer: don't recommend anything unless you clearly understand where the information came from; if you don't have it, explain why you reached that result. Output that can do neither is discarded. - Domain-specific agents, which are not a coding task at all — they are prompts ("I'm a seller, here's what I typically do") that make the agent answer in that persona. This sharpened output and killed most of the hallucination, but left it confidently wrong.

Layer 3 — humans own the strategy (~00:19:35). Knowledge and intelligence are not strategy. Asked which persona to sell to, the sales agent answered "procurement" every time; walking into a customer and announcing you're selling to their procurement organization would get you laughed out of the room. The fix was to identify the teams that own strategic content and put them in charge of contributing it — sales adding account plans, field engineering adding solution plays. Outputs became strategically relevant.

Layer 4 — role-scoped authorization (~00:20:20). At this point they had a super agent that understood the entire organization and no controls at all. Beyond "is this true?" they added "is this content related to the job that's requesting it?" — so the CTO agent won't hand over the company's P&L; it factors in your context and job role and returns only what is relevant to you.

The agent autonomy curve and compounding value (~00:20:50–00:22:56)

The autonomy curve asks, at each stage, whether the human or the agent makes the decision — and there are two decisions to separate: who requests the information, and who decides when to promote the result.

• Help — the human does both: I ask the question, I analyze the result.
• Recommend — the agent starts taking over part of it.
• Act — the biggest step change: within guardrails, the agent does both sides of the equation.
• Own — where everyone is heading: long-running agents that own the entire ecosystem.

The payoff surprised them. Building an agent went from a 30-hour hackathon to a matter of hours: with the framework in place, someone sets up guardrails and builds on the existing body of knowledge. Consumption spread from a select audience to the whole organization. Value compounded on both axes — development time and number of people benefiting.

There was also an unplanned behavioral shift: once salespeople understood that the context they create and the questions they ask feed the agents' underlying value, they did more discovery and produced better-quality content.

Finally, they turned the learnings into a platform: measure the outcomes, keep a catalog of available agents that teams can "hire and fire," and see each agent's real impact in both cost and return on investment (~00:22:42).

Quotes

"The more deterministic the outcome, the better the agent was." (~00:16:20)

Choosing the problem matters more than tuning the prompt: start where a human can verify the answer in reverse.

"Don't recommend something unless you have a clear understanding of where you're getting that information from." (~00:18:42)

The hallucination layer, specified in one sentence.

"It sharpened the output and it stopped doing a lot of the hallucinations, but it was confidently wrong. Knowledge and intelligence does not mean strategy." (~00:19:25)

Personas fixed hallucination but couldn't manufacture strategy — that stays a human-owned asset.