Toward Adaptive Agent Frameworks

TL;DR

• Three research questions his team at AWS is working on, motivated by monitoring and observability agents for large-scale cloud operations: how should an agent allocate compute; how do we make agents reason better; and how do agents evolve over time.
• Budget-aware adaptive execution: most agents today have a fixed execution policy defined by a prompt or a local heuristic. They formalize it as a constrained Markov decision process — maximize expected utility subject to a computational budget (latency, cost, tokens, whatever you like) — and learn the policy. It beats the scalar reward model commonly used in practice and is markedly more sample-efficient.
• Parallel reasoning: let the model explore multiple reasoning paths in parallel, with those paths learned during training rather than unrolled at inference time, then coordinated into a single answer. Consistent gains on math and coding benchmarks from pass@1 through pass@k, with k=1 the standout.
• Continual adaptation: principled post-training schemes that adapt to new tasks without forgetting what was learned before.

Key Points

Starting point: capable agents aren't yet dependable ones (~00:24–00:25)

Kasiviswanthan works on AI agents for monitoring and observability in large-scale cloud operations at AWS. Today's agents are already good at planning, tool execution, retrieving information, and iterating toward a solution. But as they get deployed into mission-critical applications, AWS sees critical research questions that need answers first:

1. How should an agent allocate computational resources given a fixed budget?
2. How do we make agents reason better — efficiently exploring multiple paths before deciding?
3. How do agents evolve over time, rather than staying static?

The talk covers the first two and closes on the third.

Direction 1: budget-aware adaptive execution (~00:25–00:27)

Most agents run a fixed execution policy you define with a prompt or a heuristic. The goal is to have agents learn that policy instead.

Abstractly: the agent holds some context and must pick the best next action — re-retrieve information, use a tool, verify results, update memory — and the binding constraint is a budget it must respect while executing tools. That formulation is a constrained Markov decision process: maximize expected utility subject to a computational budget, where the budget can be latency, cost, tokens, or any other notion.

Does it work? Two plots say yes. One shows adaptive agents learning a better execution policy than the scalar reward model commonly used in practice; the other shows the approach is more sample-efficient — reaching the same level of competence takes significantly fewer iterations.

Direction 2: parallel reasoning (~00:27–00:28)

Models today follow a single reasoning path. Their work lets models explore multiple reasoning paths in parallel, and the important detail is that those parallel paths are learned during training, not spun up at inference time. A coordinated reasoning step at the end collapses the multiple paths into one answer.

Tested on mathematical reasoning and coding benchmarks, the results consistently beat strong baselines from pass@1 through pass@k — and are especially impressive at k=1, the single-shot case.

Direction 3 and wrap-up: continual adaptation (~00:29)

The third thread is continual model or agent adaptation: principled post-training schemes that adapt to new tasks, where the main challenge is not forgetting what was previously learned. Solving all three, he argues, is what gets us to the next generation of frontier agents.