Mindful Agents: Human-Inspired Memories for Long-Horizon Tasks

TL;DR

• Human memory works because it is constrained. On a train through endless European fields you don't remember every blade of grass — you remember what might matter to you. That distillation is an evolved response to a storage limit. Agents have no such limit, so they log everything and then summarize summaries of summaries, which breaks down the moment you need something specific.
• Memory is a feedback loop, not a store: ingestion → consolidation (dedup/merge, the "sleep" analogue) → forgetting (decay and interference) → retrieval (cold-storage graph plus live stream) → maturation/reconsolidation to resolve conflicts. Crucially, consolidation cadence is domain-dependent, not necessarily temporal.
• Three separate evals are required: retention (is what you kept actually important? needs labels), retrieval (can you get it back out?), and task completion (closest to the user, most expensive, most important — the best memory is worthless if the harness lets the agent ignore it).

Key Points

The setup: agents don't have our constraints (~02:00–02:02)

The speaker is on the Microsoft Fabric AI team, working on long-running agents in the real-time intelligence space — high-volume streaming data.

Her framing contrast: human storage constraints are what make humans good at distilling memory. Agents get to log every bit and every trace cheaply, so the default pattern becomes capture → retrieve → summarize → keep the summary, recursively. Fine for most cases, but it fails exactly when you need to retrieve something specific — which is what proactive agents over large data volumes need.

The memory life cycle (~02:02–02:05)

Five stages, run as a feedback loop:

1. Ingestion from the observability substrate.
2. Consolidation — dedup and merge into stable candidate memories, loosely analogous to sleep. The difference from humans: consolidation cycles are domain-dependent and may not be temporal at all. Her Formula 1 example: race-day telemetry, post-race, off-week, and full-season each imply a different cadence. It need not be daily, but it must happen in some batch at some cadence.
3. Forgetting via decay and interference, so the important survives and the rest doesn't.
4. Retrieval — hybrid: stable memories live in cold storage organized as a graph for a known domain, while live data keeps streaming in. The agent must reach both and pick the right memory for the moment.
5. Maturation / reconsolidation — retrieval will surface conflicts between new events and settled memories. The decision is whether to update an existing graph entity, add a new one, or discard the incoming live data as erroneous. Blindly appending makes the graph unretrievable.

Evaluation: the ceiling, the floor, and three layers (~02:05–02:09)

Two studies bracketing best and worst case:

• Retention benchmark (deterministic end): clean labels for what matters. They tuned consolidation batch size and frequency; things stabilized around 200 events per batch, with precision improving further once forgetting was layered on.
• Retrieval benchmark using LongMemEval (hard end): conversational data with no importance labels — not the ideal shape for their domain-expert-doing-repeated-tasks setting, but usable for tuning how much to compress and what the optimal storage size is before compression becomes destructive. It's a compression/accuracy trade-off.

Generalizing, memory work needs three kinds of eval:

1. Retention — quick and deterministic, but label-dependent; they have ongoing work on getting better labels.
2. Retrieval — necessary but not sufficient: a memory store that's hard to navigate is a memory store you can't use.
3. Task completion — closest to the user and therefore the most important, and by far the most expensive to build and run for long-running agents. The harness mediates whether the agent even uses the memory you gave it.

Takeaways and what's next (~02:09)

• Logs are not memories. At this data volume, efficiency forces you to decide what to keep.
• Domain matters and needs to be pre-declared; inferring it at runtime costs you both efficiency and accuracy.
• Next: end-to-end evals with the Microsoft Foundry team, plus work on domain learning and graph ontology — how you structure the graph and its ontology turns out to affect accuracy as much as any of the individual components.

Quotes

"Logs are not memories." (~02:08)

The cleanest rebuttal to "we store all the traces, so we have memory."

"You might give the best memory to the agent and depending on how it's exposed to the agent in your harness, the agent might choose to ignore it and still might not accomplish the task." (~02:08)

Why the expensive task-completion eval is unavoidable: memory quality does not equal task success.