Observability Is Not Governance: Building a Runtime Trust Plane for Agentic AI

TL;DR

• A blunt correction of industry vocabulary: observability and governance get used interchangeably, but observability only answers "what happened." Governance has to answer "can we do something about it?" The two are complementary and neither substitutes for the other.
• The object of governance is the control path, not the agent. The control path is dynamic, so you need hooks into it to assess in real time where something could go wrong, predict what could go wrong, define an outcome for when it does, and build risk controls against that.
• The implementation is declared vs. observed. They built a contract database of declared authority for every agent in the environment (what it does, what it touches, which tools it calls), then compared it against runtime observations and classified the variance as allowed / approved / drift / control failure. Control failure is the metric that matters — drift by itself isn't necessarily bad; knowing the delta is the point.

Key Points

Where it started: an audit that stopped the project (~02:31–02:36)

Thales builds security architectures and frameworks covering both their own internal systems and systems their end users will run. What set Shende off was, in his words, "what happened last week or the week before, where an agent decided to do its own thing." That produced three questions: what are our agents doing, how are we governing them, and what should we be concerned about in an operating environment?

He grounded it in two numbers:

• Gartner's forecast that as enterprises deploy agents and ML+LLM systems at scale, over 40% will fail by 2027. (Gartner's own wording is that over 40% of agentic AI projects will be canceled by end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.)
• A survey — he was explicit it is a survey, not an industry benchmark — showing only 14.4% of organizations are approaching this from a security perspective.

Then the story that landed with a room full of engineers. Engineers are told to build it, ship it, get it out the door. Then audit arrives and pulls the brakes. It happened to him: audit checked the agents' privileges and entitlements and reported that they could just do anything they wanted. The team had to stop, re-engineer, work with the identity team, build identities, and ensure those identities carried a signed intent — a contract defining what the agent should be doing based on the outcomes it is supposed to deliver.

Before and after AI: the end of the linear environment (~02:36–02:38)

Before AI the environment was linear: user → SaaS → business systems. We knew what we were asking systems to do and what users were doing, so we could wrap governance controls and security metrics around those points.

Today a user, an agent, or both work together: a user issues a request, that request calls agents and models, orchestration layers on top — and the result is an entire ecosystem taking on a life of its own, in real time, simultaneously.

Thales's response works across three layers: execution, runtime governance, and an evidence layer. The rationale: following linear processes, a linear governance model, the basic testing we do for software, and embedded QA is not enough today, because workflows occur in tandem, functions are tied to those workflows, and sometimes we aren't sure who is responsible or accountable for those functions.

That drives the need for a database of transactions: records of what executed, what the impact was, how to build risk metrics from that impact, and from there the evidence layer.

The sub-agent privilege-inheritance problem (~02:38–02:39)

Underneath, everything is subdivided by human request and delegated authority. The concern he and his team keep returning to: when privileges and entitlements are assigned to a sub-agent, is that sub-agent acting as it should? If it acts on behalf of another agent and inherits the primary agent's privileges, entitlements, and authorizations — should that be happening?

His answer matched the room's: it should not. But he immediately named the gap — this room is a finite group, industry is much larger, and the challenge he sees is people skipping steps because they need to build and ship, which is exactly how risk gets in.

The core argument: observability ≠ governance (~02:39–02:42)

Because of the evidence layer, they watch the control path rather than the agent. The control path is dynamic, so hooks into it are required to assess in real time where something could go wrong, predict what could go wrong, define an outcome when it does, and build risk controls against that.

Then the thesis:

• Observability is good — we can see what's happening. But can we do something about it? That's the question.
• When we talk about governance, we know something is occurring because the model is executing; what's needed is to assess the control path and apply governance on that control path based on an outcome.
• That outcome exists to enforce decisions, and those decisions are bound to identity, policy, and authorization. With that data you build a decision matrix, then enforce controls within the execution path based on risk to the control path.
• Conclusion: observability plus governance — not replaceable, not substitutes, they work in tandem.

Declared vs. observed and the runtime trust plane (~02:42–02:46)

In practice they built a contract and a database of every agent in their environment, defining what each was doing, what it interacted with, what tools it called, and what systems it engaged — the declared authority.

Once everything was running, the predictable happened: variances, behavior that was not what they had declared. So they built variance analysis across their runtime trust plane, comparing declared vs. observed alongside metrics from identity, delegation, policy, and risk, plus the evidence layer.

His second counterintuitive point landed here: drift is not necessarily bad — we need to know the delta. They categorized explained variance into four buckets:

1. shown / allowed
2. approved
3. drift
4. control failure

Asked which mattered most, the room got it right: control failure.

The engineering environment was constrained on four constraints, among them ephemeral agents, observed drift, and latency — he credited the Oracle healthcare speaker earlier for the point that latency feeds directly into cost. From that data they built a containment path engineered around a defined kill switch with four actions: quarantine, roll back, obtain secondary authorization, or stop.

Finally, the pre-execution adjudication: for a function issued by an agent plus a decision proposal, they evaluate an envelope on the runtime trust plane across authority, policy, risk, and evidence before allowing an execution commit — tagged with risk categories, where low risk auto-executes and high risk requires human approval.

He summed up what they built as a record of trust running in real time across the top boundaries, covering everything from identity to action. Out of time, he mentioned one more concept in their environment — guardian agents — and left it at "we can talk about that later."

Quotes

"Observability is good. We can see what's happening, right? But can we do something about it? That's the question." (~02:40)

The talk title, unpacked in one line.

"We are a finite group. Over in larger industry, the challenge I'm seeing is people are skipping steps because we need to build and ship. And if we build and ship, we open ourselves up to risk." (~02:39)

The people in the room knowing better is not the same as industry doing better.

"Drift is not necessarily bad. We need to know the delta." (~02:43)

Governance aims to make deviation classifiable and adjudicable, not to eliminate it.