Earning Autonomy: Governance as Code for the Agentic Enterprise

Speaker substitution: the agenda lists Credo AI founder and CEO Navrina Singh; she couldn't attend, so Head of Product Eric Aldana delivered the talk ("the first thing you should know about me is I am not Navrina Singh"). These notes record the speaker who actually presented.

TL;DR

• Safety and governance are the same job. Different circles treat them as separate disciplines, but both exist to make sure AI systems act the way the people behind them intend. In a large enterprise, intent is never one person's decision — it's a coordination problem across engineers, business owners, compliance and legal, customers writing requirements into procurement contracts, and regulators and auditors writing standards with real penalties.
• The missing ladder. Enterprises grant autonomy on consequential decisions only through earned authority: intern engineers start with limited production access, traders earn their investment limits. Organizations built these ladders over years of trial and error — and for agents that ladder largely doesn't exist yet.
• The can / may / act model. Can is capability — what a model and its tools are technically able to do, established by benchmarks. May is authority — what the agent is permitted to do here and now with the data and tools it has, scoped and revocable. Act is autonomy — the authority it can exercise without waiting for a person.
• Today's tooling only covers "can". Security asks whether an attacker can make an agent do something bad; observability asks whether it's doing its job well. Both live in the can layer. Nothing addresses may.
• The insertion point is the agent's harness — not an evaluation harness but a governing one: the governance plan compiled into versioned, configurable code as skills, hooks, guidance, and managed settings, installed directly where the agent runs, not in an MCP server, API gateway, or proxy.

Key Points

Credo AI's vantage point: whose problem governance is (~00:56–00:58)

Credo AI is an AI governance platform whose users are the people at large enterprises and government agencies who are accountable for AI — not just those who build it, but those who have to decide which AI systems are safe and trustworthy to build, buy, and deploy across their organizations.

Opening on an AI safety track, he made a joining move:

Depending on what circles you run in and what jargon decoder ring you're using, safety and governance are often treated as separate disciplines. But at their core, I'd argue they share the same job — making sure AI systems act in a way that the people behind them intend.

And in a large enterprise, intent itself is a coordination problem across at least five parties:

• Engineers building the system
• Business owners defining the problem being solved
• Compliance and legal deciding what actions the organization is okay with
• Customers, who now write requirements into procurement contracts
• Regulators and auditors, writing standards that carry real penalties

So governance is how all of these different intentions get reconciled and then enforced into how an AI system actually behaves.

The ladder that doesn't exist yet (~00:58–00:59)

His central analogy: when a large enterprise lets a human act autonomously on consequential decisions, that authority was earned through some structure.

• Every intern engineer starts with limited production access and earns more as they rise.
• Every trader at an investment firm has to earn their limits on what they can invest.

These ladders of earned authority were built over years of organizational trial and error. And:

For agents, that ladder largely doesn't exist yet. Many agents receive full authority the moment they're deployed, and that is a design error — which is what I want to address today.

Compressed to a line:

Capability might come from the model, but autonomy must be earned from the enterprise.

Agents dissolved the use case as a unit of governance (~00:59)

Across six years of shipping AI governance processes and tools, the unit of governance was the use case — an AI tool plus the context it's deployed in. A chatbot that answers employees' questions about company policy, say: you can draw a very clear boundary around that.

Agents dissolve the boundary.

What is a use case for an agent that can plan its own steps? What connectors should it have access to? What data sources?

And when the use case blurs, the parameters you use to govern it blur too. That pressure produced the new mental model.

Can / may / act (~00:59–01:00)

• Can — capability: what a model and its tools are technically able to do. Capability is benchmarked.
• May — authority: what the agent is permitted to do here and now, with the data and tools it has access to. Authority is scoped and revocable.
• Act — autonomy: the authority it can exercise without having to wait for a person.

Within that framing came the load-bearing claim:

Autonomy is not just a model property. It is an enterprise permission earned through evidence.

And then the framework's real diagnostic payoff — the existing tooling stack all stops at can:

• Security asks whether an attacker can make an agent do something bad.
• Observability asks whether it can do its job well.

These systems are essential, but they all speak to that can layer — what the agent is able to do and whether it's doing it well. None of them are really addressing the may piece.

Three jobs for a governance system, landing in the harness (~01:00–01:02)

An AI and agentic governance system needs to do three things:

1. Compose — take the full context of the AI system and its organization and create a governance plan: which risks apply and which controls need to be applied.
2. Conduct — execute that plan, whether through evaluations or guardrails.
3. Enforce and keep the record of all those checks.

The way Credo AI is thinking about landing this is directly through the agent's harness — the actual runtime code that sits between the agent and the systems it can touch. He was explicit about the distinction:

This isn't an evaluation harness — instead it's a governing one.

Take the composed governance plan and embed it into a versioned, configurable set of code that a harness can enforce: skills, hooks, guidance, managed settings, installed directly where the agent runs — not something like an MCP or an API gateway or proxy.

His worked example is a governed Claude Code session:

• The agent tries to read an approved repo → the system allows it.
• It tries to open a credential store → the configuration blocks it.
• A dependency above a risk threshold appears → this gets escalated to the code owner.

And every one of these checks needs to be surfaced, which the same configuration handles.

The close returns to the ladder:

This ladder for earning autonomy that we said didn't exist for agents — this is it. A system where the enterprise can collectively decide what the agents can do, enforce those decisions where the agents act, and keep the proof.

Quotes

"Capability might come from the model, but autonomy must be earned from the enterprise." (~00:58)

The spine of the talk, and the unpacking of its title.

"Many agents receive full authority the moment they're deployed, and that is a design error." (~00:58)

Reclassifies over-permissioning from an operational slip to a design error.

"None of them are really addressing the may piece." (~01:00)

The shared blind spot of security and observability tooling.

"A system where the enterprise can collectively decide what the agents can do, enforce those decisions where the agents act, and keep the proof." (~01:02)

Governance-as-code, specified in one sentence.