The conversation about AI agent governance has matured noticeably over the last twelve months. The earliest deployments were governed mostly by hope. The deployments that survived their first incidents got governance retrofitted. The deployments going live now are increasingly built with governance in from the start. That’s progress.

It’s not yet maturity. The frameworks that exist are largely organisation-specific, the language is inconsistent across vendors and standards bodies, and the practical playbooks are scattered across blog posts and conference talks rather than codified anywhere usable. May 2026 is a useful moment to map where the field actually sits.

What “governance” means in practice

Strip away the jargon and AI agent governance comes down to a few practical questions. What is the agent allowed to do? Who approved that scope? How do we know what it actually did? What happens when it does something wrong? Who’s accountable?

The frameworks that work are the ones that answer these questions concretely for each agent in production, with names attached, audit trails attached, and incident response paths attached. The frameworks that fail are the ones that produce policy documents nobody reads.

The patterns that are working

Three patterns have emerged across organisations I’ve talked to that are running AI agents at meaningful scale.

A registry of agents. Every production agent is registered with a name, an owner, a defined scope, a defined data access list, and a defined set of approved actions. The registry is the source of truth. Agents not in the registry are not allowed to operate against production data.

Tiered approval thresholds. Agents that read data have one approval threshold. Agents that take actions in low-risk systems have a higher threshold. Agents that take actions affecting customers, money, or regulated data have the highest. The thresholds are not symbolic — they involve actual review by people qualified to assess the risk.

Continuous evaluation. The agent’s behaviour is monitored against a defined evaluation set, with regression detection on the metrics that matter. When the agent’s outputs drift, the team is alerted. When the drift is large, the agent is rolled back to a known-good version.

These three patterns are not exotic. They’re the AI agent equivalent of basic software change management. The organisations that have implemented them well are the ones whose agent deployments are stable. The organisations that haven’t are the ones whose agent deployments make the news for the wrong reasons.

The harder problems

A few problems are not yet well-solved.

Agent-to-agent interactions. When agents call other agents, the chain of responsibility gets blurry. If agent A asks agent B to do something and agent B does something problematic, where did the failure originate? The current frameworks don’t have clean answers and the deployments running multi-agent systems in production are mostly relying on the human-in-the-loop being very alert.

Audit trails for agentic systems. Logging every input, output, and intermediate decision is technically possible but produces volumes that are hard to make useful. The current generation of observability tools handles the volume; the search and retrieval over that volume is where many teams are still figuring things out.

Cross-system access controls. Agents that span multiple enterprise systems hit the limits of traditional identity and access management. The IAM patterns built for human users don’t always translate cleanly to agents that need different scopes at different times.

Regulatory alignment. Agent governance has to align with the regulatory environment the organisation operates in. In Australia that means the privacy framework, the financial services guidance, the AI policy positions of relevant regulators, and any sector-specific rules. The intersection of agent capability and Australian regulatory expectation is a working area, not a settled one.

What boards are asking

Boards in 2026 are asking sharper questions about AI agents than they were a year ago. The naive “are we doing AI” has been replaced by something closer to “where are our agents, what can they do, what’s the risk exposure, and who’s accountable when something goes wrong.”

The organisations that can answer those questions with specifics are the ones whose AI investment is allowed to continue and grow. The organisations that can’t answer are increasingly seeing their AI programs paused for governance work to catch up.

A growing number of organisations have engaged external partners — specifically AI consulting firms with delivery experience — to help build and operationalise their agent governance frameworks. The model that works is one where the external partner brings the framework, the templates, and the outside view, and the internal team builds the institutional ownership. The Team400 team has been involved in a number of these governance build-outs in Australia, particularly where the agent program is coupled with a broader AI strategy refresh.

What standards bodies are doing

The international standards conversation is ongoing. ISO and the relevant national bodies have published AI management system standards. The translation from those standards to actual agent governance frameworks is more art than science. The standards are useful as a checklist; they are not a recipe.

The national-level guidance from regulators in different jurisdictions is becoming more prescriptive about specific agent risks. The pattern is that high-risk agent applications — agents in healthcare, financial advice, legal, hiring — are getting more specific guidance, while lower-risk applications are subject to general principles.

For organisations operating across multiple jurisdictions, the governance framework has to handle the most prescriptive jurisdiction in scope. That has implications for how agents are designed.

Where this goes

By the end of 2026 I expect agent governance to look more like software supply chain governance does today — a set of practices that are obvious in retrospect, generally adopted by mature organisations, and broken in the operations that get into trouble. Two years after that, I expect codified standards that span jurisdictions to be in place for the high-risk categories.

The organisations that are quietly building agent governance now will be in a much better position than the organisations that are waiting for clearer rules. The rules will arrive. They will be substantially shaped by what the early adopters figured out.