On this page

Put Coworker to work on your stack.

Connect Salesforce, Slack, and Jira, and have your first agent live in days.

Book a demo

AI Agent Governance: A Practical Framework for 2026

AI agent governance explained by Coworker AI: the risks, the eight controls every framework needs, the standards that apply, and a checklist to start.

Dhruv Kapadia18 min read

AI agent governance is the set of policies and technical controls that decide what each AI agent can access, what it may do without a human, who is accountable for it, and how you watch and stop what it does. It differs from general AI governance because agents take actions in real systems, so the questions shift from "is this answer accurate?" to "should this agent be allowed to update that record, and who signs off?"

The gap is wide. In a Deloitte survey of 3,235 IT and business leaders across 24 countries, only 21% said their organization has mature governance in place to manage the risks of agentic AI (Deloitte, April 2026). This guide lays out the risks, the eight controls a working agentic AI governance framework needs, the standards that apply, and a checklist to start with.

I work at Coworker, which builds an AI platform with agents, so I say where our approach fits and where it does not. For the broader program, including AI policy, tool sprawl and frameworks such as the NIST AI RMF and ISO/IEC 42001, see my guide to enterprise AI governance.

What is AI agent governance?

AI agent governance applies governance to software that acts on its own. A chat assistant drafts text a person then decides to use. An agent can read a CRM, update a ticket, send an email or call another agent, often on a schedule and without anyone watching each step.

That changes what you need to control:

General AI governanceAI agent governance
Main questionIs the output accurate, fair and safe to use?Should this agent take this action on this system right now?
Unit you governModels and use casesIndividual agents, their tools and their permissions
Key risksBias, hallucination, data leakage in promptsUnauthorized actions, privilege abuse, runaway cost, agents nobody owns
Key controlsPolicies, model review, usage rulesIdentity, least privilege, approvals, monitoring, kill switch
Who is involvedLegal, risk, data scienceThose plus security, IT, system owners and the agent's business owner

Microsoft's Cloud Adoption Framework sums up what leaders need: to know which agents exist, who owns them, what they can access, and how to intervene when behavior falls outside policy (Microsoft Learn). That is a good one-line test for any agentic AI governance program.

Evaluating enterprise AI for your team? See Coworker on your own tools in a 20-minute demo.

Book a demo

Why does AI agent governance matter now?

Because agents are spreading faster than the controls around them. Recent surveys point the same way:

  • Governance lags adoption. Only 21% of enterprises report mature governance for agentic AI (Deloitte).
  • Process gets skipped. About half (47%) of senior AI executives in an EY survey said their organization has previously not applied its AI governance process for an urgent deployment (EY, September 2026).
  • Incidents are common. OneTrust's 2026 survey of 1,200 senior decision-makers found 86% of organizations had experienced AI-related incidents, yet few slowed deployment (OneTrust). OneTrust sells governance software, so read it as a vendor survey.
  • Projects get cancelled. Gartner predicts that by the end of 2027, 40% of agentic AI projects will be cancelled because of escalating costs, unclear business value or inadequate risk controls (Gartner).

The practical lesson: governance is not what slows agents down. Missing governance is what gets them shut off.

What risks does AI agent governance need to cover?

The best shared list is the OWASP Top 10 for Agentic Applications, published by the OWASP GenAI Security Project in December 2025 as its 2026 edition. Each risk maps to a control you can assign an owner to.

OWASP riskWhat it looks likeControl that addresses it
ASI01 Agent Goal HijackContent the agent reads redirects it to a harmful taskTreat retrieved content as untrusted; approvals on actions
ASI02 Tool Misuse and ExploitationUnsafe tool calls leak data, destroy records or run up costNarrow tool scopes; rate and cost limits
ASI03 Identity and Privilege AbuseAgent acts with more access than the person it servesPer-agent identity; least privilege; permission inheritance
ASI04 Agentic Supply Chain VulnerabilitiesA compromised tool, plugin or MCP serverApproved tool and server list; vendor review
ASI05 Unexpected Code ExecutionAgent runs code it should notSandboxing; no code execution by default
ASI06 Memory and Context PoisoningBad data in memory or retrieval steers future runsSource tracking; permission-aware memory; review of what is stored
ASI07 Insecure Inter-Agent CommunicationAgents trust each other's messages blindlyAuthenticate agent-to-agent calls; limit delegation
ASI08 Cascading FailuresOne agent's error spreads through othersCircuit breakers; step limits; monitoring
ASI09 Human-Agent Trust ExploitationPeople approve what the agent proposes without checkingClear approval screens that show the change and its source
ASI10 Rogue AgentsAgents running with no owner or outside policyInventory, ownership and a kill switch

Risk names are from the OWASP list as summarized by Modulos and the FINOS AI Governance Framework. The control column is my mapping.

What does an AI agent governance framework include?

Across vendor guidance, standards bodies and the OWASP list, the same eight controls keep appearing. Treat them as the minimum.

1. Inventory and ownership

Every agent is registered, with a named human owner, a business purpose and the systems it touches. Microsoft's agent registry in the Microsoft 365 admin center, for example, highlights agents without owners (Microsoft Learn), and Microsoft's governance guidance treats agents with no registered owner, no reviewed access and no monitoring as shadow agents (Microsoft Learn). If you cannot list your agents, you cannot govern them.

2. Identity for every agent

Each agent gets its own identity rather than borrowing a person's session or a shared API key, so actions can be attributed and policies applied. Microsoft gives each agent a distinct Microsoft Entra Agent ID (Microsoft Learn), and Google's Agent Identity issues a cryptographic identity per agent based on the SPIFFE standard (Google Cloud). An IETF draft proposes best practices for agent authentication and authorization built on OAuth 2.0 and workload identity standards (IETF).

3. Least privilege and permission inheritance

An agent should never see more than the person it acts for. When an agent acts on someone's behalf, it should inherit that person's permissions in each source system, not hold a broad service account. The IETF draft above makes the same point: when an agent acts on behalf of a user, the user delegates authority and the user's context is used in authorization decisions. For agents that reach tools over MCP, the protocol's authorization flow uses OAuth 2.1 (MCP docs); my post on MCP security covers the gaps.

4. Human approval for consequential actions

Decide which actions an agent may take on its own and which need a person to confirm: writing to a CRM, sending external email, changing permissions, spending money. Show the approver the exact change and the evidence behind it, because rubber-stamped approvals (OWASP's ASI09) are their own risk.

5. Observability and audit

Log every run: what triggered it, what the agent read, which tools it called, what it changed and which model it used. Logs are how you investigate incidents, prove compliance and spot agents that quietly stopped working. See my guide to LLM observability for the tooling side.

6. Runtime limits and a kill switch

Cap steps, tool calls, spend and run time per agent, and keep a way to stop an agent, or every agent, immediately. ServiceNow's AI Control Tower, since renamed SI Control Tower, added runtime monitoring and a kill switch in its August and September 2026 release (ServiceNow Community), a sign that this is becoming standard.

7. Cost controls

Agents spend tokens on every step, and a looping or over-eager agent can burn a budget fast. Set budgets per agent and per team, route simple steps to cheaper models, and give agents organized context instead of letting each one search everything from scratch. I cover the levers in AI cost optimization and AI token cost.

8. Lifecycle: intake, testing, review and retirement

Agents need a lifecycle like any software: a lightweight intake that records purpose and risk tier, testing against real tasks before rollout, periodic review of permissions and usage, and retirement when the owner leaves or the agent stops being used. Agents with no owner are where most "rogue agent" stories start.

Coworker

Put an AI agent to work on your stack

Connect Slack, Salesforce, and Jira, and have your first agent live in days.

Try the no-code AI agent builder
Book a demo

Which frameworks and standards apply to AI agents?

No single standard covers agents end to end yet, but several are converging:

  • OWASP Top 10 for Agentic Applications (2026 edition). The most practical risk list, covered above.
  • NIST SI Agent Standards Initiative. Launched February 17, 2026 as the AI Agent Standards Initiative by NIST's Center for AI Standards and Innovation, and since renamed. Its goal is agents that "function securely on behalf of its users" and interoperate (NIST). NIST's NCCoE is also running a project on identity and authorization for software agents (NCCoE).
  • Singapore's Model AI Governance Framework for Agentic AI. Launched January 22, 2026 by IMDA for organizations deploying agents, whether built in-house or bought, with key considerations in four areas and an emphasis that humans remain accountable (IMDA).
  • EU AI Act. The AI Omnibus (Regulation (EU) 2026/1744) entered into force on July 27, 2026 and delayed the high-risk rules: the European Commission now lists December 2, 2027 as the date strict obligations start for high-risk AI systems in areas such as hiring, with August 2, 2028 for high-risk AI built into regulated products (EUR-Lex, European Commission). If an agent is part of a high-risk use case, such as hiring decisions, plan for those obligations now.
  • NIST AI RMF and ISO/IEC 42001. The general frameworks still apply to agents; I cover them in enterprise AI governance.

This is general information, not legal advice. Check obligations with your counsel.

How do the major platforms govern agents?

Agent governance is increasingly a platform feature, which helps if most of your agents live on one platform and complicates things if they do not.

  • Microsoft gives each agent an Entra Agent ID and lists agents in an agent registry in the Microsoft 365 admin center, including agents without owners (Microsoft Learn).
  • Google issues cryptographic Agent Identities so agents can authenticate to MCP servers, cloud resources and other agents, on their own behalf or on behalf of a user (Google Cloud).
  • ServiceNow governs agents in its Control Tower, now called SI Control Tower, with runtime monitoring and a kill switch (ServiceNow Community).

The catch is that each platform governs its own agents. A company running agents in Microsoft, Salesforce, ChatGPT and a few internal tools ends up with several registries and several sets of rules. That is the agent version of tool sprawl, and it is why an inventory that spans platforms matters. I compare the platforms themselves in my roundup of AI agent platforms, and in Copilot Studio alternatives and Agentforce alternatives.

What does AI agent governance look like in practice?

Three common agents show how the controls change with risk.

A sales agent that updates the CRM after calls

The agent reads call transcripts and proposes updates to opportunity stage, next steps and close date. Risk: wrong data in the pipeline that leadership forecasts from. Controls: the agent acts with the rep's own CRM permissions, every update is shown to the rep for confirmation with the transcript excerpt behind it, and all changes are logged. I describe the pattern in auto-updating the CRM after sales calls.

A support agent that triages tickets

The agent reads new tickets, tags them, links related issues and drafts replies. Risk: customer-facing mistakes and exposure of other customers' data. Controls: drafts need agent approval before sending, the agent only sees tickets the assigned team can see, and a runtime cap stops it from looping on a flood of tickets. More on AI for support teams.

A finance agent that prepares invoices

The agent reads contracts and drafts invoices. Risk: money. Controls: read-only access to contracts, drafts only (never sends), mandatory review by a named approver, a narrow service identity, and a quarterly review of what it has access to.

The pattern is the same each time: the more an agent can change, and the closer that change is to customers or money, the tighter the identity, approval and logging controls.

What are common AI agent governance mistakes?

  • Governing only the platform you bought. Agents also live in automation tools, ChatGPT workspaces and scripts that teams built themselves. An inventory that misses them misses the riskiest ones.
  • Giving agents a broad service account. It is faster to set up and it breaks least privilege on day one. Prefer per-agent identities and inherited user permissions.
  • Approvals nobody reads. If approvers click through dozens of requests a day, the control is theater. Approve fewer, more meaningful actions, and show the evidence.
  • No owner after launch. Agents outlive the projects that created them. Tie each agent to a named owner and review it when that person changes roles.
  • Ignoring cost until the bill arrives. Token spend is a governance issue, not only a finance one. Budgets and alerts per agent catch loops and over-eager agents early.

My enterprise AI buyer's checklist has the vendor-side questions to ask before you buy.

How do you start an AI agent governance program?

A 90-day plan that works for most companies:

Days 1 to 30: see what you have. Inventory every agent and AI integration you can find, including ones teams set up themselves in ChatGPT, Copilot or automation tools. Assign an owner to each, or switch it off. Write a one-page policy: which actions need approval, which data agents may touch, who approves new agents.

Days 31 to 60: put controls on the riskiest agents. Start with agents that write to customer, financial or HR systems. Move them to their own identities, narrow their permissions, require approval for writes, and turn on logging.

Days 61 to 90: make it routine. Add a lightweight intake for new agents, set budgets and runtime limits, test agents on real tasks before wider rollout, and schedule a quarterly review of owners, permissions and spend.

Keep it proportional. A read-only agent that summarizes public docs does not need the same review as one that edits contracts. A simple way to do that is three risk tiers. Tier 1 agents only read and summarize, and need an owner and logging. Tier 2 agents draft changes for a person to approve, and add narrow permissions and testing. Tier 3 agents act on their own in customer, financial or HR systems, and add a named approver for launch, spend limits and a quarterly access review.

AI agent governance checklist

ControlQuestion to answerDone when
InventoryCan we list every agent?Every agent has a registry entry
OwnershipWho answers for each agent?Every agent has a named owner
IdentityCan we tell which agent did what?Each agent has its own identity
Least privilegeCan an agent see more than its user?Agents inherit user permissions or narrow scopes
ApprovalsWhich actions need a human?Writes and external actions require confirmation
ObservabilityCan we reconstruct any run?Runs, tool calls and changes are logged
Runtime limitsCan a looping agent run forever?Step, time and spend caps are set
Kill switchCan we stop an agent now?One action stops an agent or all agents
CostDo we know cost per agent?Budgets and alerts per agent or team
LifecycleWhat happens when the owner leaves?Agents are reviewed and retired on schedule

How does Coworker handle agent governance?

Coworker's approach is to make company context the governance boundary. OM2, Coworker's organizational memory, is a permission-aware knowledge graph built from 50+ connectors. Access rules from the source tools travel with every fact, so people and agents only recall what they can already see in Salesforce, Slack, Google Drive and the other connected tools.

On top of that:

  • Agents have owners and roles. Every Coworker agent has an owner and is shared as a Member, who can use it, or a Manager, who can edit and share it, either person by person or with the whole company. People outside the company can only be Members.
  • Approvals where you want them. Each agent can be set to draft its write actions, such as CRM updates and Jira tickets, for a person to confirm, per agent and per tool. Agents without that setting act directly, so turn it on for anything that touches customer, financial or HR data.
  • One governed context across AI tools. The same graph is available in Coworker's apps and in Claude, ChatGPT and Cursor through Coworker MCP, which an admin enables per company, so teams keep their preferred assistant without each one building its own connections.
  • Compliance basics. Coworker has SOC 2 Type II, GDPR and CASA Tier 2.

Where it is the wrong choice: if your program requires ISO/IEC 42001 certification from vendors today, Coworker does not have it. If your agents live almost entirely in Microsoft 365, Microsoft's agent registry and admin controls are built for exactly that. And Coworker governs its own agents and context; it is not a registry for agents built on other platforms. To see how it works on your data, book a demo.

Frequently asked questions

What is AI agent governance?

AI agent governance is the set of policies and technical controls that decide what each AI agent can access, which actions it may take without a human, who owns it, and how you monitor and stop it. It extends general AI governance to software that takes actions in business systems.

What is the difference between AI governance and AI agent governance?

AI governance covers models and use cases: accuracy, fairness, privacy and acceptable use. AI agent governance adds controls for software that acts on its own: per-agent identity, least-privilege access, approvals for actions, runtime limits, audit logs and a kill switch.

What should an AI agent governance framework include?

At minimum: an inventory with owners, an identity for each agent, least-privilege permissions, human approval for consequential actions, logging and audit, runtime limits with a kill switch, cost controls, and a lifecycle covering intake, testing, review and retirement.

What is agentic AI governance?

Agentic AI governance is another name for AI agent governance. "Agentic" refers to AI that plans and takes actions with some autonomy, so the governance focus is on what the AI is allowed to do, not only what it says.

What is the OWASP Top 10 for Agentic Applications?

It is a list of the ten most critical security risks for AI agents, published by the OWASP GenAI Security Project as a 2026 edition. The risks run from ASI01 Agent Goal Hijack to ASI10 Rogue Agents and include identity and privilege abuse, tool misuse, memory poisoning and cascading failures.

Does the EU AI Act apply to AI agents?

The EU AI Act regulates AI systems by risk level, so an agent used in a high-risk area, such as hiring, falls under those rules. After the AI Omnibus, the European Commission lists December 2, 2027 as the start of strict obligations for high-risk AI systems. This is general information, not legal advice.

What is an AI agent registry?

An AI agent registry is a central list of every agent in the organization, with its owner, purpose, identity, permissions and status. Microsoft 365 includes one in its admin center that highlights agents without owners. A registry is the foundation for every other control, because you cannot review, limit or stop an agent you do not know exists.

How do you govern AI agents across several platforms?

Start with one inventory that spans platforms, even if it is a spreadsheet at first, and one policy for approvals, data access and ownership that applies wherever an agent runs. Then use each platform's native controls to enforce it. Reducing the number of places agents connect to company data also helps, which is one reason teams route agents through a shared, permission-aware context layer instead of separate connections in every tool.

Who should own AI agent governance?

Ownership is shared. A central team, often security or IT with legal and risk, sets policy and runs the inventory and tooling, while each agent has a named business owner accountable for what it does. Singapore's agentic AI framework stresses that humans remain ultimately accountable for agents.

Ready to get started?

Put Coworker to work inside your actual stack

Connect Salesforce, Slack, Jira, or whatever you use, and have your first agent live in days.