How should organizations govern the risks of agentic AI?
IMDA's governance framework organizes oversight into four specific areas—risk bounding, human accountability, technical controls, and end-user responsibility—tied to agent action-space and autonomy. This matters because agents act adaptively rather than generating fixed outputs, requiring governance approaches distinct from traditional AI principles.
Singapore's Infocomm Media Development Authority lays out its "Model AI Governance Framework (MGF) for Agentic AI" around "four areas for agentic AI": "1. Assess and bound the risks upfront 2. Make humans meaningfully accountable 3. Implement technical controls and processes 4. Enable end-user responsibility." The framework builds on IMDA's "previous model governance frameworks" but argues that general principles for trusted AI — "transparency, accountability and fairness" — "need to be translated in practice for agents," because agents "take actions, adapt to new information, and interact with other agents and systems to complete tasks on behalf of humans" rather than only generate content. IMDA states the framework is meant for organizations "deploying agentic AI, whether by developing AI agents in-house or using third-party agentic solutions."
IMDA grounds the four areas in two agent-specific dimensions it defines explicitly: "action-space (or authority, capabilities)" — "range of actions the agent can take... determined by the tools it is allowed to use and permissions on those tools" — and "autonomy (or decision-making)" — "degree to which an agent can decide how to act towards a goal... determined by its instructions and level of human involvement." The framework treats these as levers organizations tune directly rather than abstract properties: an agent can be bound by "detailed instructions and SOP" or left to use "its own judgment," and its action-space can be narrowed through "access controls," "guardrails," and "human approvals." The rationale for the accountability pillar specifically is that "agent actions emerge dynamically and adaptively from interactions instead of fixed logic," which "diffuses accountability" across "multiple stakeholders" in the agent lifecycle, compounded by "automation bias" as a risk that grows "as humans supervise increasingly capable agents."
This gives a more granular governance vocabulary than Does AI risk increase with the autonomy we give it?, which treats autonomy as a single ceiling to cap; IMDA instead lets organizations bound risk by scoping tools and permissions (action-space) independently of how much judgment the agent exercises within that scope (autonomy), while reaching a similar practical conclusion that unmanaged autonomy is the main risk driver. IMDA's autonomy dimension also complements Can we design AI autonomy separately from capability? by naming what determines the level — instructions plus human involvement — rather than only enumerating the levels. And the "technical controls" pillar (access controls, guardrails, human approvals, logging) echoes the stack view in How do agent security layers connect across the stack?, naming controls distributed across the agent lifecycle rather than treating security as one checkpoint. A companion note, When should AI agents require human approval before acting?, details the accountability pillar's specific checkpoint logic.
The excerpt is a framework's stated method, not an evaluation: it offers no evidence on adoption rates, how often organizations actually implement the four areas, or whether multi-agent interactions make the recommended controls harder to apply once agents call other agents as tools. IMDA presents these as considerations for organizations to weigh, not as findings about what currently happens inside deployed agentic systems, so the framework should be read as a proposed structure for governance conversations rather than a description of current practice.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
When does governance become the bottleneck for agentic AI? How should authorization govern AI agent autonomy? Does expanding agent autonomy undermine human oversight capacity?Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does AI risk increase with the autonomy we give it?
Explores whether the risks posed by AI agents scale monotonically with the level of autonomy they're granted, and what the tradeoffs are between human control and agent independence.
IMDA's action-space/autonomy split offers a finer-grained way to bound the same risk this note treats as a single ceiling
-
Can we design AI autonomy separately from capability?
Explores whether an AI agent's autonomy—how much it operates without user involvement—is a design choice independent of what the agent can actually do. This matters because it opens autonomy to governance and calibration rather than treating it as fixed by capability.
IMDA names the two levers (instructions, human involvement) that determine where an agent sits on this scale
-
How do agent security layers connect across the stack?
Agent security is often treated as separate challenges at each layer—inputs, delegation, routing, containment. But do defenses at one layer fail if others aren't secured? This explores whether securing agents requires end-to-end integration.
IMDA's technical-controls pillar matches this stack view with concrete control types
-
When should AI agents require human approval before acting?
IMDA proposes four checkpoint triggers—high-stakes actions, irreversible decisions, outlier behavior, and user-defined boundaries—to decide when human oversight is necessary. Understanding when approval gates actually prevent harm matters for designing effective AI governance.
companion note detailing this framework's accountability pillar
-
Does granting agents more autonomy undermine human oversight?
Explores whether the design of autonomous AI systems—by giving agents greater independence—actually weakens the human overseer's ability to catch problems. Matters because oversight is a key safeguard against AI failures.
Qualifies IMDA's human-accountability control: agent autonomy itself erodes users' position and skill to exercise the oversight it assumes
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Model AI Governance Framework for Agentic AI (Version 1.5)
- AI Agents Push Humans Out of the Loop
- Governing AI Agents
- Explaining AI Agents Through Execution Traces
- Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems
- Practices for Governing Agentic AI Systems
- Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
Original note title
IMDA's agentic AI governance framework organizes oversight into four areas — risk bounding, human accountability, technical controls, end-user responsibility