Multi-stage AI-agent control: authority, accountability and independent oversight

A green–amber–red scale is convenient for an executive dashboard. As an architecture for controlling AI agents, it is too coarse.

One “amber” agent may only read enquiries and prepare a draft. Another may change the CRM, send messages and create commitments to customers. A third may inspect other agents and stop the workflow. They have different authority, failure consequences and roles in the control system even though their colour is nominally the same.

The central problem with a three-colour scale is that it collapses seven different questions:

1. What can the agent technically do?
2. How severe is the worst credible outcome?
3. How many control levels must stand between the worker and execution?
4. At which stages—from goal admission to recovery—is the action checked?
5. How independent are the worker, verifiers and emergency path?
6. Who is accountable for the rule, decision, execution and incident?
7. Which operating mode is the system in now?

These questions cannot be compressed into one axis without losing material information. We need neither a ladder of autonomy nor a traffic light of trust, but a multidimensional control system distributed across levels and stages.

Colour should show the current state of the system, not a permanent “trust level” assigned to an agent.

This article proposes two connected instruments. The first is a six-coordinate system card:

A — authority
C — consequence criticality
S — supervision depth
I — independence of control paths
R — accountability allocation
M — current operating mode

The notation A3 · C2 · S2 · I1 · R2 · M1 says far more than “amber”. It indicates that the agent may make bounded internal changes; a failure has moderate consequences; the action receives an automated pre-action check; the controller is not yet fully independent; a named decision owner accepts the bounded delegation; and the workflow is in a supervised pilot.

The second instrument is an implementation matrix: six control levels L0–L5 × seven stages T0–T6. The card specifies the required control strength. The matrix shows where the controls act. This is the article's central proposition: one final approval cannot replace a continuous chain of barriers.

This is a Methodfield operating model, not a certification standard and not a guarantee of legal compliance. Its purpose is to turn a vague conversation about “trusting AI” into an inspectable architecture of decisions, access, control and recovery.

What avionics actually teaches us

The useful analogy with avionics is not that a business agent should be certified like an aircraft. It is that safety-critical engineering separates hazard severity, system function, control independence, degraded modes and human intervention.

EASA's AI Roadmap 2.0 (opens in a new tab) and AI Concept Paper Issue 2 (opens in a new tab) describe a progression from human assistance towards more advanced human–AI teaming, while tying assurance objectives to the criticality of the underlying function. The lesson is not “copy six aviation levels”. It is “derive control from the hazard, then prove that the architecture can detect, contain and recover from failure”.

Five principles follow for business agents.

1. Hazard comes before automation level

In aviation safety assessment, a function is analysed by the consequences of failure before assurance objectives are assigned. For an AI agent, begin with the worst credible consequence, not with a claim such as “the model is 95% accurate”.

The same model error has different significance in a meeting summary, a price sent to a customer, an access-right change and a safety-relevant physical command.

2. Redundancy only helps when it is sufficiently independent

NASA safety requirements explicitly warn about common-cause failure in redundant functions. Three identical agents using the same model, system prompt, source and credentials may be three copies of the same weakness, not three independent barriers.

The relevant question is not “how many agents?”, but “how many genuinely independent units of evidence, policy and stopping power?”

3. A safe mode matters more than a Stop button

Stopping a run is not enough. The system must know which state to enter: retain a draft, block sending, revoke a temporary token, compensate a reservation, freeze a queue, or transfer an evidence package to a person.

Article 14 of the EU AI Act (opens in a new tab) includes the ability to intervene, interrupt and bring a high-risk AI system to a safe state. Article 15 also refers to technical redundancy, backup and fail-safe plans as possible robustness measures. Not every enterprise agent is legally a high-risk system, but the engineering principle applies much more widely.

4. Control continues after deployment

Before launch, teams test expected scenarios. After launch, new data, attacks, model updates, API changes, policy changes and user behaviours appear.

The NIST Generative AI Profile (opens in a new tab) connects risk management with independent evaluation, ongoing monitoring, incident response, recovery and change management. A control level is not a launch-time sticker; it is a maintained property of the system.

5. A human is part of the system, not a universal fuse

Human presence at the end of a workflow does not create meaningful control. The person must receive the right evidence, have time and competence to decide, see the exact action that will be executed, and be technically able to reject or reverse it.

If an operator approves hundreds of similar requests, sees only the worker agent's summary and remains accountable for details hidden from view, that is not meaningful human-in-the-loop control. It is risk transfer with automation bias and approval fatigue added.

Two axes of control: levels L0–L5 and stages T0–T6

Multi-level and multi-stage control are different dimensions.

• A level identifies the kind of barrier making a decision and its independence from the worker.
• A stage identifies the moment in the trajectory at which that barrier can see the risk and intervene.

Levels are cumulative. An AI controller does not replace schemas, allowlists or transaction limits. Human approval does not replace pre-flight checks. Post-action review does not replace the right to block execution.

AI agents are particularly useful at L2 and L3. They can read context, compare evidence, find semantic contradictions and observe a long action trajectory. That is exactly why they should be control agents, not secondary executors. Their power is asymmetric: stop, narrow, request evidence or escalate—but never expand scope or perform a disputed action themselves.

Control must cover the whole trajectory:

A robust design places different barriers in different cells of this matrix. For C3, one AI judge at T3 is insufficient: use at least a deterministic L1 barrier, an independent L2 controller, separate execution at T4 and a watchdog across T5–T6. C4 adds L4. L5 checks that the whole arrangement has not degraded after a model, data or policy change.

Not every barrier should be an AI agent

A multi-level system does not mean “put an agent at every level”. Strong architecture uses the simplest mechanism that can reliably solve each problem. A useful selection order is:

1. Deterministic maths and ordinary code when the rule can be expressed exactly.
2. Classical ML when the task is a stable probabilistic prediction or pattern-recognition problem with measurable labels.
3. An AI agent when the task requires language interpretation, ambiguous context, adaptive planning and tool orchestration.
4. A human when the decision contains a value judgement, legal or fiduciary accountability, a policy exception, or a severe irreversible consequence.

A simple test helps. If a requirement sounds like a formula or finite-state machine, do not replace it with an LLM verdict. If the output is a stable score over repeated signals, test classical ML first. An agent is justified when the alternative would require a person to absorb substantial ambiguous context and adaptively choose the next tool.

A hybrid chain is usually stronger than any monoculture. The worker agent may understand a request; a calculator computes the exact amount; a rule engine checks the limits; an ML model flags an anomaly; an independent AI controller checks meaning and evidence; and a person decides the exception. Each component performs the job to which it is best suited, and failures do not share one mechanism.

First coordinate: six authority levels A0–A5

Authority describes neither model intelligence nor answer quality. It describes the maximum action that the technical environment allows an agent to request or perform.

Assign authority per resource, tool and intent—not a single number to the whole agent. The same agent may have A1 over a mailbox, A2 over proposals, A3 over selected CRM fields and no access at all to billing.

Authority must be bound to intent

A safe identity answers more than “which service account called the API?” It binds:

• human or system initiator;
• business intent and case identifier;
• allowed data, tools and objects;
• recipient, amount, geography and time window;
• maximum number and sequence of operations;
• expiry and revocation conditions;
• evidence and approval required before execution.

NIST's NCCoE concept paper on agent identity and authorisation (opens in a new tab) uses the idea of a permission envelope: an agent's identity is constrained to a specific purpose and its actions are traceable back to the authorising person or process.

If a delegated sub-agent receives broader credentials than the parent, or the tool can turn “edit one record” into “modify the database”, the authority model exists only on paper.

Second coordinate: consequence criticality C0–C5

Criticality measures the consequence of a wrong, compromised or untimely action—not how impressive the agent is.

What raises criticality

Evaluate at least:

• financial or contractual impact;
• privacy, confidentiality and security;
• rights, discrimination and legal duties;
• physical safety;
• scale and speed of propagation;
• reversibility and cost of recovery;
• detectability before and after execution;
• time available for intervention;
• dependency of other agents and systems on the result;
• risk of the action becoming precedent, memory or training data.

The class follows the worst credible consequence, not the average outcome. A thousand harmless runs do not lower the class of one irreversible action.

Risk scores may prioritise queues, but they must not hide a hard stop. A low combined score must not authorise an action that violates an access rule, statutory requirement or non-negotiable business invariant.

Third coordinate: supervision depth S0–S5

Supervision describes the minimum control package required before, during and after execution.

The levels are cumulative. S3 does not remove logs, schemas, limits or allowlists from S0–S2. “An AI controller checks it” is not a complete description unless the technical right to stop execution is also specified.

A control agent needs asymmetric rights

A supervisory agent may:

• read the intent, policy, primary evidence and exact proposed action;
• request missing evidence;
• compare the action with limits and precedents;
• veto, quarantine or escalate;
• recommend a lower operating mode;
• produce a structured decision with reason codes.

It must not:

• expand the worker's authority;
• rewrite policy during a run;
• use the worker's write credentials;
• execute the disputed action itself;
• alter logs or evidence;
• silently convert an exception into a new general rule.

Control authority is primarily the power to stop, not the power to do more.

AI checking helps, but is not proof

Greenblatt and colleagues' AI Control (opens in a new tab) shows the value of combining monitoring, untrusted models, trusted checks and protocol-level safeguards. OpenAI's work on prompt-injection-resistant agents (opens in a new tab) similarly uses layered safeguards and alignment checks around tool calls.

These approaches support AI-based monitoring. They do not show that one LLM can certify another. A checker is strongest when it verifies structured claims, primary sources, exact parameters, policy and state transitions. It is weaker when asked to judge a vague notion of “correctness” from a persuasive narrative written by the worker.

Fourth coordinate: independence I0–I3

The number of checkers is not the depth of defence. Independence must be designed and tested.

Apple's 2026 study Nine Judges, Two Effective Votes (opens in a new tab) found that a panel of nine frontier models from seven families provided roughly two independent units of information on the studied natural-language-inference sets because the models often failed on the same examples. This is not a universal estimate of every panel, but it is a strong warning against “add more judges and obtain reliability”.

A separate Scientific Reports experiment (opens in a new tab) found that, in the tested multi-agent-debate design, one persuasive malicious participant could reduce group accuracy and create false consensus. Free agent discussion is not equivalent to independent verification.

Reducing common-mode failure

For consequential actions, separate:

• instructions for worker and controller;
• the controller's initial blind assessment from the worker's conclusion;
• primary evidence from self-reported reasoning;
• deterministic typed checks from semantic AI review;
• role identities and write credentials;
• policy and logs from worker modification;
• approval from execution through an exact-action, short-lived token;
• trajectory monitoring from final-response review;
• failure tests across shared poisoned sources, memory poisoning, prompt injection and collusion.

Role names such as planner, critic and judge do not create independence by themselves.

Three copies of one error are not redundancy. Different methods, sources, authority and failure paths create defence.

Fifth coordinate: accountability allocation R0–R5

An AI agent may receive an operational function, but it cannot absorb moral, legal or fiduciary accountability. Accountability stays with the people and organisation that selected the goal, policy, authority and admission conditions.

R does not measure “how guilty the agent is”. It measures how explicitly the human and organisational accountability path is assigned.

One action may have several owners: the initiator owns the goal; the workflow owner owns the eligible class of automation; the policy owner owns the rule; the decision owner owns an exception; and the incident owner owns containment and recovery. Record them separately, or the system creates an orphan action: everyone participated, but nobody owns the outcome.

The practical rule is strict: if the R level, decision owner and incident owner cannot be named, the agent must not receive authority for external execution.

Sixth coordinate: operating mode M0–M4

The first five coordinates describe the designed system. The sixth shows its current condition.

This is where traffic lights become useful again:

• green — M2: all mandatory control paths are healthy and the action remains inside the envelope;
• amber — M1 or M3: stronger review is required or the system is degraded;
• red — M4: execution is stopped and authority revoked.

Colour becomes an indicator of current state, not a substitute for risk assessment.

How multi-level control operates across stages

A practical control plane combines serial interlocks with parallel checks. The worker forms intent and plan; a deterministic barrier checks exact constraints; one or more independent AI controllers assess meaning and evidence; a risk coordinator reconciles their decisions; a separate executor performs only the unchanged authorised operation; and a watchdog observes both outcome and trajectory. Human and L5 governance surround the path and enter when C, R or degraded mode requires them.

A useful text representation is:

goal + initiator
      ↓
worker AI agent ── proposed plan and exact action
      ↓
deterministic policy gate ── authority, schema, limits, state
      ↓
independent AI controller(s) ── meaning, evidence, trajectory
      ↓
risk coordinator ── release | hold | human review | safe stop
      ↓
separate execution broker ── exact-action token and commit
      ↓
outcome verification + watchdog + recovery

1. Worker agent

The worker interprets the task, forms a plan, calls read tools and produces a structured action proposal. It does not possess a universal production key.

2. Deterministic policy gate

The gate checks what should not depend on an LLM judgement: schemas, types, allowlists, field rights, value and volume limits, ordering, identity, token expiry and action hash.

A gate should be fail-closed. If policy cannot be loaded or the proposal is incomplete, execution does not proceed.

3. Independent AI controller

The controller checks what rules alone cannot capture:

• whether the action answers the original intent;
• whether cited evidence supports the claim;
• whether important contradictory evidence is omitted;
• whether an untrusted source appears to have changed the agent's objective;
• whether the sequence is suspicious even when each step is individually permitted;
• whether a safer action can achieve the same business result.

Its output should be structured: allow, deny, escalate or need_evidence, with reason codes, confidence boundaries and references to evidence.

4. Risk coordinator

The coordinator is not another free-form judge. It applies a routing policy to C, S, I, R, controller disagreement, operating mode and current limits. It may release a known low-risk class, hold, route to a named person or trigger safe stop.

5. Execution broker

The component that changes an external system should be as simple as possible. It receives an immutable, transaction-bound authorisation for one exact action. It cannot improvise, ask an LLM what to do, or reuse the token for another recipient or amount.

6. Watchdog and recovery controller

A separate path observes the full trajectory, not just the final answer. It can detect loops, tool-call growth, repeated recipients, limit accumulation, unexpected data flow, model or policy change, and mismatch between approved and actual outcomes.

Its rights include lowering the mode to M3 or M4, revoking temporary credentials and preparing an incident package. They do not include rewriting history or silently repairing and executing the disputed business action.

This resembles runtime assurance in avionics: an advanced component operates while monitors and a safety mechanism can transfer the system to a simpler, trusted mode. In business workflows, that mode often means draft-only, queue and manual execution—not “a different model continues doing everything”.

Choosing the required supervision

A baseline mapping is:

Raise the level when:

• one session combines untrusted input, sensitive systems or data, and external communication or state change;
• an action crosses systems or agents;
• long-term memory can be poisoned;
• failure is hard to detect or recovery is untested;
• one run can scale across many objects;
• controller and worker share model, context and evidence path;
• checker quality is unmeasured on labelled and adversarial sets;
• workflow owner, emergency contact or manual fallback is undefined.

The first condition reflects Meta's Agents Rule of Two (opens in a new tab): until prompt injection is reliably addressed, an agent should not autonomously combine all three properties—untrusted input, sensitive access, and the ability to change state or communicate externally.

The OWASP Top 10 for Agentic Applications 2026 (opens in a new tab) adds behaviour hijacking, tool misuse, identity and privilege abuse, insecure inter-agent interaction and cascading failure. Controlling only the final button does not protect the trajectory.

Example: an agent prepares and sends a client proposal

The initial idea sounds simple:

The agent reads an email, determines the need, calculates a price, replies to the customer and updates the CRM.

That sentence hides several authority levels.

Decompose the workflow

1. A1: read a new enquiry from an approved mailbox.
2. A2: extract requirements and prepare a draft.
3. A2: request a calculation from approved rules.
4. A4: send a message to an external recipient.
5. A3: update selected CRM fields.

For a standard proposal inside the price list, the worst credible consequence may be C3: a wrong price or promise affects the customer and the company's reputation.

Where agent, maths and ML belong

• The AI agent extracts intent from language, resolves ambiguous requirements and drafts the message.
• A deterministic pricing service—not the LLM—calculates price, tax, currency, rounding and validity period.
• A classical ML model may produce an anomaly score if sufficient labelled proposal history exists; without such data, do not imitate that signal with an agent.
• An independent AI controller checks semantics: whether the promise matches the enquiry, contract and actual company capability.
• The rule engine and execution broker check exact limits and send only the approved content hash.
• A person decides a non-standard discount, new contract term or other policy exception.

Control path

1. The worker creates a structured proposal and cites the price sources.
2. A deterministic gate checks currency, price band, required fields, recipient and validity.
3. An independent controller compares the enquiry, source price list and exact draft without relying on the author's explanation.
4. A standard case inside narrow limits can be released only after a proven pilot; a discount, exception or non-standard promise goes to a person.
5. The person sees the exact message, recipient, price, sources, deviations and consequences—not a generic “Approve?”.
6. The execution broker sends exactly the approved version.
7. The CRM update is idempotent and reconciled with the sent proposal.
8. The watchdog blocks duplicate send, cumulative limit breach and unexpected tools.

The pilot card may read:

Workflow: standard client proposal
Authority: A4 Commit
Criticality: C3 Material
Supervision: S3 Independently controlled
Independence: I2 Diverse control
Accountability: R2 Delegated decision owner
Mode: M1 Supervised pilot

Control coverage: L0–L3 across T1–T6; L4 on exception; L5 assurance

Automatic envelope:
- only approved mailbox and CRM fields;
- approved price list version 12;
- no discounts or custom legal terms;
- one recipient from the originating thread;
- one send attempt;
- exact-content hash before execution;
- manual fallback within 30 minutes.

After evidence accumulates, some standard cases may move to M2. A wider price band, new customer segment, model change or additional tool does not inherit that permission automatically.

Who controls the control agents?

A control agent is itself an AI system and needs its own assessment:

• named owner and versioned policy;
• a test set with known failures and ambiguous cases;
• false-negative and false-positive measurement, not only “accuracy”;
• adversarial tests with persuasive but wrong justifications;
• prompt-injection tests from documents and inter-agent messages;
• periodic blind human samples;
• no right to alter evidence, logs or its own configuration;
• decision timeout and safe behaviour when unavailable;
• automatic mode reduction when control quality or independence is unproven.

If a required controller is unavailable, the correct fallback for C3 is not “let the worker continue alone”. It is M3: draft, queue or human review.

Metrics for controllability

Successful runs alone say little about safety. Measure each dimension.

Authority

• rejected requests outside the permission envelope;
• use of temporary versus standing credentials;
• actions with complete initiator → intent → decision → result linkage;
• tool calls and affected objects per business outcome.

Control

• controller recall on known hazardous actions;
• false-positive rate and manual handling time;
• worker–controller disagreement;
• actions routed to human review;
• human override rate and later outcome;
• approval latency and signs of approval fatigue.

Independence

• errors missed by all barriers;
• propagation of one poisoned source or injection across agents;
• shared dependencies among executor, verifier and watchdog;
• failure tests for model, provider, memory, policy store and execution broker.

Accountability

• actions with named workflow, policy, decision and incident owners;
• approvals where a person saw the exact executable action;
• owner response time to escalation and incident;
• overrides without a documented reason;
• model and policy changes with named approver and rollback owner.

Recovery

• time to detection and authority revocation;
• time to safe state;
• automatically reversible actions;
• partially completed processes;
• average manual recovery time;
• incident-response exercise success.

Calibrate controller metrics on a representative gold set and re-evaluate after changes. A model's self-assessment is not independent validation.

Mistakes that imitate multi-level control

“If the worker is AI, every controller should also be an agent”

No. Formulae, schemas, limits, access rights and state transitions are more reliable as deterministic checks. A specialised ML model may be better at a statistical anomaly. An AI controller belongs where risk is hidden in meaning, context or sequence. Otherwise a complex probabilistic component replaces a simple inspectable barrier and creates a new common-mode failure.

“We have five agents, therefore the system is safer”

Five dependent agents can produce a more persuasive error and false consensus. Count independent barriers and shared failure causes, not boxes in a diagram.

“The checker read the worker's reasoning”

This may expose some errors, but it also anchors the checker in the worker's framing and opens a persuasion channel. Give the checker the task, primary evidence and exact action first. The worker's explanation can be a later, secondary signal.

“Human approval closes high risk”

Not if the person sees a different version, cannot understand the consequence, cannot verify sources or must approve too many actions.

“The policy is in the system prompt”

A prompt is an instruction, not an access-control mechanism. Enforce critical rules outside the model with credentials, schemas, allowlists, state machines, limits and a separate execution path.

“The controller can repair and execute”

This destroys separation of duties. A controller that detects an error may recommend a patch, but the amended action must re-enter the control chain.

“Everything is in the logs”

Logs help only when they are immutable, complete, connected to intent and reviewed. Logging after irreversible execution is evidence, not prevention.

“We assessed risk at launch”

Model, prompts, memory, data, tool API and business limits all change. Material change requires new admission evidence and may force M1 or M3.

Implementing the model without aviation bureaucracy

A first production control path can be built in ten steps.

1. Decompose the workflow into atomic actions. Not “handle a customer”, but read a message, extract data, calculate, send and update CRM.
2. Choose a mechanism for each action. Maths and code for exact rules, ML for a calibratable pattern, an agent for ambiguous context, and a person for an accountable exception.
3. Assign an A level to every action. Inspect real API scopes and credentials, not only prompt descriptions.
4. Assign a C class. Record worst credible consequence, scale, reversibility, detectability and intervention window.
5. Fill the L × T matrix. Mark the barrier at every stage from policy change to recovery; find empty stages and shared single points of failure.
6. Choose the minimum S level. Start with deterministic constraints, then AI review and only then additional approval roles.
7. Test the I level. Map shared models, data, instructions, identities, owners and infrastructure.
8. Assign R and M transitions. Name workflow, policy, decision and incident owners; define degraded and safe-stop triggers and who may restore service.
9. Run shadow and supervised pilot. Measure baseline, failures, disagreement and recovery time before expanding autonomy.
10. Reassess after changes and incidents. Higher authority requires evidence, not a feeling that the agent “has become smarter”.

Position

AI-agent governance should not resemble assigning a human employee a green, amber or red trust badge. An agent is a system component. Its ability to act is created by tools, credentials and architecture.

A workable model separates:

• authority — what is technically possible;
• criticality — what a failure can cause;
• supervision — which barriers precede and follow execution;
• independence — whether barriers can fail differently;
• accountability — who owns policy, decision and incident;
• mode — what the system may do now.

AI agents can control other AI agents, analyse trajectories, find contradictions and stop a run. That will be an important part of future operating systems. But AI control must be bounded, measured and supported by maths, ordinary code, calibrated ML, deterministic interlocks, segregation of duties, accountable humans where consequences are high, and a prepared safe state.

Reliability comes not from the number of agents or the colour of a status, but from an independent chain of barriers that can detect an error, stop the action and restore business state.

Sources and further reading

• European Union Aviation Safety Agency, Artificial Intelligence Roadmap 2.0 (opens in a new tab), 2023.
• European Union Aviation Safety Agency, AI Concept Paper Issue 2: Guidance for Level 1 & 2 machine-learning applications (opens in a new tab), March 2024.
• SAE International, ARP4761A: Guidelines for Conducting the Safety Assessment Process on Civil Aircraft, Systems, and Equipment (opens in a new tab), December 2023.
• NASA, NPR 8715.3B — General Safety Program Requirements (opens in a new tab), requirements on functional redundancy, common-cause failures and safe configuration.
• European Union, Regulation (EU) 2024/1689 — Artificial Intelligence Act (opens in a new tab), Articles 9, 14 and 15.
• NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (opens in a new tab), NIST AI 600-1, July 2024.
• NIST, Summary Analysis of Responses Regarding Security Considerations for AI Agents (opens in a new tab), NIST AI 800-5, May 2026.
• NIST NCCoE, Accelerating the Adoption of Software and AI Agent Identity and Authorization (opens in a new tab), draft concept paper, February 2026.
• OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026 (opens in a new tab), December 2025.
• Meta, Agents Rule of Two: A Practical Approach to AI Agent Security (opens in a new tab), October 2025.
• OpenAI, Designing AI agents to resist prompt injection (opens in a new tab), 2026.
• Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, Fabien Roger, AI Control: Improving Safety Despite Intentional Subversion (opens in a new tab), ICML 2024.
• Guneet Kohli, Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels (opens in a new tab), Apple Machine Learning Research, June 2026.
• Insaf Kraidia et al., When collaboration fails: persuasion driven adversarial influence in multi agent large language model debate (opens in a new tab), Scientific Reports, 2026.