Cybersecurity Research

How Auditors Could Test Whether AI Agent Monitoring Actually Works

METR · 14 Oct 2026
Key Takeaway If your business deploys AI agents with broad access such as internet use or credentials, confirm that their activity is logged and monitored, and limit their permissions to what each task requires.

METR, a nonprofit that evaluates frontier AI, has described how an outside assessor could investigate "control" of internal AI agents. Control assessments ask whether an AI developer's agents could cause serious harm without being stopped by the developer's security and oversight measures. Today, developers rely heavily on monitors to catch potentially harmful AI behaviour. METR currently considers five core sub-questions, and says it will update them as new capabilities and misaligned behaviours emerge.

The proposal focuses on monitoring AI agents with other language models. Standard security measures, such as limiting an agent's permissions through sandboxing, may be enough when an agent only needs a small, known set of actions. But many agents need broad permissions. An agent with internet access can take an enormous range of actions, and spotting a harmful one often requires understanding its wider context. For that reason, language model monitoring is described as a critical part of automated oversight at scale and the main source of evidence inside frontier developers about AI misbehaviour.

One key question is coverage, because a monitor only helps if it is running on the relevant activity. The assessor would estimate what fraction of internal agent activity is actually monitored, weighting by risk. Agents in highly secure sandboxes pose less risk than those with internet access, internal credentials or infrastructure access. Doing this well requires inference logs, monitor logs to cross-check them, and employee interviews to make sure no sources of activity are missed.

AI agents AI monitoring AI safety agentic AI oversight
Running AI agents? Eight controls for securing them, mapped to the ISM ->

Summarised by CISO AI from METR, written with Claude Sonnet 5.5. We link back to every original so you can read it yourself.