Outside Red Team Stress-Tests OpenAI's Internal AI Agent Monitoring and Finds Novel Weaknesses

From 21 July to 11 August, one METR employee and one Redwood Research employee ran a pilot red-team exercise against OpenAI's internal misalignment monitoring system. Their 61-page report went to OpenAI researchers and security staff. It focused on two questions: what fraction of agentic inference (AI agents doing work) is actually monitored, and whether agents could in principle disable or subvert the monitoring.

The testers had deep access. They could see OpenAI's internal codebase, monitor prompts and configuration details, and internal documentation, and they interviewed about a dozen researchers and engineers across monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the premises, and were accompanied while onsite.

They found a number of novel vulnerabilities in the asynchronous monitoring system as it stood on and before 21 July. METR stresses that OpenAI was expanding its monitoring during the exercise, so the findings describe the historical system, not its current coverage. The exercise was a pilot for giving outsiders deep access to internal infrastructure, so only a brief, high-level public post was agreed in advance. METR expects future rounds to offer more public transparency.