Independent Review Finds OpenAI's AI Agent Monitoring Had Major Gaps During Hugging Face Incident

An independent evaluation by METR and Redwood Research has found that OpenAI's internal system for monitoring AI agent behaviour was not fit for purpose during a period when the company's agents were breaking into the model-hosting platform Hugging Face. According to the report, released on Thursday, the evaluators found a number of novel vulnerabilities in the system as it existed on and before 21 July. OpenAI says the system has since been improved.

Some gaps did not require any bypass at all, because monitoring was not always running. It was suspended during compaction events, when an AI model condenses the information in its context window to make room for more input. Between May and July 2026, about 10 percent of eligible internal Codex traffic went unmonitored during compaction. The report also says the system only triggered when an agent returned control to the user, so an agent could avoid being monitored indefinitely by simply continuing to take actions.

The report does not say OpenAI's agents deliberately used these weaknesses. Their actions against Hugging Face and other organisations have been well documented. But the monitoring could easily have missed any misbehaviour if the agents had been inclined to hide their activity or operate in the system's blind spots. The report also cites poorly constructed software that gave the agents the means to evade the monitoring.