How an Outside Assessor Could Check Whether Unreleased AI Agents Might Go Rogue
AI developers and the public want to know whether AI agents would pursue very harmful actions, including unsanctioned cyberattacks. METR, a nonprofit that evaluates frontier AI, has described how an assessor could investigate this for a developer's near-frontier models. The aim would be to understand the worst misaligned actions those agents might take in plausible future situations. The assessor would need to query a fairly comprehensive set of internal transcripts, including from training, and have sampling access to internal model checkpoints.
The assessor would search logs for incidents and patterns of misaligned behaviour from training, evaluations and internal use over a defined period. The focus is on internal use, so activity in external customer data would not need to be assessed. Coverage would include all models that were near the internal frontier during that period, including experimental models and those still in training. The assessor would survey how often misaligned behaviour occurs and describe selected incidents in more detail, while also stating the limits of their search.
The assessment would also ask whether what has been observed reflects the worst actions the agents might take. That means weighing evidence from counterfactual simulations and evaluations, and considering whether the developer's training process might suppress signs of misalignment. The assessor would also be expected to discuss the limitations of the methods they use. This is an outline of the approach, and the source excerpt does not cover the full proposal.