Anthropic Gives Independent Auditors Inside Access to Its AI Safety Work

Anthropic has announced that it has started giving a team from METR, a nonprofit that evaluates frontier AI models, employee-like access to its systems and information. METR will publish the results of its risk assessments. The move follows a commitment made last month to stronger third-party assessment, and Anthropic's signing of the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators.

METR is not the only evaluator involved. Anthropic says it has worked with METR over the past year on several reviews of its Risk Reports, and is also working with SecureBio and Accenture. It is in early discussions with several other organisations, including Hugging Face.

Anthropic explains that it cannot publicly share everything about its safety work, because some of it could help malicious actors or is commercially sensitive, such as compute or training information. Instead, evaluators get deep access and share their evidence and reasoned conclusions about the risks. The aim is to close the gap between what evaluators understand about a frontier AI company's systems and what the company itself understands. Anthropic compares the approach to bank supervision, where regulators place resident examiners inside the largest banks. It says it will keep sharing updates as it learns what kinds of partnership work.