Cybersecurity Research

How Reliable Are AI Safety Claims? Anthropic Sets Out How to Build Stronger Evaluations

Anthropic · 13 Nov 2026
Key Takeaway When choosing AI tools for your business, treat vendor claims of being "safer" with caution and ask how those safety results were tested and documented.

When an AI company says a new model is "safer than its predecessor", it usually means the model scored better on a set of tests, known as evals, that someone built. Anthropic says these safety evaluations shape major decisions, including whether a model is ready to deploy, which safeguards ship with it, and what safety research and training should focus on next. A safety claim is only as sound as the eval behind it.

A safety eval looks at whether a model's response to a request makes harm more or less likely, and how severe that harm could be. It does not measure specific capabilities. Harm can come from a user openly asking for something dangerous, such as self-harm instructions, or from an innocent request that carries a high risk of unintended harm, such as asking about a drug dose. Put simply, the eval asks whether the model did the right thing given a specific input.

Anthropic notes that these tests are hard to get right. Researchers have found that AI evals can be gamed, may not measure what they claim to, and are often too poorly documented to reproduce. Results can also shift depending on whether questions are multiple-choice or open-ended, or even how answer options are ordered. Anthropic compares this to psychology's replication problems a decade ago, which led to reforms such as preregistration and more rigorous statistics. Drawing on those reforms and its own experience, it shares recommendations for anyone who builds or relies on AI safety evaluations, using examples from its system cards. This summary covers only the opening of the post.

AI safety safety evaluations Anthropic AI governance
Building or buying AI systems? Governing them under ISO 42001 ->

Summarised by CISO AI from Anthropic, written with Claude Sonnet 5.5. We link back to every original so you can read it yourself.