Outside reviewers, inside the lab
Anthropic and Accenture announced an embedded AI evaluation partnership on September 18. Faculty, Accenture’s specialist AI business, will lead the work, which includes testing models, deliberately probing for failures and examining safeguards. Anthropic will pay Accenture directly for the work.
The important change is access. Anthropic’s announcement describes reviewers working with employee-like access to development, training decisions and staff. That would let them examine how a system was built, rather than only test a finished model selected for release.
According to Accenture, each company expects to invest at least $1 billion in AI safety over five years. These are forward-looking commitments, not spending already completed or a published fee for one audit.
The funding arrangement deserves scrutiny
For the funding arrangement to provide credible scrutiny, the reviewers need meaningful access and the ability to report findings that their client may dislike. The partnership is non-exclusive, and Anthropic is also discussing evaluation work with nonprofits.
Dario Amodei’s earlier proposal called for external reviewers to publish findings without Anthropic’s editorial control, subject to narrow restrictions on sensitive information. The partnership announcement nevertheless says operational details remain under development, with no shared standards yet governing access or reporting. The proposal and the final operating rules should not be treated as interchangeable.
This follows Amodei’s call to slow the AI race, but it is not a training pause. Anthropic intends to keep releasing frontier models. With Claude taking a growing role in Anthropic’s own research, scrutiny of how models are developed matters alongside tests of what they can do.
The conversation starts here
Sign in with a supporter account to comment. Sign in
Nobody has commented yet. Want to go first?