Anthropic and Accenture Commit $1 Billion Each to Build an Embedded AI Evaluator Team

MakeCryippreferred on

• Anthropic and Accenture will each invest at least $1 billion over five years to build an embedded AI evaluation team
• Accenture’s specialist AI business Faculty will conduct independent evaluation, red-teaming, and alignment assessments of Anthropic’s models
• Embedded evaluators will have employee-level access to Anthropic’s operations, including monitoring models during training

Anthropic announced this week that it is partnering with Accenture, through Accenture’s specialist AI business Faculty, to build a team of embedded evaluators who will conduct independent assessment of Anthropic’s frontier AI models. Both companies expect to invest at least $1 billion over the next five years to build out the evaluation capacity the partnership requires.

The initiative fulfills a commitment Anthropic CEO Dario Amodei made to embed external evaluators directly within the company rather than relying solely on evaluations conducted at arm’s length. According to Anthropic, embedded evaluators will evaluate and red-team models, conduct alignment assessments, test model safeguards, monitor models during training, observe company decision-making processes, verify safety commitments, identify organizational blind spots, and report incidents as they arise.

The arrangement marks a departure from traditional external AI evaluation models, in which outside evaluators typically test a finished model against a defined set of benchmarks without ongoing access to the company’s internal processes. Embedded evaluators, by contrast, will have employee-level access to Anthropic’s operations, a structure intended to let them catch problems during model development and internal decision-making rather than only after a model has already been finalized and released.

Anthropic said the partnership is non-exclusive, and the company plans to continue working with other evaluators including METR, a nonprofit focused on evaluating frontier AI systems’ capabilities and risks. Anthropic framed the distinction this way: “Independent embedded evaluators do not reduce our accountability, but help to make it more verifiable. The safety of our models remains our responsibility.”

The size of the financial commitment, a combined $2 billion over five years between the two companies, signals that both Anthropic and Accenture view embedded evaluation as a sustained operational function rather than a one-time audit, though neither company has disclosed how the evaluation team’s findings will be reported publicly or how disputes between evaluators and Anthropic over a model’s safety would be resolved.

The partnership follows a period of heightened scrutiny of frontier AI labs’ internal safety practices industry-wide, with outside researchers and former employees at multiple companies raising questions in the past year about how rigorously safety testing is conducted ahead of major model releases. Embedding evaluators with the kind of internal access Anthropic has described is a more structurally invasive form of oversight than the benchmark-based external audits most labs have relied on to date, and its effectiveness will likely be judged over time by whether it surfaces problems that a purely external evaluation process would have missed.