- Anthropic and Accenture will each invest at least $1 billion across five years into embedded evaluation.
- The program is intended to place external evaluators close to model development so risks can be found earlier in the cycle.
- The Accenture engagement will be executed through Faculty, Accenture’s specialist AI business.
- Evaluators may receive access similar to employees, including observing models during training and reviewing decisions related to development and deployment.
- Work will include red teaming, alignment assessments and safeguards testing for Claude and future models.
- Anthropic retains ultimate responsibility for model safety and deployment decisions despite the embedded evaluator role.
- The Accenture agreement is non‑exclusive; Anthropic is also in talks with nonprofit evaluators (e.g., METR) and plans to announce additional partners.
- Anthropic suggested pooled funding or government‑backed mechanisms as possible future options to reduce evaluator financial dependence on model developers.
- The program may change where independent testing occurs in the development timeline, bringing scrutiny earlier.
Useful?