Anthropic has partnered with nonprofits METR and SecureBio to provide independent oversight of its AI models, safety practices, and risk management.

Anthropic announced on Thursday that it is embedding independent evaluators within its organization to monitor its AI development. The company has granted six researchers from the nonprofit METR employee-level access to internal tools and hardware. This initiative is part of a three-tier evaluation system designed to increase transparency regarding the company's safety practices and risk management.

Tiered Evaluation Framework

Anthropic has established a three-tier system for external evaluators, with each level providing different degrees of access. METR, classified as a third-tier evaluator, receives the highest level of access, including company-issued laptops and office space. This allows them to conduct independent assessments of risks and incidents, with the right to publish their findings without Anthropic's veto power, subject only to narrow redactions for sensitive information. SecureBio, a first-tier evaluator, operates with more limited access. The organization is currently reviewing the chemical and biological weapons sections of Anthropic's August risk report. Anthropic noted that it is also in discussions with other organizations, such as Hugging Face, to join its evaluation program.

Commitment to Accountability

The move follows a proposal by Anthropic CEO Dario Amodei in September to embed outside evaluators in AI labs to improve safety and accountability. The initiative aligns with a voluntary accord signed by several major AI companies, including Google, Meta, and OpenAI, which encourages partnerships with independent auditors. Anthropic stated that it does not fund METR's work and emphasized that it aims to close the gap between internal and external understanding of its systems.