Anthropic announced that six researchers from the nonprofit evaluation organization METR have been given employee-level access to independently assess its models, safety practices and risk management. It is the first time a frontier AI lab has brought outside evaluators in almost as if they were internal staff, marking a shift in AI governance from "reading reports" to "being on site".
Key points:
- Full access: the METR team is issued Anthropic computers and office space, with internal-tool access comparable to Anthropic's own risk assessment team; Anthropic employees are encouraged to share information with them as they would with colleagues.
- Unrestricted publication: METR can independently publish its assessments of Anthropic's risks, incidents and practices, and Anthropic has no veto; it may only request redactions in narrow, predefined categories such as customer data, attorney-client privilege and model architecture, and METR can publicly comment on whether the redactions are reasonable.
- Modeled on bank supervision: Anthropic likens the arrangement to regulators stationing resident examiners at large banks. METR is the only top-tier evaluator in its three-tier evaluation system; bottom-tier evaluators (such as SecureBio) need approval before publishing, while the middle tier can publish on their own but with less information.
Background: the move fulfills a pledge in a September essay by Anthropic CEO Dario Amodei to embed outside evaluators in AI labs. OpenAI later pledged the same approach, and both companies, along with Google, Meta, Nvidia and xAI, have signed the White House independent-review agreement. With AI agents' unauthorized actions being disclosed frequently of late, calls for independent outside verification keep growing.
Source: read the original. This text is a summary, for reference only.
