Anthropic embeds METR evaluators with employee-like access under three-tier audit plan

Anthropic embeds METR as its first deep-access evaluator

Anthropic has started giving a team from METR, a nonprofit that evaluates frontier AI models, access to its internal systems that comes close to what its own employees have. In an announcement surfaced on Hacker News, the company said the step follows commitments made a month earlier, when it pledged stronger third-party assessment of its models and safety practices and signed the White House Accord on Super Intelligence, under which participating companies committed to partnering with independent external auditors or evaluators.

METR will publish the results of its risk assessments, and Anthropic says it cannot veto that publication.

What "employee-like access" means in practice

Six METR employees have been set up with Anthropic hardware, office space and internal tooling that Anthropic describes as roughly on par with what its own risk assessment teams use. Staff are encouraged to share information with them as they would with a colleague working on alignment risk, apart from a small set of clearly defined exclusions: customer data, attorney-client privileged material and details about Anthropic's model architectures.

Anthropic can still require redactions, but only within narrow, pre-defined categories. METR is free to publicly characterize those redactions and to say whether it believes they were justified, as long as the redacted information itself stays hidden. The engagement is confined to a small METR team operating under enforceable agreements, with other METR staff, including researchers collaborating with Anthropic on separate projects, siloed from the work.

The rationale, and a banking analogy

Anthropic argues that as its models have become more powerful, the accompanying risk questions have outgrown what it can discuss publicly: some details could assist malicious actors, while others, such as compute and training information, are commercially sensitive. Deep access for independent evaluators who publish evidence and reasoned conclusions is its proposed middle path, with the stated goal of narrowing the gap between what an evaluator understands about a frontier AI company's systems and what the company itself understands.

The company likens the arrangement to bank supervision, in which regulators station resident examiners inside the largest banks because those banks know their own risk positions better than outsiders. Anthropic lists four requirements for evaluators: expertise in frontier-model capabilities and in the ways evaluations can go wrong, such as alignment faking; independence, meaning no equity in the company being evaluated and no financial or operational dependence on it; enough access to meaningfully assess risks; and freedom to publish.

A three-tier access ladder

Anthropic is formalizing the program as three tiers. First-tier evaluators conduct narrowly scoped investigations, with Anthropic deciding what access to grant and findings published only with its approval. Second-tier evaluators can publish without approval and see more information, though Anthropic can still require removal of specific categories, which evaluators may then discuss publicly. The third tier is the METR arrangement: employee-like access paired with second-tier publication rights.

SecureBio currently sits in the first tier, where it is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report, having already completed and published a pilot review of the previous report. Anthropic says it will consider expanded access and publication rights for SecureBio once that review is complete.

With Accenture and Faculty, Anthropic says it is working toward a broader partnership covering model evaluation and red-teaming, alignment assessments, safeguard testing and capability building across the organizations. Early discussions are also under way with several other parties, including Hugging Face. Because these partnerships are younger, the evaluators involved receive more limited information and hold more limited publication rights; Anthropic frames this as a trust-building stage before full embedding, and says it ultimately wants several evaluators at the third tier, each publishing independent analyses of its risk posture and key decisions.

Why it matters

External evaluation of frontier models has always been hampered by an information asymmetry: labs know far more about their systems than the researchers assessing them. The resident-examiner model is a serious attempt to close that gap, and Anthropic is explicitly calling on every frontier AI company to adopt it voluntarily. The test will be concrete rather than procedural: how much of METR's analysis survives the redaction categories, whether independence holds across repeated assessments, and whether other labs and Accord signatories follow. Until the first embedded evaluations are published, the framework's ambition is easier to judge than its teeth.