Skip to content

Anthropic embeds six external evaluators inside the company: METR will publish its findings, with no veto

Published on 15 October 2026

Mesa de oficina con un portátil cerrado y una tarjeta de acceso, y al otro lado de un cristal esmerilado la silueta de una persona de espaldas mirando una sala de servidores.

Anthropic announced on 15 October 2026 that six people from METR, a non-profit that evaluates frontier AI models, are now working inside the company with corporate hardware, desk space and access to internal tools and systems at a level comparable to that of its own risk assessment teams. The line that sums it up: METR will publish its analysis of the company's risk levels, incidents and practices, and Anthropic cannot block that publication — it can only require that a handful of pre-defined categories of information be removed.

What was announced

The arrangement has its own label in the announcement: employee-like access. In practice it means that, outside a few narrow cases, Anthropic staff may share information with that team exactly as they would with a colleague working on alignment risk. Three things fall outside: anything touching customer data, anything covered by attorney–client privilege, and anything about model architectures.

On publication, the balance of power is written down. The company gives up any right of veto and keeps only the right to ask for the removal of passages falling into those narrow categories. METR may say publicly that redactions happened and whether it considers them justified, provided it does so without revealing what was taken out.

The arrangement is bounded by enforceable agreements and covers a small group. The rest of METR's staff is walled off from that team's work, including researchers collaborating with Anthropic on other projects. The company insists this is not a route for another organisation to obtain its information, or that of its customers and partners: it is a few qualified people operating under its own security standards.

The announcement also sketches a three-rung ladder of access. On the first rung, the evaluator runs narrowly scoped investigations, Anthropic decides what it shows, and publication requires sign-off. On the second, that prior approval disappears and the available information grows, although the company can still require removals, with the same right to comment publicly on them. The third rung is precisely the METR set-up: employee-like access and publication rights like those of the second rung. The stated goal is to have several teams at that level eventually, each publishing its own reading of the company's risk posture.

For now, SecureBio sits on the first rung and is reviewing the chemical and biological weapons chapter of the August Risk Report, having already completed and published a pilot review of the previous one. Once that is done, Anthropic says it will consider granting wider access and more publication latitude. With Accenture and Faculty it is working towards a broader arrangement covering model evaluation and red-teaming, testing of safeguards, alignment reviews and capability building on both sides. Early conversations are under way with other organisations, Hugging Face among them.

Where this comes from

According to the announcement, the company had committed the previous month to stronger external assessment of its models and safety practices, and had signed the White House Accord on Super Intelligence, under which signatories pledged to work with independent auditors or evaluators. With METR there was already a track record: over the past year it has taken part in several reviews of the Risk Reports.

The reasoning Anthropic gives is information asymmetry. The more capable the models, the harder they are to assess, and much of the relevant material cannot be made public: some of it would help malicious actors, some is commercially sensitive, such as compute or training details. Its proposed way out is to show that material to third parties and let them publish evidence and reasoned conclusions.

The comparison the company reaches for is bank supervision: because a bank understands its own exposure better than any outsider, regulators place examiners inside the largest institutions to spot risks to the public in advance. Anthropic concedes the analogy is not exact.

The announcement also arrives during a run of institutional moves by the company, including the rewrite of its Usage Policy, which takes effect on 12 November, and the memorandum signed with the Government of Singapore, which already covered model evaluation and red-teaming exercises.

What it means

For anyone building or maintaining software on top of these models, the change is one of sources. Until now, almost everything known about a frontier model's risk profile was signed by whoever trained it. If this works, within months there will be documents about the same systems carrying a different signature, a different methodology and the explicit option of contradicting the vendor.

That has one immediate practical use: supplier assessment. In a security questionnaire or an AI vendor due diligence exercise, it is now possible to ask something sharper than "do you run audits?": what level of access third parties get, whether they can publish without permission, and whether they can say publicly when something was withheld. It is a new bar, and it can be checked.

There is a second detail that matters to anyone sending customer data through these APIs: customer information is listed among the categories kept outside the evaluators' perimeter. Opening the house to an external auditor does not mean opening the data other people have left inside it. Worth having clear before someone asks in a security committee.

What the announcement does not say

There is no date for METR's first public report, no publication schedule and no term for the agreement. Nor is it spelled out what "internal tools and systems" covers in practice, or how the embedded team's work is funded: the text requires evaluators to hold no equity and to be neither financially nor operationally dependent, but does not explain how the relationship is paid for.

Model architectures stay out of reach unless explicit internal approval is granted. The company's argument — high risk of inadvertent leaks, usually low relevance to risk assessment — is its own, not a third party's, and Anthropic itself decides when a case is relevant enough to make an exception. The announcement also leaves open who arbitrates whether a given redaction genuinely fits the agreed categories, beyond METR's ability to disagree in public.

And the big one: all of this is voluntary. There is no legal obligation behind it, the company says every frontier AI firm should do the same, and nothing in the announcement describes what happens if an evaluator concludes the risk is unacceptable. The power to publish is not the power to stop anything.

Source: Anthropic

Did reading this raise a question?

Ask us. We answer even if you never become a client.