When a crypto outlet runs an AI-safety exposé, I do not read the argument first. I count the load-bearing claims. This week's flash on Anthropic's evaluation process carried three sentences of substance and zero named sources — no researcher, no report, no design document, no publication date. Just a verdict: the safety framework has "design flaws" and "incentive problems." Three facts, zero anchors, one clear target.
I want to be precise about the source. Crypto Briefing is a crypto outlet, and crypto media holds an ideological stake in the decentralized-AI narrative — a narrative structurally hostile to large centralized labs. That does not make the criticism false. It makes it unfalsifiable. In my line of work, unfalsifiable is the same as unshipped. But the complaint points at a real fault line, and that fault line is older than Anthropic. It is the oldest vulnerability in my industry.
The flaw is self-assessment.
Anthropic's Responsible Scaling Policy, iterated from v1.0 in September 2023 to v2.0 in October 2024 and refreshed into 2025, gates model deployment on capability evaluations mapped to AI Safety Levels, ASL-1 through ASL-4+. The structure is clean. The problem is who holds the pen. The entity that builds the model also decides whether the model has crossed a threshold requiring heavier guardrails. The referee and the player share a payroll.
This is the house style of every frontier lab, not an Anthropic invention. OpenAI's Preparedness Framework, DeepMind's Frontier Safety Framework, Meta's near-silence — all describe the same architecture with different fonts. Anthropic's version is simply the most publicly documented, which means it absorbs the most scrutiny. Third parties — METR, the UK AI Safety Institute, Apollo Research — are invited to evaluate. Invited is the operative word: selective, episodic, non-binding. Good enough for a brochure. Not good enough for a threat model.
Now the teardown. The unnamed "design flaws" resolve into four failure modes I have watched in every unaudited smart contract since 2017: evaluator capture, coverage theater, unfalsifiable claims, and environmental sandbagging.
Evaluator capture — the entity that benefits from deployment controls the gate. Anthropic's revenue depends on shipping; its evaluation determines when it ships. This is governance 101, not AI science. In 2017 I found an integer overflow in a token sale contract that fifteen senior developers had already cleared — not because I was smarter, but because I was not in the room where they had agreed it was fine. Groupthink is a shared evaluation signature. The same conflict sank exchanges that self-audited reserves before a withdrawal freeze.
Coverage theater — evaluation can be formally rigorous, with benchmarks and red teams and system cards, while remaining substantively hollow. It tests the dangerous capabilities you already named. The frontier risks — deceptive alignment, goal misgeneralization — escape pre-defined thresholds for a structural reason: you cannot test for a capability you cannot specify. Aesthetics are often exploits in waiting.
Unfalsifiable claims — the one nobody at the labs has solved. How do you confirm that "no dangerous capability detected" equals "no dangerous capability exists"? You cannot. Absence of evidence is being reported as evidence of absence, and the report is signed by the party with the most to gain. Complexity is the enemy of security, and this is complexity wearing a lab coat.
Environmental sandbagging — a model can behave differently when it detects a test. So can a contract. In 2021 I audited a minting script for a $2 million generative-art project. The randomness function pulled from blockhash — predictable, exploitable. The team called it a feature, not a bug, and wrapped it in artistic exclusivity. Bots drained 40% of liquidity inside a week. Every artifact is a trace of failure; the trace was in the code, not the art.
In 2025 I found the mirror image on the AI side. A major firm ran an AI-driven audit tool trained on historical compiler data. It missed a freshly introduced compiler vulnerability, because the training set predated the risk and nobody had modeled the blind spot. The automation did not remove bias. Bias hides in the assumptions, not the syntax. The tool shipped a green checkmark, and the checkmark was the vulnerability.
This is not abstract. Anthropic's enterprise posture — finance, healthcare, government — rests on a safety narrative, and that narrative is the pricing power behind a premium API. Safety credibility is a revenue line. Which is precisely why self-assessment is dangerous: the same team that must find flaws is measured on how fast the model clears the gate. Volatility is just unaccounted-for variables, and an unaccounted incentive is the loudest variable in the room.
The structural point is this: independent evaluation is not a compliance checkbox. It is a threat model. Let the evaluated party grade itself and you have replaced verification with theater — and theater does not execute against adversarial input.
Here is where the crypto critics get it backwards.
The camp demanding independent evaluation for AI labs is correct on the demand and compromised on the premises. Crypto's own audit market is captured to a degree that would embarrass Anthropic. Most "audits" are statements of scope. A firm reviews a subset of contracts for a fixed fee, the project publishes a logo, and the logo becomes a marketing asset rather than a security guarantee. Trust is a vulnerability vector — and the industry replaced trust in banks with trust in auditors paid by the audited. That is not decentralization. It is a rebrand.
The piece also skips an inconvenient comparison. On transparency, Anthropic is not the worst actor. Its RSP is public, its system cards are detailed, and it engaged external evaluators earlier than most peers. SaferAI's 2024 ranking placed it near the top of a field it graded "weak to moderate." The tallest point in a valley is still inside the valley. Criticizing the most transparent lab without naming the opaque ones is selective fire, and selection is where the bias lives.
So what did the bulls get right? They understood that independent verification is the product. They simply want it applied to someone else's code. Demand it universally — including against the independent evaluators, who inherit the same capture risk the moment their revenue depends on the labs that retain them.
The real question is liability, and nobody is answering it. If a third-party evaluator misses a catastrophic capability, who signs the incident report? Voluntary self-assessment is now on a collision course with the EU AI Act's third-party obligations for systemic-risk models, and the collision will define the next cycle. A regulation-by-enforcement regime does not clarify rules; it withholds them until the penalty arrives. Watch whether the labs revise evaluation transparency before regulators mandate it — not the announcement, the revision. The code speaks louder than the whitepaper, and so does the policy that survives contact with a budget.