Anthropic pitches Congress an AI safety bill that acts as a legal shield
Anthropic wrote its own AI safety bill, setting the bar to halt models at $1B in damages or 50 deaths while locking out other agencies.
Policy · Source: Reuters
What happened
Anthropic took a draft AI safety bill straight to Capitol Hill. The company behind Claude is not waiting for lawmakers to figure out frontier AI regulation. Instead, they wrote the rules themselves. Politico reported that Anthropic representatives quietly pitched this draft to lawmakers. The proposed law allows the government to block an AI model only under extremely narrow conditions.
The intervention threshold is massive. A model can only be halted if there is a foreseeable, imminent risk of more than $1 billion in damages. Alternatively, it must risk killing or seriously injuring more than 50 people. The harm must come from just three specific sources. These are the creation of a biological weapon, a cyberattack, or a model evading its developer controls.
Enforcement goes through a single bottleneck. Independent evaluators would register with the Commerce Department to conduct periodic assessments. AI companies can pay these evaluators directly, as long as payment is not tied to the outcome. If an evaluator finds a qualifying risk, they have 72 hours to report it. Then, only the Commerce Department can ask a federal judge in Washington D.C. to suspend the model. The draft explicitly bans any other federal agency from stepping in.
Key facts
- $1 billion — Minimum damage risk required to halt an AI model
- 50 — Minimum number of deaths or serious injuries required to trigger intervention
- 72 hours — Time evaluators have to notify the Commerce Department of a qualifying risk
- 3 — Number of specific harm categories that qualify for government intervention
Why it matters
This is regulatory capture disguised as safety. By setting the damage threshold at $1 billion or 50 lives, Anthropic ensures everyday AI failures fly under the radar. Normal copyright issues, bias claims, or smaller-scale economic damages do not meet the bar. Builders working on frontier models get a massive buffer against routine government interference. The regulated are writing the rules to protect their own operations.
The second-order effect is a total freeze-out of other regulators. The bill explicitly prohibits agencies like the FTC or FDA from suspending AI models. It preempts the messy patchwork of state-by-state laws that companies currently face. If passed, AI companies only have to worry about one department and one court. This drastically lowers their legal and operational risks while stripping power from consumer protection agencies.
For builders
Lower compliance overhead for frontier models
If this framework becomes law, AI developers face a single regulatory bottleneck instead of a maze of federal and state agencies. You only need to clear the Commerce Department. This saves millions in legal fees for well-funded labs.
A booming market for AI evaluators
The bill mandates independent evaluators to test models for catastrophic risks. AI companies will pay these evaluators directly. This creates a massive, legally mandated market for AI auditing and red-teaming startups.
Immunity from routine agency enforcement
Other federal agencies lose their teeth. If the FTC or FDA cannot suspend a model, founders building high-risk applications face less immediate threat of being shut down over standard consumer protection issues.
My take
When a tech giant asks for regulation, you have to look at the fine print. I see this not as a request for oversight, but as the construction of a massive legal moat. By setting the intervention bar at $1 billion and locking out agencies like the FTC, Anthropic is securing a federal shield against the everyday risks of AI.
Original reporting: Reuters. This is my rewrite and opinion.