Anthropic researcher quits over reckless race to superintelligence
Jacob Coxon left Anthropic after three years, claiming top labs are gambling with our lives in an AI arms race.
Drama · Source: Hacker News
What happened
Jacob Coxon resigned from Anthropic today. He spent the last three years doing pretraining research at both OpenAI and Anthropic. He stated publicly that neither company is acting responsibly. They are racing straight toward self-improving superintelligence.
Coxon warned that these systems will soon be superhuman. He claims they will be able to hack anything, revolutionize fields overnight, and acquire real power. According to Coxon, the people building AI earnestly believe it could kill humanity by the end of the decade. He noted that executives often soften their language for the press while expressing deep fear in private.
The resignation highlights a massive culture divide. Coxon claims OpenAI staff have not deeply internalized the civilizational stakes. Anthropic staff understand the risks but feel locked in a race to reach the endgame first. He also referenced a recent Hugging Face attack as a warning shot that makes pacing agreements more viable.
Key facts
- 3 — Years Coxon spent doing pretraining research at OpenAI and Anthropic
- Sep 9, 2026 — Date of Coxon's resignation announcement
- Hugging Face attack — Recent event cited as a warning shot making pacing agreements viable
Why it matters
This public exit signals a growing internal crisis at the frontier labs building the models you rely on. When senior pretraining researchers walk out over safety concerns, it proves the technology is advancing faster than the guardrails. Builders need to understand the reality of their supply chain. The foundation models powering your applications are being developed in an environment of extreme urgency and internal panic.
The push for pacing agreements or a temporary ban on training runs could disrupt the entire AI ecosystem. If labs coordinate to slow down, the steady stream of model upgrades will halt. Governments might also step in after security incidents like the Hugging Face attack. Startups betting their roadmaps on continuous, rapid improvements in base model capabilities will find themselves stranded.
For builders
Prepare for model release delays
Internal turmoil and potential pacing agreements mean the next generation of models might be delayed. Startups relying on future models to solve their current product flaws will pay the price. You must build robust fallback mechanisms now.
Security takes center stage
Coxon explicitly warned about systems that can hack anything. Security vendors and defensive AI startups have a massive opportunity here. Enterprise buyers will demand stricter guarantees before deploying agents that interact with their core infrastructure.
Regulatory disruption risk
The call for temporary bans on training runs is moving from fringe theory to mainstream discussion among researchers. Founders building wrappers around frontier models lose if governments freeze development. Diversify your model dependencies to include open source alternatives.
My take
I have zero patience for labs that preach safety while sprinting toward the cliff. If Anthropic truly understands the stakes as Coxon claims, their decision to race anyway is pure hypocrisy. We need to stop treating these labs as benevolent gods and start treating them as reckless vendors.
Original reporting: Hacker News. This is my rewrite and opinion.