Dario Amodei’s AI slowdown plan arrived this weekend dressed as crisis management, but the response it has drawn from experts, politicians, and his own rivals suggests the AI industry’s self-appointed safety turn may already be in trouble. The plan is sensible in parts. The criticism is sharper.
From Coxon’s Resignation to Capitol Hill
The backdrop matters. Last week, CNBC reported that Jacob Coxon’s resignation post on X had been viewed more than 70 million times. The 27-year-old Anthropic researcher, who specialises in pre-training, the discipline of feeding AI models vast quantities of data, told the BBC that his peers feared the danger from advanced AI could arrive within two years, shorter even than the ‘end of the decade’ he cited in his resignation post.
That tighter timeline jangled nerves harder than the original warning. Coxon’s concern, according to the Wall Street Journal, was specifically that he did not want to participate in an industry-wide rush to build AI systems capable of improving themselves, fearing such systems could spiral out of control. His colleague Evan Hubinger, described as an alignment lead at Anthropic, publicly echoed those concerns and stated he believes there is a more than 10% chance AI could kill all humans within the next decade, while adding: ‘we do not yet have a plan to solve alignment for superintelligence.’
Then Anthropic disclosed that users had been dodging controls to use existing models in ways that could support biological weapons development. Lawmakers in London and Washington demanded brakes. AP News reported that Senator Bernie Sanders said he would soon introduce legislation to pause AI development and ban superintelligence outright.
Dario Amodei’s AI Slowdown Plan Draws Scepticism
Into this came Amodei’s three-point proposal. He wants embedded third-party evaluators given ongoing access inside each US AI company to check safety compliance. He wants all frontier AI companies in democratic countries to adopt common safety standards and limits on unchecked progress. And he wants democratic AI powers to coordinate with autocracies, notably China, beginning with a narrow agreement prohibiting obviously dangerous uses of AI, such as biological weapons development.
On paper, the first point is the most concrete and the most valuable. Until now, outside watchdogs have either been brought in at the companies’ own invitation or given only limited access when things go wrong. Embedding them permanently is a genuine step forward.
The rest is hazier. The third plank, coordination with China, faces the stiffest headwinds. Donald Trump said last week he had no concerns about AI leading to human extinction and doubled down on Sunday, saying people were ‘bringing up things that won’t happen.’ His treasury secretary, Scott Bessent, framed the AI question entirely through the lens of geopolitical competition: ‘There is no day after tomorrow if China wins at this. If they were to pull away from us on AI, then nothing else would matter.’ Trump’s meeting with Xi Jinping in Washington DC on 24 September will test whether any Sino-American coordination is even worth discussing.
David Sacks, co-chair of Trump’s council of advisers on science and technology, cut to the bone: ‘The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system.’ It is a pointed observation, whatever one thinks of its source.
Sam Altman at OpenAI and Elon Musk at SpaceX both backed the plan, which in a different context might have looked encouraging. Given that OpenAI launched GPT-6 Astra just days ago, marketed partly as a tool for booking tennis courts and ordering takeaways, the conversion has a slightly convenient feel.
PBS NewsHour notes that Anthropic has long pitched itself as the more responsible of the leading AI companies since its founders left OpenAI to form the startup in 2021. The company recently said it was ‘prioritize safety over speed when the two are in tension.’ The word ‘prioritise’ does a lot of work when the company simultaneously warns that AI agents could take over the entire internet within six to 12 months.
My read is that Professor Stuart Russell has the most penetrating critique. Amodei proposed slowing development to buy time to advance AI safety measures. Russell called this ‘completely backwards’: ‘We don’t just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right. We set the safety requirements and further progress occurs only when they are met.’ His pharmaceutical analogy is brutal in its clarity: no regulator would accept a cancer drug company releasing a new treatment every year and hoping clinical trials caught up.
David Krueger, an AI professor and former founding director of the UK government’s AI Security Institute, was blunter still: ‘Too little, too late.’ He called for an immediate, indefinite, international moratorium on frontier AI development. That position may be unrealistic in practice, but it at least has the virtue of internal consistency. Amodei’s plan, for all its good intentions, asks the people building the accelerator to also design the brake.
The real test is whether the embedded evaluators Amodei proposes acquire genuine independence, real access, and the authority to stop a product launch. Without that, the three-point plan is a document, not a mechanism. Watch what the evaluators are actually permitted to do when their findings conflict with a release date.
