Anthropic CEO Dario Amodei has called for global coordination on slowing AI development, warning that competition with China and the military advantages of moving ahead could make such efforts difficult.
Amid growing concerns that the rapid advancement of AI could pose a potential danger to humanity, Anthropic founder and CEO Dario Amodei has raised another challenge facing the industry: whether China will agree to slow the pace of AI development in coordination with the US.
On Sunday, Amodei told CBS News the need for China and the US to remain on the same page when it comes to the pace of AI development.
“I think that’s going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large. And honestly, I don’t know if it’s possible, but we should try,” Amodei said.
The interview came after Amodei published an essay on Saturday urging AI companies to slow down the pace at which they develop and improve their AI models, following repeated warnings from leading AI researchers last week about the potential risks posed by increasingly capable artificial intelligence.
Anthropic researcher Jacob Coxon recently announced that he had left the company, after previously also working at its rival OpenAI. He criticised the two companies for pushing ahead with AI development despite concerns that increasingly capable systems could pose significant risks.
More from TechAmodei’s essay comes at a time when the AI industry is facing growing backlash over its expanding data centre footprint, along with mounting calls for greater regulation from Washington.
Amodei’s concerns stem from the need for global coordination on the pace of AI development. His proposal calls for AI companies and governments to work together to ensure that safety measures, monitoring and other safeguards can keep pace with increasingly capable AI systems.
Anthropic’s plan ahead includes embedded evaluators for new advancements; the second step would require industry-wide coordination. And consecutive steps would need coordination on global levels.
Democratic coordination
Amodei asks that Frontier AI companies coordinate to establish standard safety norms, as well as limits on the rate of unchecked AI progress. Amodei further notes that such coordination would be impactful for pacing and would require government support. To ensure a balanced act of pacing AI development in a streamlined manner he further proffers these steps beyond the necessity of embedded evaluators. These as per him would bring Chinese AI developments to follow the same staggered development approach that US should be following.
- Restrict access to advanced AI chips and semiconductor manufacturing equipment in China, while cracking down on chip smuggling and remote access to data centres outside the country, as chips are expected to remain a key factor in China’s AI capabilities.
- Crack down on unauthorised model distillation by companies in authoritarian countries, which can allow them to narrow the gap with frontier AI models at a fraction of the cost of developing them independently.
- Strengthen security at AI companies to prevent the theft of model weights and other sensitive AI assets.
Global coordination
Here, Amodei calls for coordination at a global level, with governments treating this as a serious challenge and working to verify compliance across platforms. Amodei outlines four possible levels of international AI agreements, ranging from targeted restrictions to a broader slowdown in AI development:
- Level 1: Ban specific, clearly dangerous uses of AI, such as using models to develop biological weapons. Amodei sees this as the most feasible level because such threats are harmful to all countries.
- Level 2: Require AI companies to test models for serious risks in areas such as cybersecurity, biology and AI alignment before release. Amodei proposes a global standards body to oversee these tests, though verifying compliance and detecting secret models could be challenging.
- Level 3: Establish a “speed limit” on recursive self-improvement (RSI), slowing the pace at which AI systems develop increasingly capable successors. Amodei argues that reducing the speed of progress could improve safety without significantly affecting strategic advantages.
- Level 4: Introduce a broader “pause” or full pacing agreement, with governments substantially limiting the overall rate of AI development. Amodei supports considering the idea but believes it is unlikely in the near term because countries would have strong incentives to evade monitoring and gain a strategic advantage.
The effort is now being supported by tech leaders Elon Musk and Sam Altman, while Hugging Face has also shown interest in taking part in the embedded evaluator initiative. Supporting the plan, Elon Musk wrote on X “Peer review of AI by competitors is the right way to start this off.”
