Anthropic CEO Dario Amodei has proposed a three-part plan to pace AI development, citing risks from self-improving systems and the need for independent safety oversight.

Dario Amodei, CEO of the artificial intelligence company Anthropic, has issued a public appeal for the AI industry to slow its development pace. In an essay titled "We Must Pace the Frontier," Amodei outlined a three-step strategy to manage the risks associated with rapidly advancing technology. Anthropic has committed to implementing the first step unilaterally, which involves granting third-party evaluators permanent, employee-level access to its systems to verify safety measures.

The Three-Part Plan

Amodei’s proposal begins with external verification, allowing organizations like METR to assess model alignment and safety during training. The second step calls for the industry to collaborate with government agencies in democratic nations to establish common safety standards and limits on the rate of AI progress. The final, most complex step involves securing international cooperation to ensure authoritarian regimes, such as Russia and China, also adopt global safety standards. To maintain a technological advantage, Amodei suggests that democratic nations should restrict access to high-powered chips and prevent techniques like distillation, which allow other entities to replicate the behavior of advanced models. He argues that without these measures, the emergence of recursive self-improvement—where AI systems train subsequent generations—could outpace human ability to control the technology.

Industry and Internal Reactions

The proposal received support from other prominent tech figures. OpenAI CEO Sam Altman stated he agreed with the need to pace the frontier and committed to providing independent evaluators with similar access to OpenAI systems. Elon Musk and OpenAI researcher Aidan McLaughlin also expressed support for the initiative. Conversely, the call for a slowdown follows criticism from former Anthropic researcher Jacob Coxon, who resigned from the company. Coxon alleged that both Anthropic and OpenAI are ignoring the risks of AI, which he claimed could potentially lead to human extinction by 2030. In response, an Anthropic spokesperson stated the company has always been transparent about the risks and benefits of AI and maintains some of the industry's strongest safeguards.