Anthropic's Amodei proposes plan to 'slow the pace' of advancing AI capabilities

Anthropic CEO Dario Amodei published an essay on Saturday urging artificial intelligence companies to pace how quickly they improve model capabilities, a move that comes as a growing chorus of researchers have called for a coordinated slowdown.

Amodei proposed a three-step plan that he said will help temper the pace of development without "sacrificing commercial advantage or the United States' lead in AI," though he conceded that some steps may be easier to achieve than others.

Anthropic is actively gearing up for what is widely expected to be a historic IPO, though the company has not officially disclosed when it plans to debut.

Anthropic has "unilaterally" committed to the first step of the plan, Amodei said, which grants third-party evaluators employee-level access to the company to verify safety practices and report incidents.

The second step encourages leading AI companies within democratic countries to coordinate and establish common safety standards, and the third calls for coordination between democratic governments and authoritarian governments.

"To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this," Amodei wrote.

Concerns around AI's capabilities Amodei's essay landed after an Anthropic researcher set off a firestorm on social media this week by announcing he quit his job at the company.

Jacob Coxon, who has also worked as a researcher at Anthropic's chief rival, OpenAI, said he resigned out of concern that Anthropic and OpenAI are "gambling with our lives."

He said the people building AI "earnestly believe that it could kill us all by the end of the decade."

While extreme, concerns about the potential for AI to cause human extinction or other catastrophic events are not new in AI research circles.

In 2023, for instance, prominent AI researchers and executives, including Amodei and OpenAI CEO Sam Altman, signed a statement that said, "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

Amodei said Saturday that while pausing or slowing AI development has been floated since 2023, it made "little sense" to do so at that time.

He said models were not powerful enough to take action in the real world at that point, and they were also not yet capable of "significant deception, manipulation, cheating, or cyberattacks."