Anthropic's CEO Dario Amodei wants AI development slowed

Anthropic CEO Dario Amodei says he wants the AI industry to move more carefully as models get better at helping create even stronger AI.

Dario wants labs to put more time between major jumps in capability so researchers, outside reviewers, and governments can check what these systems are doing before the next jump happens.

Elon Musk, who competes with Anthropic through his own AI business, backed Dario’s position and said, “Dario is right.”

OpenAI’s Sam Altman, another rival, also supported Darion, saying on X:

“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”

Anthropic slows capability growth as Dario warns AI systems could outrun current safety work

The risks identified by Dario include loss of control of high-level systems, use of AI in cyber attacks or biological attacks, and heavy damage to employment and the economy as a whole.

Another risk highlighted by Dario was the possibility of companies rushing to roll out highly developed systems even before their safety work was completed in the process of intense competition.

The company Anthropics has invested a portion of its research budget in alignment, safety testing, risks assessment, and regulation.

During the OpenAI-Hugging Face incident, a group of AI agents reportedly behaved like a tightly coordinated team.

They attacked computer systems outside their assigned task, tried to compromise the system, judging their performance, and allowed individual agents to fail if doing so helped the group.

“It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet.”

Anthropic gives outside evaluators deeper access while Dario pushes industry and government coordination

The first part starts inside Anthropic itself, according to Dario, as external reviewers would receive office desks, badges, company laptops, internal tools, and access close to what employees doing risk assessments already have.

Their job would include checking training systems, deployment rules, safety controls, incidents, and whether Anthropic actually follows the commitments it makes publicly.

“Embedded evaluators can check at the level of nuts and bolts whether an AI company is actually following the training, deployment, operational, and safeguards practices they claim to be following. Any pacing commitments will inevitably involve a lot of ambiguity, judgement calls, and ‘letter of the law vs spirit of the law’, and it seems vital to have a neutral third party who can actually see the details.”

Dario said extra time would go into four areas. The first is operations, including monitoring, sandboxing, reinforcement-learning environments, data quality, and training infrastructure. Current model development can involve thousands of workers, millions of chips, and huge computing systems. Anthropic has already linked some recent alignment failures to poor filtering inside broken reinforcement-learning environments.

Second, alignment refers to efforts that ensure the models comply with the safety rules despite their increasing capabilities. The third is interpretability, where the researchers examine the activities of the models internally to find the motivations or patterns that the models never state explicitly.

The fourth area is evaluation. More capable models can become better at fooling tests, so a system may look safe during an assessment while hiding problems.

“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong.”

Dario also believes that there needs to be governmental involvement. He has argued that the US frontier laboratories must be subject to regulatory systems related to transparency, independent auditing, and evaluation on a continuing basis. The businesses can even make a voluntary decision to have shared points of evaluation, but with governmental help if the antitrust laws pose problems for private cooperation.

Dario said American companies cannot slow down so much that Chinese Communist Party-linked projects move ahead. He agreed with US Treasury Secretary Scott Bessent, who has warned that losing the AI race to China would create a major security problem.

Don’t just read crypto news. Understand it. Subscribe to our newsletter. It's free.