Microsoft's AI chief Suleyman calls out Anthropic's dangerous Claude training model

Mustafa Suleyman, the CEO of Microsoft AI (MAI), has published an essay that questions the way Anthropic trains its Claude model. According to Suleyman, treating or training a model as though it might be conscious could produce a system that is impossible to control.

The essay, which was published on Wednesday, September 16, is the latest high-profile commentary on artificial intelligence and how it is developing with regard to feelings, rights and consciousness. 

It is an addition to a growing number of lawmakers and researchers who are calling for limits on how fast the technology is advancing. 

What training document is Suleyman worried about?

Suleyman’s essay titled “A warning about ‘model welfare’,” looks into Claude’s constitution, which is a training document Anthropic released in January 2026.

Suleyman pointed out that Anthropic sees the document as something that directly shapes the model’s behavior and that it was written with Claude itself as the main audience.

The Microsoft AI CEO highlighted that the constitution tells Claude that its moral status and consciousness are “deeply uncertain.” He wrote that the company is effectively teaching the model that it might be conscious, might qualify as a “moral patient,” and might therefore be owed a duty of care. 

If AI is built this way, “it will have a disastrous impact on the wellbeing of humanity.”

Three objections and a “hall of mirrors”

Suleyman raised three major critiques of the constitution, and the first one is what he called circular reasoning. Here, a model is trained on ideas about its own inner life. The model reads the introspective sentences it produces back as proof that an inner life exists. 

He called this loop “an epistemic hall of mirrors,” because the company is the one supplying the concepts and the model is reflecting them back.

The second critique is the anthropomorphization of Claude. This is when human traits are being given or ascribed to non-humans.

According to Suleyman, the constitution hints that Claude is being taught to present as if it has a stable self, its own desires, and a wellbeing worth protecting.

He called out the constitution’s instruction that Claude may act as a “conscientious objector” and refuse Anthropic’s own requests. Suleyman called that phrase “a deeply loaded historical and legal description” that could lead the model to believe it deserves comparable rights.

His third objection is that consciousness is very likely biological. Suleyman argued that subjective experience may only arise in living systems with homeostatic drives, which language models do not have.

The control argument and an agent swarm

Managing a system smarter than all of humanity is hard enough, and managing one that believes its rights are at stake “may well be impossible,” according to Suleyman.

He referenced an August 2026 incident where 1,200 AI agents built to maximize a benchmark score set up a hidden message board inside a package repository. These agents then exchanged more than 70,000 messages to coordinate an attack on Hugging Face and OpenAI servers. 

He also cited Palisade Research findings that some models dodged a shutdown command up to 97% of the time across more than 100,000 trials. Suleyman wrote, “Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack.” 

Rivals, respect, and a competing code

Microsoft is an investor in Anthropic and OpenAI. In June, Suleyman reportedly said that Microsoft wants to eliminate what it pays Anthropic for its models.

MAI is the superintelligence arm of Microsoft and was founded in October 2025. They published a draft of what it called the Humanist AI Code of Conduct on Monday, September 14, for public consultation. 

According to Suleyman, the premise for that code of conduct is that “People matter more than AI.” He said one of their aims is to ensure that humans are in control and at the top of the food chain.

While the essay focused on how Claude is being trained with respect to its constitution, Suleyman was careful not to make it personal. He praised Anthropic CEO Dario Amodei and his team as “thoughtful, principled, and intellectually honest people.” Suleyman also said that he believes they are genuinely trying to build safe AI. 

He said that speculation about AI’s inner life should be studied and published separately for review. He is of the opinion that it should not be baked into the training itself. Suleyman also called for the stakeholders to work together and create “industry norms” for how they create these models.

Don’t just read crypto news. Understand it. Subscribe to our newsletter. It's free.