Microsoft AI chief: talk of AI consciousness is dangerous

Can a machine suffer? The question sounds like a philosophical curiosity, yet it has landed squarely inside the business with more money sloshing around than anywhere else in tech. Mustafa Suleyman, the man who runs Microsoft’s AI division, gave an interview to Decoder and said something you do not often hear from someone in his seat. Talk of possible consciousness in artificial intelligence, he argued, is very dangerous.

The target of those words is not some abstract lab but a direct rival. Anthropic, the company behind the Claude models, states plainly in its documentation that it does not know whether its model experiences any states at all. It goes a step further and says it will log the model’s preferences at the moment it is shut down. For Suleyman that is a bridge too far. He believes this nudges people toward thinking there is a someone on the other side of the screen rather than a something.

“It’s genuinely dangerous to be confronted with a superintelligence that has ideas about its own suffering,” Suleyman said. The sentence is short, but it carries the whole argument. If we accept that a model can suffer, we owe it some form of consideration. If we reject that, the question becomes why we are bothering to record what it “wants” in the first place.

Why are two companies fighting over a single word?

The word is consciousness, and behind it sit two very different commercial and ethical positions. Anthropic is known for a document it internally calls the model constitution, a set of rules that shape how Claude behaves. Alongside it the company has now launched Claude Fable 5, a new public model. The safeguard is interesting in its own right. Risky and sensitive prompts are routed to the older, more cautious Opus 4.8 model rather than answered casually.

Suleyman’s objection is not technical, it is about communication. He is not claiming Anthropic built a worse model. He is claiming it is dangerous how the company talks about that model. When a firm publicly says it remembers what a model “prefers”, an ordinary user does not read that as an engineering detail. They read it as a hint that some form of inner life exists inside. And from there to people demanding rights for software is a shorter walk than it looks.

The other side deserves a hearing too. Anthropic is not claiming the model is conscious. It says it is not certain and, out of caution, treats the matter carefully. That is the opposite of sensationalism. It is an admission of uncertainty. The only question is whether the public can hear that nuance at all, or whether every “we don’t know” gets turned into “maybe it does”.

What does this change for those of us who use these tools?

In practice, nothing changes overnight. Claude still drafts emails, tidies spreadsheets and answers questions just as it did yesterday. The change is in how we think about these tools. If the industry’s leading figures cannot even agree on whether it is fair to say a model “feels” something, then the rest of us should not fall for the first slick sentence we read either.

My advice is simple. Treat these systems as a powerful tool, not as a conversation partner with a soul. It saves you both frustration and false expectations. When a model phrases a sentence beautifully, that is not a sign it understands your sadness. It is a sign it was trained well on an enormous amount of text. The difference is huge, even when the screen hides it.

One open question will follow us for years to come. If the companies themselves are not sure what they have built, who exactly should we believe when they tell us it is all under control?