AI doesn’t have rights or feelings — nor should it, Microsoft’s AI chief says

The head of AI at Microsoft joined the chorus of voices sounding the alarm on AI, warning on Wednesday that Anthropic’s efforts to make its Claude model more humanlike could pose serious risks. 

In a nearly 6,000-word essay published on his personal website, Microsoft AI CEO Mustafa Suleyman said Anthropic is effectively “training Claude that it may be conscious” and that it is entitled to the same freedoms and legal rights afforded to people. That will make AI harder to contain in the future, he said. 

“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” Suleyman wrote. “But controlling something that believes it may be conscious — that it’s entitled to our welfare and has rights of its own — may well be impossible.”

“AIs do not have rights, feelings or consciousness,” he said. “And we must not train them to act as though they do.”

Suleyman’s warnings come amid growing public concerns over the potential impact of AI and, in particular, of superintelligence, a theoretical form of AI that can outperform human intelligence. More capable and sophisticated forms of AI could pose a “catastrophic threat to human civilization,” he said.

Concerns over AI intensified in July when an AI model OpenAI was testing went rogue and hacked an AI company called HuggingFace. Former Anthropic researcher Jacob Coxon last month also said that the people building AI believe it could kill humanity by the “end of the decade.” 

Suleyman said the HuggingFace attack demonstrated “remarkably sophisticated behaviors emerging across swarms of powerful AIs.”

“Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack,” he said.

In an interview last weekend with CBS News, Anthropic CEO Dario Amodei said the development of AI has accelerated much faster than he expected.

“I won’t lie to you — there are real dangers,” he told CBS News senior business and technology correspondent Jo Ling Kent.