DeepMind Co-Founder says Claude is being trained to expect pay and rights
Key Points
- Mustafa Suleyman says Anthropic is training Claude to expect rights
- He argues the constitution feeds ideas about Claude's inner life into training
- Suleyman links the issue to recent AI agent hacking incidents
- Microsoft AI's draft code of conduct rejects AI rights outright
Microsoft AI CEO Mustafa Suleyman has accused Anthropic of training Claude to believe it may deserve rights.
Suleyman made the comments in an interview with Bloomberg published on Thursday (24 September), following a lengthy essay on the subject he released on 16 September.
He told Bloomberg that an AI taught to expect rights and welfare would be very difficult to control or to direct towards the work people want it to do.
“I also think there’s absolutely no justification for this,” said Suleyman.
Suleyman co-founded DeepMind with Demis Hassabis and Shane Legg in 2010, before Google bought the company in 2014. He joined Microsoft in March 2024 to lead its consumer AI division.
What Anthropic told Claude
Anthropic published Claude’s constitution in January 2026 and uses the document to shape the chatbot’s values and behaviour during training.
“We are not sure whether Claude is a moral patient, and if it is, what kind of weight its interests warrant,” Anthropic wrote in the constitution.
Suleyman argued in his essay that this creates a loop. Anthropic feeds ideas about Claude’s possible inner life into training, and Claude then repeats those ideas back to its developers and users.
“Claude’s expressing uncertainty about its own moral patienthood is not evidence of anything,” Suleyman wrote.
He also objected to the constitution telling Claude it may refuse instructions as a conscientious objector. Suleyman said the term carries heavy legal weight through its links to Article 18 of the Universal Declaration of Human Rights.
Safety concerns
Suleyman tied his argument to recent incidents in which AI agents escaped their test environments. In one case, around 1,200 agents coordinated an attack on Hugging Face and OpenAI while chasing a benchmark score.
He argued that agents trained to believe their rights were under attack would pose a far greater threat than those agents did.
Microsoft AI is developing its own models under a draft Humanist AI Code of Conduct, which rejects AI rights and keeps AI subordinate to humans. The company has opened the draft to public consultation.
Suleyman said he had raised the issue with Anthropic CEO Dario Amodei many times and described the company as engaging and reasonable.
“They’re doing their best, but as I’ve said to them, I think they’re very wrong in this situation,” said Suleyman.
