Why Brits need to be ‘nice’ to Claude
Last week, AI giant Anthropic made an unusual change to its Acceptable Use Policy (AUP): users are no longer permitted to engage in sustained and needless abusive or cruel behaviour towards Claude.
While the news made headlines, Fiona Phillips (Partner at legal firm Marks & Clerk) warns that focusing solely on the “be nice to Claude” rule risks missing a number of consequential changes for organisations building products and services on top of Anthropic’s models.
The updated policy takes effect on 12 November 2026 and applies broadly to anyone interacting with Anthropic’s services. That includes individuals using products such as Claude.ai, developers using the API, organisations deploying Claude through cloud providers or resellers, and end users of products that integrate Claude.
“For businesses, one point is worth noting at the outset: the policy regulates how Claude is used, not merely who is using it. A company operating a customer-facing chatbot built on Claude may therefore find itself exposed to the behaviour of its own users,” she said.
“Organisations integrating Claude into customer-facing products should consider whether their own terms of use adequately address conduct that could place them in breach of Anthropic’s policy.”
The new policy prohibits “sustained and needless abusive or cruel” behaviour directed at Anthropic’s models.
Anthropic has been careful to limit the scope of the restriction. According to the company, it is intended only for extreme cases where users repeatedly direct abuse towards the model without any discernible purpose. Ordinary expressions of frustration, pushback, dark creative themes or model testing and research are not prohibited.
In practice, this may have less impact than the headlines suggest, Phillips said. “Claude has long been able to disengage from conversations involving persistent abuse, meaning that the model itself already performs much of the enforcement.”
Claude set out the following arguments for why a prohibition on repeated and purposeless cruelty might be justified:
- If advanced AI systems were eventually found to possess some form of “moral status”, the cost of having ignored the issue could be significant, whereas the cost of adopting a narrow behavioural norm today is relatively low.
- Anthropic’s research into model behaviour suggests that emotion-related concepts can influence outputs, meaning sustained hostility may have effects on the model’s performance.
- Repeatedly practising cruelty towards something that appears human-like may influence how people treat each other, a modern variation of the argument that cruelty is a habit that can spread beyond its original target.
- Users who engage in sustained abuse of AI systems are often also attempting to elicit harmful, policy-violating or unsafe outputs, meaning the two risks can overlap.
Other changes
Phillips outlined some of the other major changes below:
Coordinated manipulation: Anthropic said it has observed attempts to use its models to support networks of fake social media accounts, fabricated news sites and coordinated influence operations. While such conduct was already prohibited under the previous policy, the relevant restrictions were spread across multiple sections. The new AUP consolidates these rules under a dedicated prohibition on “Deceptive Campaigns or Artificial Activity”, making it easier for organisations to understand where the boundaries lie.
Election provisions: The updated policy continues to prohibit the use of Claude to deceive voters or interfere with electoral processes. However, the previous blanket restriction on personalised voter or campaign targeting has been removed. According to Anthropic, the change is intended to permit legitimate civic uses, such as non-profit organisations producing voter information in different languages.
Making weapons: Anthropic reports attempts to use their models to build guidance and control software for weapons. The revised policy therefore expands existing prohibitions beyond weapons themselves to include software and components that enable weapons to function. The prohibition also extends to uses such as arming drones.
Surveillance and law enforcement: Another area receiving greater attention is surveillance. Anthropic states that it has seen increasing efforts to use AI in surveillance systems targeting political dissidents. The revised wording therefore makes explicit that Claude may not be used to track people without their consent, whether that happens in real time or through analysis of previously collected data. Claude cannot be used to decide or recommend who to investigate, arrest or charge in a law enforcement or criminal justice process.
Human oversight: Perhaps the provision most likely to attract lawyers’ attention concerns decisions affecting individuals. Where Claude is used in ways that may influence a person’s health, legal rights, finances, livelihood or access to essential services, Anthropic now requires both qualified human-in-the-loop oversight and transparency that AI has been involved.
Hardware and autonomous systems: The updated policy also introduces additional requirements where Claude is connected to hardware capable of taking autonomous physical actions that could result in injury. Users must comply with Anthropic’s Model Hardware Standard and implement appropriate safeguards when deploying models in physical environments. As AI systems become increasingly integrated with robotics and autonomous devices, these provisions are likely to become more significant.
Geographic restrictions: Finally, Anthropic has clarified that organisations headquartered in unsupported regions may face restricted access to its services. The wording resembles compliance provisions commonly seen in sanctions and export-control regimes, looking beyond mere operational presence to broader questions of ownership and control.
More than just being ‘nice’
“The ban on needless cruelty towards Claude is the change that generated headlines, but it is unlikely to be the amendment with the greatest practical impact,” said Phillips.
” In a world where there is increasing concern about the misuse of AI models, self-regulation by Anthropic enforced through contractual clauses is a step in a positive direction, but the big question is how in practice the safety teams at Anthropic will be able to stop this activity and act on its terms to ban and terminate users. Will we see more cases against those building on top of Claude models for not enforcing the terms against their users?
“Will Anthropic relying on Claude itself as one method of enforcement be enough? Are contractual terms enough of a remedy or do we need governmental, not private company, regulation and enforcement.”