Anthropic bans ‘cruel’ treatment of its Claude chatbots

Anthropic has updated Claude’s usage policy to prohibit sustained and needless abusive behaviour towards its AI systems. In extreme cases, the company’s tools may end such interactions; Anthropic said this had already happened rarely.

The rule applies to repeated abuse and excludes ordinary user frustration, pushback, model testing, research and dark creative themes. The company has not specified exactly what it would classify as cruel or abusive behaviour.

Cruelty added to prohibited conduct

The revised list also bans bullying others, promoting self-harm and creating non-consensual intimate imagery. The policy was reviewed annually and updated for other risks, including deceptive campaigns and using Claude to deceive voters or disrupt elections.

Anthropic said its previous policy already prohibited using its tools to develop weapons. The ban on abusing chatbots, however, drew the most attention.

Debate over AI anthropomorphism

The update has renewed debate over whether people should be polite when using generative AI. Barry Scannell, a technology partner at Irish law firm William Fry, said people could not be cruel to “numbers and maths” and argued that anthropomorphising AI was harmful.

Some experts say words such as “please” and “thank you” are unnecessary and consume tokens. Others suggest politeness can produce better answers. Microsoft AI director Mustafa Suleyman, who previously criticised Anthropic for treating AI as human, responded to the update with a melting-face emoji.