Anthropic bans ‘needless abusive or cruel behavior’ toward Claude

Anthropic has banned sustained and needless abusive or cruel behavior toward its models, including the Claude chatbot. The Verge was first to report the policy change.

The company’s user policy says the restriction does not cover ordinary user frustration, model testing or “dark creative themes.” An Anthropic spokesperson did not clarify what conduct would qualify as abusive or cruel.

Claude can end harmful conversations

Anthropic’s language models can end a conversation when a user behaves persistently harmfully toward them. The feature was introduced last August as a safeguard for AI welfare.

The company says it remains “highly uncertain” about the possible moral status of Claude and other large language models, now or in the future. It is nevertheless researching the issue and implementing low-cost measures in case model welfare is possible.

Executives debate AI consciousness

Anthropic CEO Dario Amodei has said he cannot rule out the possibility that AI systems are conscious. OpenAI CEO Sam Altman has taken a more skeptical view of assigning that status to models.

Altman called religious interpretations of AI and the surrender of human judgment a real safety issue. His comments came after The New York Times reported on extensive discussions between Anthropic leaders and religious scholars.