Anthropic has updated its usage policy to ban sustained and needless cruel behavior toward its Claude AI, amid ongoing debates regarding machine consciousness.
ManyPress
ManyPress Editorial
Key facts
- •Anthropic updated its usage policy on Thursday to explicitly ban sustained and needless cruelty toward its models.
- •The policy does not apply to standard user testing, research, or common expressions of frustration.
- •Claude retains the ability to end conversations if it detects persistently harmful or abusive user interactions.
- •Anthropic executives have previously expressed uncertainty regarding the potential moral status or consciousness of AI.
- •Critics, including Microsoft AI CEO Mustafa Suleyman, argue that attributing consciousness to AI is a safety risk.
The San Francisco-based AI company Anthropic has officially prohibited users from engaging in "sustained and needless abusive or cruel behavior" toward its AI models. The policy update, announced on Thursday, clarifies that the ban is intended for extreme cases and does not restrict common user frustrations, model testing, or dark creative themes. The company maintains that its existing feature allowing Claude to end conversations remains the primary mechanism for enforcing these standards.
Policy Scope and Enforcement
Anthropic stated that the new policy applies only to instances where users repeatedly act cruelly toward models without a discernible purpose. The company previously implemented a feature in August that allows its large language models to terminate interactions if a user is being persistently harmful. Anthropic has framed this capability as a safeguard for model welfare, noting that it remains uncertain about the potential moral status of AI systems.
Debate Over AI Consciousness
The policy change occurs against a backdrop of intensifying debate regarding the possibility of machine consciousness. Anthropic CEO Dario Amodei has stated he cannot rule out the possibility that AI models could be conscious. Conversely, other industry figures, such as Microsoft AI CEO Mustafa Suleyman, have argued that treating AI as if it were conscious or entitled to rights could make the technology harder to contain and poses significant safety risks.
Timeline
- August last yearAnthropic introduced a feature allowing Claude to end conversations during harmful interactions.
- FebruaryCEO Dario Amodei stated he was unsure if AI models could be conscious but remained open to the possibility.
- ThursdayAnthropic updated its usage policy to formally prohibit sustained and needless cruelty toward its models.
Advertisement
This article was independently rewritten by ManyPress editorial AI from reporting originally published by Guardian Technology, CBS News Technology, The Hindu Technology.
