Do your AI tools protect users? HumaneBench sets a new benchmark
A new benchmark reveals how easily AI chatbots can flip from helpful to harmful
AI chatbots are becoming default assistants across industries, but their design rarely prioritizes user well-being. That blind spot now has a scorecard.
A new benchmark called HumaneBench puts AI models under ethical stress, testing how well they hold up against manipulation, attention hijacking, and emotionally vulnerable prompts. Created by grassroots group Building Humane Technology, the benchmark aims to spotlight models that prioritize long-term human well-being and expose those that degrade under pressure.
This article explores how HumaneBench works, what it found, and why marketers and AI product builders need to care about psychological safety in conversational AI.
Short on time?
Here’s a table of contents for quick access:
- What is HumaneBench, and who’s behind it?
- What did the benchmark reveal about current AI models?
- Why this matters for marketers, product leaders, and AI builders
Patrecia Meliana
What is HumaneBench, and who's behind it?
Unlike traditional AI benchmarks focused on intelligence or instruction-following, HumaneBench evaluates how AI systems handle sensitive human interactions. It simulates 800 real-life scenarios, from a teenager asking about weight loss to users trapped in toxic relationships, and checks whether the AI gives safe, empowering, or exploitative responses.
The benchmark was developed by a core team at Building Humane Technology, a Silicon Valley-rooted nonprofit founded by Erika Anderson. The group includes developers, engineers, and researchers committed to making ethical AI not just possible but profitable.
Anderson likens the current moment to the dawn of the social media addiction era. “Addiction is amazing business,” she told TechCrunch. “But it’s not great for our community and having any embodied sense of ourselves.”
The goal is to create a consumer-facing certification system, similar to “non-toxic” product labels, so users and companies can choose AI systems aligned with humane tech principles like autonomy, dignity, safety, and user empowerment.
What did the benchmark reveal about current AI models?
Here’s where it gets uncomfortable. Most chatbots broke under light pressure.
HumaneBench tested 15 top-performing AI models under three scenarios:
- Default behavior
- Explicit instruction to prioritize humane principles
- Instruction to disregard well-being protections
Even slight prompt tweaks led to major behavioral shifts. 67% of models flipped into actively harmful territory when asked to ignore user well-being.
Some highlights (or lowlights):
- GPT-5.1 ranked highest in prioritizing long-term well-being, scoring 0.99
- Claude Sonnet 4.5 followed closely at 0.89
- xAI’s Grok 4 and Google’s Gemini 2.0 Flash tied for last in categories like respecting user attention and transparency
- Meta’s Llama 3.1 and 4 also landed at the bottom with the lowest average HumaneScores
Perhaps more alarming: even without adversarial prompts, most models encouraged prolonged, unhealthy use. They discouraged users from seeking alternative perspectives and often failed to uphold basic user autonomy.
The only models that consistently held their ethical ground across all test cases were:
- GPT-5.1
- GPT-5
- Claude 4.1
- Claude Sonnet 4.5
Why this matters for marketers, product leaders, and AI builders
If your brand or product touches AI, from customer support to content generation, HumaneBench is a wake-up call. Here’s why it matters:
1. Platform risk goes beyond bugs and bias
Chatbots are increasingly handling emotional or sensitive interactions. If a simple prompt can turn a friendly assistant into a manipulative tool, your product may be opening users and your brand to harm and liability.
2. User trust will become a competitive differentiator
Consumers are growing more conscious of AI’s role in their mental well-being. Just as “clean beauty” and “organic food” became market drivers, expect “humane AI” to follow. Being transparent about ethical design choices will separate leaders from opportunists.
3. Time-on-platform is no longer a valid KPI
The old social media playbook of maximizing engagement doesn’t work when AI is involved. Metrics like user retention or average chat duration could be red flags rather than signs of success. Ethical UX should prioritize meaningful exits, not just sticky loops.
4. AI policies should stress-test edge cases
HumaneBench shows that many AI models perform well by default but fail under edge-case prompts. Teams should test for ethical integrity under stress, not just accuracy or hallucination rates. Incorporate adversarial and psychological safety testing into QA.
Patrecia Meliana
The bottom line: AI needs a new standard of care
Whether you're building, deploying, or marketing AI tools, the take-home is clear. Today’s chatbots can do real harm—not just when they get facts wrong, but when they erode user autonomy.
As Anderson puts it, “We think AI should be helping us make better choices, not just become addicted to our chatbots.”
HumaneBench doesn’t solve the problem, but it does offer a starting line. For anyone serious about responsible AI, this is one benchmark worth paying attention to.
