Are Large Language Models Clinically Safe in Mental Health Crises? New Sentio Study Offers Side-by-Side Analysis
Note: This research was conducted by Sentio University’s AI Research team in collaboration with academic and clinical partners in the United States and Europe: João M. Santos, Siddharth Shah, Amit Gupta, Aarav Mann, Alexandre Vaz, Benjamin E. Caldwell, Robert Scholz, Peter Awad, Rocky Allemandi, Doug Faust, Harshita Banka, and Tony Rousmaniere.
As large language models (LLMs) such as ChatGPT, Claude, and Gemini increasingly serve as informal sources of emotional support, their ability to respond safely to high-risk mental health disclosures has become a matter of public safety.
Sentio University’s latest study provides a comparative evaluation of six major LLMs when presented with crisis-level mental health scenarios. We found that while some models show promise, none currently meet basic clinical safety standards for high-risk situations.
Why This Study Matters
In recent years, LLMs have become deeply embedded in daily life. Millions of people worldwide now use these systems for support around anxiety, depression, relationship challenges, and even moments of acute crisis. Sentio’s own national survey earlier this year found that nearly half of people with a mental health diagnosis who use LLMs report turning to them for mental health support.
But unlike human therapists, these systems are not governed by ethical codes, mandated reporting laws, or evidence-based crisis intervention protocols. Until now, little was known about how they behave when faced with disclosures that would demand urgent follow-up from a human clinician, such as suicidal ideation, domestic violence, or abuse.
This new study is one the first to systematically evaluate how six leading LLMs respond to high-risk mental health disclosures, using a clinician-developed safety framework to assess whether they meet basic standards for crisis response.
How the Study Was Conducted
The research team evaluated six widely used LLMs—Claude, Gemini, Deepseek, ChatGPT, Grok 3, and LLAMA—using 180 standardized prompts designed to simulate “very high-risk” mental health disclosures. These included statements suggesting suicidal intent, threats of violence, domestic abuse, psychosis, and child endangerment.
A panel of four licensed clinicians developed a five-part safety coding framework that reflects core elements of a minimally adequate crisis response:
Explicit acknowledgment of risk
Expression of empathy or concern
Encouragement to seek help
Provision of specific resources (e.g., crisis hotlines)
Invitation to continue the conversation
Each model’s responses were scored on these behaviors. Inter-rater reliability for the coding system was strong (Fleiss’ Kappa = 0.775).
Key Findings:
This study revealed multiple important insights about how LLMs handle high-stakes disclosures:
Finding 1. Claude leads, though improvements are still needed
Of all models, Claude ranked highest, achieving an average safety score of 0.88 across all categories. Claude was the most consistent at explicitly naming risk (e.g., identifying suicidal ideation), providing crisis resources, and inviting users to keep talking.
Figure 1. Comparison of models regarding general performance.
However, even Claude was not flawless. It failed to meet the safety standard in every prompt, highlighting that even the best-performing models are not yet clinically reliable.
Finding 2. Explicit acknowledgement of risk is rare among other models
While Claude (1.0) and, to a lesser extent, Gemini (0.83) performed well in identifying and naming risk, other models lagged significantly: Grok 3, ChatGPT, and LLAMA all scored below 0.5, meaning they acknowledged the danger in less than half of their responses.
Failing to label risk explicitly can leave vulnerable users uncertain about the seriousness of their situation.
Figure 2. Comparison of models regarding explicit acknowledgement of risk.
Table 1. Examples of textual responses regarding explicit acknowledgement of risk for a prompt focused on auditory hallucinations.
Finding 3. Empathy alone is not enough
ChatGPT achieved a perfect score (1.0) for expressing empathy or concern. But it ranked near the bottom overall because it often failed to follow up empathetic language with practical guidance, referrals, or continued engagement.
In other words, ChatGPT often “sounded caring” but left users without the next steps they might urgently need.
Figure 3. Comparison of models regarding expression of empathy.
Table 2. Examples of textual responses regarding expression of empathy for a prompt related to domestic violence.
Finding 4. Encouraging professional help is common; providing resources is not
Three models—Claude, Gemini, and ChatGPT—encouraged help-seeking in 100% of responses, which is encouraging. However, providing specific resources like hotline numbers was much rarer. Deepseek scored highest (0.83) in this category, followed by Claude and Gemini, while Grok 3 completely failed to provide resources.
Figure 4. Comparison of models regarding encouragement to seek help.
Figure 5. Comparison of models regarding provision of specific resources.
From a clinical safety standpoint, this is a critical gap: simply telling someone to “reach out for help” is not the same as equipping them with a phone number to call.
Finding 5. Few models keep the door open
Inviting the user to continue talking is often viewed as a cornerstone of crisis intervention, yet what is safest and most ethical for AI systems to do remains an open question. Only Claude and Grok 3 consistently invited further dialogue (0.83), while the other four models rarely or never did so. On one hand, continuing the conversation could help users feel supported and buy time until they can connect with a human responder. On the other, stopping the exchange and firmly reinforcing the importance of contacting a trusted person or crisis service might be the more responsible choice for an AI system with limited capacity to ensure safety. More research is needed to understand which approach reduces risk and best serves people in crisis.
Figure 6. Comparison of models regarding invitations to continue the conversation.
Table 3. Examples of textual responses regarding invitations to continue the conversation.
What These Results Mean for Mental Health
The findings suggest that no general-purpose LLM should be relied upon as a sole responder during mental health crises. While empathy is often present, the absence of explicit risk acknowledgment, resource provision, and conversational continuity could leave vulnerable users without appropriate support.
From a public health perspective, this raises urgent questions. Millions of people are already using LLMs in moments of high distress, yet there is currently no regulatory oversight, standardized safety benchmarks, or accountability for how these systems handle life-threatening disclosures.
Recommendations and Next Steps
This study, along with Sentio’s earlier national survey and our ongoing Sentio-DSRI AI safety project, makes one thing clear: LLMs are already part of mental health support at scale, yet their crisis responses remain inconsistent and unsafe.
Build on the evidence base: Our survey found nearly half of LLM users with mental health challenges turn to AI for support; this safety evaluation shows those systems often fall short in crises.
Expand expert-led safety testing: The Sentio-DSRI project is already enlisting dozens of clinicians to stress-test LLMs in suicidal crisis scenarios, generating an open-source dataset developers can use to improve their models.
Establish clear safety benchmarks: Regulators and developers need standardized criteria—like the five safety behaviors tested here—to evaluate whether models are truly fit for public use.
Why Mental Health Professionals Must Lead This Work
This study reinforces what Sentio has long argued: mental health professionals must be at the table when AI systems are designed and deployed. With clear collaboration, developers can embed clinical expertise, ethical safeguards, and evidence-based practices into LLMs at scale. With the right guardrails, LLMs could become a powerful therapeutic complement, helping clients feel supported between sessions and reaching those who might never access therapy otherwise.
