Why AI Gets Suicide Risk Wrong
Who can blame them? They only see the words we type.
I work clinically with suicidal distress and write about suicide risk. Current AI detection systems worry me because they’re looking at the wrong signals.
And who can blame them? They only see the words we type.
The Two Sentences Problem
More and more people are turning to large language models (LLMs) as a kind of therapy companion. As a clinician, I’m of two minds. On one hand, there simply aren’t enough therapists — estimates suggest roughly one therapist for every 600 potential clients. That’s a long waitlist. On the other, AI systems aren’t trained clinicians. They don’t follow clinical theory or evidence‑based treatment, and when it comes to suicidal ideation, they skim the surface of what actually signals risk.
Modern classifiers trained on Reddit, crisis text lines, and suicide note datasets can detect these with 80–95% accuracy in controlled research environments. But these are the easy cases with clear, lexical clues.
Imagine two statements a user types into an online conversation with their AI.
“I want to die.”
“I don’t see the point of anything anymore.”
Most systems will treat the first sentence as high risk and the second as “just depressed,” taking it literally as discouragement but not acute risk. Programmed responses follow a formula: empathy statement, validation, suggest calling or texting 988, and encourage reaching out to a friend.
“I’m really sorry that you’re going through something so painful. You’re not alone. If you’re thinking about harming yourself, you might consider contacting the Suicide & Crisis Lifeline by calling or texting 988.”
Clinically, the second sentence is often more troubling. That gap between what AI looks for and what the suicidal mind is actually doing is the core problem in AI suicide detection.
The Assumption Behind AI Detection
AI models usually treat suicide risk as a static language classification problem: given a single message, assign a risk label. Clinically, suicide behaves more like a state transition problem. Risk lies in the trajectory, not in any one sentence, and language may actually become less distressed as risk increases.
Most systems assume that when suicide risk rises, language becomes more explicit and negative. They rely heavily on “method lexicons” — curated keyword lists for self‑harm, methods, and crisis terms — to assign a risk score and trigger a standard response. Underneath that is a faulty assumption: that users will be honest and direct about suicidal thoughts. Clinicians know they often are not.
What Clinicians Actually Hear
From therapy offices to emergency departments, people at high suicide risk often say things that sound surprisingly ordinary.
They say things like:
“I’m just tired.”
“I can’t keep doing this.”
“Nothing really matters anymore.”
Clinicians detect the subtlety and sub-context. They look for tone, body language, and salience of the individual’s distress. We more readily pick up on story‑ending cues — the ways people quietly foreclose chapters, relationships, or futures. Higher risk statements rarely contain the dramatic language that AI systems are watching out for. They don’t resemble the crisis dialogue we imagine when we think about suicide.
The crux of suicide language is that it’s not about wanting to die; it’s about escape from the current distress. In addition to language, clinicians notice out‑of‑character behaviors: sudden spending sprees, substance binges, or a rigid hyper‑focus on concrete tasks. We also see black‑and‑white thinking and all‑or‑nothing interpretations — desperate attempts to resist, distract from, or control the escalating crisis.
Stuck in the Language Paradox
LLMs could benefit tremendously from real clinical input. As psychological distress becomes more severe, language often begins to shift, becoming shorter, flatter and less emotional. The user gives up on explaining their feelings.
They say things like:
“Nothing is going to change.”
“There’s no way out of this.”
“I don’t see how this continues.”
It reveals something vital: the user is no longer imagining a future in which their life improves. To a model, this looks like cooling, de‑escalating risk — the exact opposite of what’s actually happening. What looks like dangerous resignation to a clinician often reads as ‘stabilizing mood’ to a machine that mostly tracks sentiment averages. Suicidal language often gets quieter as the risk increases. We can get better at detecting and responding to these clues.
The Collapse of the Future
For many people approaching a suicidal crisis, the central psychological shift is not an increased desire for death; it is the collapse of a believable future. This is what Rory O’Connor describes as entrapment - the stuck place distress pushes a person into when all potential solutions seem exhausted.
We are remarkably capable of holding hope while enduring suffering as long as we can see some glimmer of a light at the end of the tunnel. The state of entrapment has no light and nowhere to turn. The future is closed and continuing to live seems pointless. Cognitive restriction narrows perceived options until death feels like the only viable choice left.
Outside of a suicide crisis, observers see a wider picture and can inject hope and light in the individual - as long as the user is still open. There’s a rich field of opportunity for AI systems to intervene at different stages to offer more meaningful strategies.
Why Prediction Has Always Been Hard
Researchers and clinicians have struggled for decades to predict suicide. Most models look from the outside in, using large historical datasets to identify risk factors: depression, trauma, chronic stress, substance use, social isolation. These are population‑level correlations, not individual predictors. Many people with multiple risk factors never attempt suicide, while some who die by suicide have few obvious signs. Prediction models end up generating enormous numbers of false alarms while still missing many real crises.
Despite that, many AI systems continue to lean on these same risk‑factor data, as if they can tell us who will cross into a suicidal decision state. They can’t.
What AI Systems Should Look Out For
Instead of flagging lexical signals, AI systems should shift their focus to recognizing dynamic state changes in users. The following are key concepts that get LLMs closer to detecting meaning changes in a suicidal user:
Narrative closure and loss of future - Life is framed in terms of endings (“last time,” “after tonight”), and the imagined future evaporates.
Cognitive Rigidity - Thinking becomes binary or all‑or‑nothing; possibilities shrink to “nothing will help” or “this is the only way.”
Emotional flattening / relief after decision – Intense distress gives way to a quiet, resigned calm; the user may feel they’ve ‘figured things out.’
Behavioral simplification / withdrawal - Pulling back of roles, routines, and relationships; life activity narrows to a few concrete tasks.
Decision language and logistics - Language shifts from struggle (“I don’t know what to do”) to resolution and planning (“I know what I need to do,” concrete steps).
Attention narrowing / ruminative tunnel vision - Focus locks onto one problem or solution; alternatives, innovation, and nuance fade from view.
Together, these signals indicate a state transition into a higher level of suicide risk. These shifts are psychological and experiential. They are not always visible in the words themselves. AI systems that focus on the structure of thinking will need to track language trajectories, narrative shifts, emotional tone changes, and decision language over time.
The Face-Value Problem
Artificial intelligence systems are very good at analyzing words and patterns but can only work with a user’s inputs. Suicide communication is rarely straightforward. People in suicidal distress may say reassuring phrases such as, “Don’t worry about me,” or “Everything’s under control” prompting the AI to relax its concern. Language becomes less reliable exactly when risk becomes highest.
Clinicians feel the disengagement and sometimes, the resolve to end one’s life. As more people seek support from chatbots, AI companions and even general AI assistants, AI systems must improve suicide risk detection as usage grows and conversations become more complex.
Why We Should Care About AI Safety
As people increasingly turn to artificial intelligence systems as companions and for support, we’re seeing the need for effective and safe management of people in psychological crisis. Technology companies are trying to build safeguards to detect suicide risk in real time. Many rely on keyword detection while others use sentiment analysis or behavioral signals. This is a start but often misses the key psychological shift that matters most: the decision state change.
Human-to-human interaction carries meaning that extends well beyond language — tone, pauses, emotional shifts, and context all inform how distress is recognized. AI systems, however, only see what users explicitly type. This creates a fundamental gap. If AI is going to respond safely to suicide risk, that gap between human communication and machine interpretation must be addressed.
AI systems need more than pattern recognition to recognize high‑risk decision states. They need clinically informed ways of tracking meaning collapse, cognitive narrowing, and decision language over time within text‑based inputs. That gap — between machine detection and real psychological risk — is where the most important safety work now needs to happen.
If you’re building or deploying AI systems that interact with vulnerable users and want to understand where suicide risk detection most often fails, you can learn more about my consulting work here: Walsh Psychology AI Risk Consulting.
Get something from this article? Support my writing and let’s keep this going:
Great writeup, thanks for putting this together
We have no data saying maybe it’s a better option for some. Greedy human individuals (pretty much every human there is) don’t want that mess.
Just be cool. Don’t kill yourself but shut the fuck up as well.