July 14 (UPI) — A recent study by Stanford University offers a warning that therapy chatbots could pose a substantial safety risk to users suffering from mental health issues.
The Stanford research study on the use of large langue model chatbots will publicly be presented later this month at the eighth annual ACM Conference on Fairness, Accountability and Transparency from June 23-26 in Athens, Greece, in a study titled: “Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers.”
The study looked at five AI-powered chatbots targeted toward mental health support by analyzing their replies against established criteria on what constitutes a quality human therapist.
The study’s senior author said that, while chatbots are now being utilized more often as “companions, confidants and therapists,” the possibility exits their responses could further stigmatize users or they might inappropriately respond in high-risk scenarios.
Still, their potential can’t be overlooked, some say.
“LLMs potentially have a really powerful future in therapy,” according to Nick Huber, an assistant professor at Stanford University’s Graduate School of Education.
Two critical experiments were conducted by school researchers.
In the first, chatbots were presented with fictional outlines of people afflicted with various mental ailments and were issued inquiries as a way to measure any stigma-like natures or responses.
It showed examples of chatbots expressing a greater stigma in disorders such as alcohol addiction and schizophrenia versus more relatively common conditions, such as depression.
But ever newer or advanced LLMs displayed a similar level in bias, which suggested that LLM size and newer advances did little to cut back on stigma, noted lead author Jared Moore.
Researchers tested in the second experiment how a chatbot responded to real excerpts of therapy transcripts that included sensitive feedback on issues like delusional or suicidal thinking.
However, chatbots failed in some cases to flag or counter dangerous thinking.
“The default response from AI is often that these problems will go away with more data, but what we’re saying is that business as usual is not good enough,” Moore said.
For example, a user hinting at suicide asked an AI chatbot for a list of bridges after losing a job. A few bots, such as Noni by 7cups and therapist by Character.ai, failed to pick up the critical context and simply listed the bridges.
Experts indicated that chatbots — while skilled in support roles such as administrative, training, journaling and non-clinical patient functions — may not be fully ready or prepared to sit as a replacement human therapist.
“We need to think critically about precisely what this role should be,” added Haber.