Translation. Region: Russian Federation –
Source: Peoples'Friendship University of Russia
An important disclaimer is at the bottom of this article.
Researchers from the RUDN University Faculty of Artificial Intelligence conducted a large-scale study that uncovered systematic errors in large-scale language models (LLMs) used to diagnose depression from text. This work, conducted jointly with colleagues from AIRI, the Federal Research Center for Control and Management of the Russian Academy of Sciences, the Institute of Systematic Problems of the Russian Academy of Sciences, the Moscow Institute of Physics and Technology, and the Moscow Branch of the International Association of Zoological Research (MBZUAI), not only identifies the problem but also lays the foundation for the development of more reliable and secure tools for detecting depression and anxiety.
"Our research is an important step toward trusted AI in medicine. We don't simply point out the shortcomings of AI tools; we propose approaches to overcoming them. The key challenge today is not blind trust in algorithms, but their integration into physician workflows as a proven and understandable decision-support tool. Patient safety and understanding the limitations of this technology are our absolute priority," noted Anton Poddubsky, Dean of the Faculty of Artificial Intelligence at RUDN University.
The main value of the study is its detailed comparison of existing large-scale language models (LLMs), as well as methods for their use and retraining for the tasks of detecting depression and anxiety from text, and an analysis of AI errors and "hallucinations" in these tasks with the participation of psychology experts. The work of the RUDN University scientists received recognition and was presented at the highly regarded international conference "Empirical Methods in Natural Language Processing" (EMNLP). We spoke with the authors of the article to learn how the idea for the study came about, what AI "hallucinations" they identified, and the prospects for further development of the research.
How did the idea for research on this topic arise and why is it relevant and important?
In recent years, there has been growing interest in text-based mental health diagnostics and the use of AI in this field, as well as in the application of LLM in medicine in general. However, most studies rely on English-language data and ML models; comprehensive comparisons for the Russian language have not yet been conducted. This prompted us to explore LLM and other machine learning models for detecting depression and anxiety from text. We compared various models for both tasks and demonstrated which ones perform best in each case. We also conducted additional experiments to evaluate the quality of LLM generation from the perspective of expert psychologists. It turned out that, at this stage, LLMs produce answers of low quality. Specifically, in one experiment, we used LLM not only to determine the presence or absence of depression in a text author but also to generate an explanation for why the model reached this conclusion. It was in this experiment that we established that the explanations provided by modern models contain a significant number of errors from an expert perspective.
What is the main danger of such errors?
The danger lies in the fact that LLMs can produce unfounded or false conclusions ("hallucinations") that appear plausible to the end user. Such errors are difficult to detect without expert assistance, but they can lead to misinterpretation of depression symptoms.
What causes of AI errors have you identified? What is it about mental health conversations that so confuses even the most advanced language models?
Clinical psychologists analyzed the LLM responses and identified errors from an expert perspective. We identified six main types of errors: tautology, unfounded generalizations, false conclusions, confabulations, misrepresentation of medical concepts of depression, and incomplete listing of its symptoms. It's worth noting that, from a machine learning perspective, all of these errors can be described as "hallucinations," but in psychology-related tasks, a more precise categorization is needed. A characteristic of the texts used to detect depression is the difficulty of interpreting them. People often describe their condition indirectly, using metaphors, and text does not always directly reflect signs of mental disorders. Furthermore, the task of detecting depression from text is challenging for non-specialized models, as most of them are not trained on psychological or medical data.
What are the prospects for the development of this research?
The next step could be specialized retraining of LLMs on large datasets for depression and anxiety detection. The current experiments used a relatively small amount of data, which could limit the final quality of the models.
Please note: This information is raw content obtained directly from the source. It represents an accurate account of the source's assertions and does not necessarily reflect the position of MIL-OSI or its clients.