Challenges and opportunities of personalizing a chatbot for young people with depressive symptoms

Depression is a pervasive challenge affecting a substantial number of young individuals. For various reasons, such as limited treatment resources or a desire to solve problems on their own, a lot of young people use chatbots instead of human therapists for support. Despite their great promise and encouraring preliminary evidence, mental health chatbots face various hurdles that hinder them from reaching their full potential. Our work delves into two key facets of improving mental health chatbots. First, it addresses the issue of selecting relevant treatment modules for the individual user. Secondly, our work explores the integration of large language models (LLMs) to enhance the capabilities of mental health chatbots. While LLMs hold the promise of moving beyond rule-based chatbots to provide more personal and effective support, they introduce challenges concerning dialogue quality and user safety.

To tackle these challenges, we developed Cady, a mobile app designed to assist young individuals dealing with depression. Cady utilizes a rule-based chatbot approach and delivers content derived from CBT and IPT. Our research comprises two significant components.
... mehr

First, we conducted an RCT with 152 young people, in which we compared three approaches to module selection: (1) a standardized treatment plan, (2) user-led personalization, and (3) hybrid personalization. Users used one of the three Cady versions for 14 days. While all three versions of Cady led to a reduction in depression severity, users achieved larger reductions in the personalized treatment groups. However, hybrid treatment plan personalization, in which users were able to adjust a treatment plan personalized by the chatbot, did not lead to larger reductions in depression severity than user-led personalization, in which users chose the modules. Nevertheless, our post-hoc interviews revealed a clear preference for hybrid personalization.

Second, we followed an iterative development process to integrate LLMs, specifically GPT-3.5 and GPT-4, to elevate Cady’s natural language capabilities. We experimented with different techniques, such as prompt engineering, fine-tuning and topic management, to instruct the LLMs to deliver CBT-modules, while ensuring fidelity, safety and an engaging user experience. We will share our experiences and discuss the current state of the LLM-based version of Cady.

In conclusion, our work contributes insights into the critical question of personalizing mental health chatbots for individual users, both in terms of module selection and content personalization. These insights have the potential to enhance user experience, user engagement, and the overall effectiveness of mental health chatbots in supporting young individuals dealing with depression.