How people in different cultures talk to a chatbot about sadness
- Article
- Mental Health
Many people type their worries into a chatbot before they say them to anyone else. A 2023 study by Hyojin Chin, Hyeonho Song, Gumhee Baek, Mingi Shin, Chani Jung, Meeyoung Cha, Junghoi Choi, and Chiyoung Cha, published in the Journal of Medical Internet Research, asked what those conversations look like and whether culture changes them.
The data
The authors pulled 152,783 chats from SimSimi, an open-domain social chatbot, sent between May 2016 and December 2020. Every chat contained a keyword related to depression or sadness. The messages came from eight countries: Canada, the United Kingdom, and the United States on the Western side, and Indonesia, India, Malaysia, the Philippines, and Thailand on the Eastern side. That gave 96,197 Eastern chats and 56,586 Western ones.
Two studies
The first study profiled the language. Using Linguistic Inquiry and Word Count (LIWC), the team scored 42,312 chats on six dimensions (affective, biological, social, temporal, perceptual, and pronoun use), then compared Eastern and Western means with Welch’s t-tests. They also extracted the top unigrams and bigrams after preprocessing with NLTK and scikit-learn.
The second study classified chats by topic. Each message was turned into a 6,841-dimension vector: a 6,027-term bag of words, 46 part-of-speech counts, and a 768-dimension BERT embedding. The team trained a support vector machine, naive Bayes, random forest, XGBoost, and a multilayer perceptron on 1,348 hand-labeled chats with an 80/20 split, then used a semisupervised loop that pseudo-labels unlabeled chats when the model is at least 90 percent confident and retrains.
What they found
Eastern users showed more positive sentiment even while sad, with a LIWC positive score of 0.81 against 0.52 for Western users. Western users swore more, at 0.41 swear words per chat against 0.19, and were more likely to share dysfunctional thoughts such as self-hate (11.36 percent of chats against 9.68 percent). Eastern users asked for coping advice more often (8.24 percent against 6.82 percent). Across the whole dataset, 49.83 percent of messages were plain expressions of apathy or sadness.
The multilayer perceptron was the best classifier, with an accuracy of 0.64 and an area under the ROC curve of 0.86 on short chat snippets.
What the authors recommend
The paper argues that emotional support chatbots should adapt their flow to the user’s culture: more empathetic prompts and advice for Eastern users, and moderated venting and reflection tools for Western users. It also calls for bias audits on classification thresholds, safeguards around pseudo-labeling, and future work that keeps context across several turns and supports more languages.