Masked Prediction of Onomatopoeic Expressions in Depressive Tweets Using LLMs

Onomatopoeia is a key linguistic feature for expressing emotions and psychological states. As large language models (LLMs) are increasingly applied in mental health contexts, it is important to examine whether they can interpret onomatopoeia appropriately. This study evaluates the capabilities of various LLMs through a masked prediction task using 319 depressive tweets in Japanese, where onomatopoeic expressions were masked and models were prompted to predict them. We compared the performance of three LLMs—GPT-4o mini, GPT-4 Turbo, and o1-preview—against a fine-tuned BERT model and human annotators, employing several prompt strategies including ReAct-style reasoning and dictionary-based in-context learning. Our results show that o1-preview achieved prediction accuracy comparable to human annotators. In contrast, the fine-tuned BERT model performed significantly worse, highlighting the limitations of conventional masked language models in handling emotionally nuanced expressions and adapting to prompt-based tasks. These findings suggest that LLMs, despite lacking physical embodiment, can effectively predict emotionally charged language such as onomatopoeia through large-scale pretraining and inference capabilities.