Harnessing Natural Language Processing for Automated Exposure Therapy Coding in Youth with OCD

This paper presents a complete automated classification system for labeling Exposure Process Coding System (EPCS) quality codes during in-person exposure therapy sessions. Our system is based on automatic speech recognition (ASR) and natural language processing techniques. It is trained and tested on 360 manually labeled exposure therapy sessions from three pediatric OCD clinical trials. To establish feasibility, we focus on detecting two EPCS technique behaviors: exposures and encourage events. We investigate the impact of audio quality, preprocessing strategies, and transformer-based models such as BERT, SBERT, and Llama-3.3, among others. We evaluate the performance of OpenAI's Whisper and Google Speech-to-Text by manually transcribing two minutes of each audio session. Comparing the manual transcripts with the ASR-generated transcripts, we found that Google Speech-to-Text had a word error rate (WER) of 0.51, while OpenAI's Whisper achieved a lower WER of 0.31. We also explore two classification settings: sequence classification and token-level classification. In the sequence classification setting, SBERT-based classifiers outperformed two traditional approaches (Bag-of-Words and TF-IDF) as well as a standard BERT model, achieving 0.81 accuracy in detecting exposures and 0.68 accuracy in identifying encourage events within single-label text segments. The token-level classification approach is used when the boundaries of exposure and encouragement events are unknown. This technique achieved high accuracy in identifying both exposure and encourage events, with AUC scores of 0.85 and 0.75, respectively. Finally, we discuss challenges and future directions for optimization, including improvements in ASR accuracy, expanded training datasets, and multimodal data integration. These results demonstrate, the feasibility of automated quality coding during in-person therapy sessions. Such a system could eventually enable rapid quality assessment in clinical settings and support scalable research into effective therapy methods.