DIMS Dashboard for Exploring Dynamic Interactions and Multimodal Signals

Social interaction is a complex, multimodal phenomenon with varying timescales and meaning-making structures. Research in this area has progressed along two largely separate paths: qualitative researchers focus on fine-grained analysis, while quantitative researchers computationally identify broader patterns. To bridge this gap and promote cross-disciplinarity, we developed the Dynamic Interaction and Multimodal Signals (DIMS) Dashboard, an open tool for visualizing multimodal data, enabling a qualitative-quantitative synergy in social interaction research. We overview its development, and conduct a proof-of-concept qualitative-quantitative (“quali-quanti”) analysis using neural and behavioral time-series data combined with video recording. Our exploratory case study reveals that 80% of segments with sharply increased neural activations in the right temporoparietal juncture align with highly engaged interaction, while 20% correspond to topic transitions. Through triangulation with qualitative insights, we observed that social brain synchrony relates in meaningful moments to head motion synchrony. Finally, we discuss how visualization tools like DIMS enhance multimodal, cross-disciplinary research in social interaction, and tool development.