Automated Inference of Social Anxiety from Behavior in Social Virtual Reality

Background: Social anxiety often manifests in real-world interactions through behavioral patterns such as reduced eye contact and lower speech volume. While these markers have been examined in face-to-face paradigms, assessment of such behaviors in real-time social interactions has remained relatively limited. Social Virtual Reality (Social VR) offers a promising alternative by allowing for naturalistic yet experimentally controlled interactions, in which fine-grained social behaviors can be captured automatically and at scale. Examining how social anxiety is expressed in these contexts can support clinical applications such as early risk detection, tracking therapeutic progress, or tailoring interventions to individual behavioral patterns.Objective: This study aimed to assess whether behavioral and physiological markers associated with social anxiety can be similarly detected in dyadic Social VR interactions, and whether these patterns overlap with those linked to general psychopathology and verticality.Methods: A total of 128 participants engaged in 30-minute, avatar-mediated dyadic conversations within a Social VR environment. Behavioral markers such as gaze towards the partner’s eye region, smiling and speaking behavior were assessed, alongside physiological measures including heart rate and high-frequency heart-rate variability (HF-HRV). Psychological traits were assessed using self-report questionnaires, and composite factors for social anxiety, general psychopathology, and verticality were derived via principal component analysis (PCA). Relationships between traits and behavioral and physiological markers were analyzed using linear mixed-effects models.Results: We found that behavioral and physiological patterns previously associated with social anxiety also emerged in interactions in Social VR, replicating patterns observed in face-to-face interactions. Specifically, higher social anxiety was linked to less gaze toward the partner’s eyes while speaking and quieter speech, alongside reduced HF-HRV. Similar patterns were observed for general psychopathology. In contrast, higher verticality was associated with shorter turn-taking gaps and increased HF-HRV. Correlational analyses revealed highly similar behavioral profiles for social anxiety and psychopathology, whereas verticality displayed a largely opposite pattern.Conclusion: The findings demonstrate that behavioral and physiological patterns associated with psychological traits in face-to-face interactions are also traceable in immersive Social VR settings. Elevated social anxiety and general psychopathology were linked to increased submissive behaviors and heightened physiological stress, while higher verticality was associated with more assertive, dominant interaction styles — suggesting it may represent the opposing end of a shared social-behavioral spectrum. Notably, the strong convergence between social anxiety and broader psychopathology supports the view that many social interaction patterns are not uniquely tied to specific disorders but may reflect general psychological vulnerability. Although the passive and automated nature of behavioral data in Social VR presents novel ethical and privacy challenges, it also affords scalable assessments of social behavior in ecologically valid settings. These results highlight Social VR’s potential as a tool for assessing natural social behaviors in clinical and applied settings.