Building a Multi-Sensor Lab for Interactive Communication Research: Challenges, Workarounds, and Lessons Learned

Designing multi-sensor experiments to study interactive communication is costly, complex, and often poorlydocumented, particularly in terms of implementation challenges and the rationale behind decision-making. Here, we aimto demystify the process by detailing the development of a custom multi-sensor lab designed to monitor comprehensivebehavioural and physiological responses during interactive communication. We describe a sample paradigm in whichtwo adults are engaged in natural conversation while listening to realistic background noise scenes delivered via openSennheiser HD-800 headphones. Multimodal data were collected using DPA headset microphones, Tobii Pro Glasses 3,a Vicon motion capture system (with Tobii integration), BIOPAC amplifiers (PPG, EDA, ECG, respiration, temperature), aFitbit Sense 2 smartwatch, and six Logitech BRIO 4K ultra HD video cameras recording via Open Broadcaster Software(OBS). Due to high bandwidth demands and varying sampling rates (25 Hz to 48 kHz) among the different devices,all systems recorded independently which posed a significant synchronisation challenge. A central RME soundcarddelivered trigger pulses to the Tobii, BIOPAC, OBS, and Vicon systems to mark stimulus onset/offset. Redundantsynchronisation mechanisms were implemented to mitigate the risk of trigger failure and bespoke processing pipelineswere developed to synchronise data streams. We share key challenges and creative solutions encountered in buildingsuch a lab, to help researchers understand and anticipate common pain points and make more informed decisionswhen implementing their own multi-sensor setups.