A Deterministically Synchronized Widefield Imaging and Virtual Reality Platform for Multimodal Brain–Behavior Recording
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
Simultaneous recording of brain activity, behaviour, and virtual environments is essential for understanding large-scale neural dynamics during behaviour. However, existing systems often rely on software-based synchronization or post hoc alignment, introducing latency, jitter, and drift that obscure fast brain–behavior interactions.
Approach
Here, we present a deterministically synchronized widefield calcium imaging platform that unifies neural imaging, high-speed behavioural monitoring, and closed-loop virtual reality (VR) under a shared hardware-defined clock. This system enables millisecond-precision temporal alignment across modalities, including dual-wavelength hemodynamic correction, pupil and orofacial tracking, locomotion sensing, and VR rendering.
Main results
The platform achieves stable hardware-level synchronization across neural imaging, behavioural recordings, and VR rendering without reliance on software timestamps. It supports widefield imaging rates up to 100 Hz and integrates seamlessly with both ViRMEn and Blender VR engines, exhibiting a mean locomotion-to-VR update latency of ∼1.5 ms. Multimodal recordings during VR navigation demonstrate robust temporal alignment between cortical activity, facial dynamics, pupil signals, and locomotion.
Significance
This system provides a deterministic multimodal framework for studying brain–behaviour relationships during active behaviour. By enabling millisecond-precision synchronization across neural imaging, behaviour, and virtual environments, this platform enables causal investigation of brain–behaviour interactions at millisecond precision and provides a foundation for next-generation closed-loop neuroengineering experiments.
