Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework

Contemporary research on language and communication has expanded beyond its traditional focus on spoken and written forms to encompass signing, gestures, facial expressions, and other bodily actions. This shift has been accompanied by methodological advancements that extend beyond classical tools such as tape recorders or video cameras and include motion-tracking systems, depth cameras, and multimodal data fusion techniques. While these tools enable richer empirical insights, they also introduce significant conceptual and practical challenges, particularly for researchers new to multimodal data collection. This paper presents a structured, decision-oriented workflow for multimodal data collection in language and communication research. We introduce a flexible framework that guides researchers through key methodological choices, including the alignment of research questions with data streams, study design and acquisition strategies, synchronization and technical requirements, ethical governance, and data management, dissemination, and reuse. The framework is illustrated with case studies spanning controlled laboratory experiments, large-scale annotated sign language corpora, and field-based research, including non-human primates. Rather than advocating a one-size-fits-all approach, our discussion emphasizes key decision points, trade-offs and real-world examples to help researchers navigate the complexities of multimodal data collection. By integrating perspectives from different disciplines, our flexible decision-making framework is intended as a practical tool for researchers seeking to design, implement and to address common conceptual and methodological challenges in the rapidly developing area of multimodal data collection.