Data collection in multimodal language and communication research: A flexible decision framework

Zusammenfassung

The contemporary study of human language and communication has expanded beyond its traditional focus on spoken and written forms to incorporate gestures, facial expressions, and sign languages. This shift has been accompanied by methodological advancements that extend beyond classical tools such as tape recorders or video cameras and include motion-tracking systems, depth cameras, and multimodal data fusion techniques. While these tools enable richer empirical insights, they also introduce significant conceptual and practical challenges, particularly for researchers new to multimodal data collection. This paper provides a structured exploration of the methodological workflow essential to multimodal language and communication research. We present a flexible decision-making framework that guides researchers through key considerations, from data selection and its alignment with research questions, to data collection methods, technical requirements, and data management, including ethical considerations and data sharing. We also address critical factors such as equipment choice, data synchronization, and ethical concerns (e.g., privacy and data protection) while illustrating these processes with examples from different research contexts (i.e., lab-based experiments, large-scale annotated corpora, field studies including non-human primates). Rather than advocating a one-size-fits-all approach, our discussion emphasizes key decision points, trade-offs and real-world examples to help researchers navigate the complexities of multimodal data collection. By integrating perspectives from different disciplines, our flexible decision-making framework is intended as a practical tool for newcomers to address common conceptual and methodological challenges in the rapidly developing area of multimodal data collection.

Datum