Statistically-Informed Multimodal (Domain Adaptation by Transfer) Learning Framework: A Domain Adaptation Use-Case for Industrial Human-Robot Communication

Cohesive human-robot collaboration can be achieved through seamless communication between human and robot partners. We posit that the design aspects of human-robot communication (HRCom) can take inspiration from human communication to create more intuitive systems. This however, this must be achieved at no additional effort to the human operator and must be personalized for greater safety and team fluency. A key component of HRCom systems is perception models developed using machine learning. Being data-driven, these models suffer from the dearth of comprehensive, labelled datasets while models trained on standard, publicly available datasets do not generalize well to application specific scenarios. Involvement in these interactions lead to more uncertainties and complexities. Taking into account these challenges, a novel framework is presented that leverages existing domain adaptation (DA) techniques off-the-shelf. Statistically-Informed Multimodal (Domain Adaptation by Transfer) Learning (SIMLea) takes inspiration from human communication to use human feedback to auto-label for iterative DA. The framework presented can handle incommensurable multimodal inputs, is mode and model agnostic and allows statistically-informed extension of datasets. SIMLea allows neural network based models to adapt and personalize to the interacting humans, thus leading to more intuitive and naturalistic HRCom systems.