MPEG-V compliant 3D simulation tool for multimedia playback with sensory effects

Traditional multimedia systems normally include audio-visual content that only stimulates the senses of sight and hearing. However, stimulating additional senses can provide more immersive and realistic experiences, increasing the users’ Quality of Experience (QoE). For years, the research community has been working on the development of multimedia systems that include sensory effect metadata associated with the audio-visual content and capable of generating these effects, thus stimulating all the users’ senses. Examples of effects are scents (smell), flavours (taste), vibrations, pressure, wind effects (touch), special lighting, temperature, humidity, smoke, sprays (environmental effects), etc. There already exist some related solutions and standards (e.g., MPEG-V) that enable the integration of real sensory effect generation devices into multimedia systems. However, once these integrations are designed, having a complete physical setup with multiple physical devices in different positions around the user to test their performance is costly and allows little flexibility. A faster and cheaper alternative method involves the use of simulators. In this article, an MPEG-V compliant web-based 3D simulator is presented. The user can select audio-visual content, visualise it and check the correct activation/deactivation of each sensory effect during playback, as well as the position from which they are generated, among the 45 positions around the user defined in the standard. Additionally, a communication module with a controller device has been included to be used when it is integrated in a real mulsemedia environment.