Infrastructure for Digital Multisensory Systems

The primary goal of this project is to provide an infrastructure for digital multisensory systems (mainly mulsemedia). This project tries to answer how to bind different mulsemedia technologies in a smart way without decreasing QoE. Building such systems imposes a number of challenges and requirements due mainly to their ever-evolving and heterogeneous traits. As a result, few mulsemedia systems have remained reusable to be applied to different research purposes as opposed to the use of open mulsemedia datasets. Thus, the main product generated by this project so far has been PlaySEM SER – a framework for delivering sensory effects such as wind, smell, vibration, and light to heterogeneous systems. It deals with variability of scenarios in mulsemedia delivery systems, particularly with the variation of end-user applications and sensory effect devices through the support and reuse of even unforeseen protocols of communication and connectivity, and sensory effects metadata.

Partners:
Brunel University London and the University of Kent (UK).

Funding:
– Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Brasília)
– European Union’s Horizon 2020 Research and Innovation Programme

PlaySEM SER’s Github repository:
https://github.com/estevaosaleme/PlaySEM_SERenderer

Related thesis:
– SALEME, E. B., A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems, PhD thesis, Federal University of Espírito Santo, 2019.
– Alternative link to the thesis at ACM SIGMM Records: http://records.sigmm.org/?phd-thesis-summary=title-a-conceptual-architecture-and-a-framework-for-dealing-with-variability-in-mulsemedia-systems

Related publications:
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., A Mulsemedia Framework for Delivering Sensory Effects to Heterogeneous SystemsMultimedia Systems, Springer, 2019.
– SALEME, E. B.; COVACI, A.; MESFIN, G.; SANTOS, C. A. S.; GHINEA, G., Mulsemedia DIY: A Survey of Devices and a Tutorial for Building your own Mulsemedia EnvironmentACM Computing Surveys, 2019.
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., Coping with the Challenges of Delivering Multiple Sensorial MediaIEEE MultiMedia, 2019.
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., Improving Response Time Interval in Networked Event-Based Mulsemedia SystemsACM MMSys’18.
– SALEME, E. B.; CELESTRINI, J. R.; SANTOS, C. A. S., Time Evaluation for the Integration of a Gestural Interactive Application with a Distributed Mulsemedia PlatformACM MMSys’17.
– SALEME, E. B.; SANTOS, C. A. S., PlaySEM: a Platform for Rendering MulSeMedia Compatible with MPEG-VWebMedia’15.

Photo gallery: