Infrastructure for Digital Multisensory Systems

The primary goal of this project is to provide an infrastructure for digital multisensory systems (mainly mulsemedia). This project tries to answer how to bind different mulsemedia technologies in a smart way without decreasing QoE. Building such systems imposes a number of challenges and requirements due mainly to their ever-evolving and heterogeneous traits. As a result, few mulsemedia systems have remained reusable to be applied to different research purposes as opposed to the use of open mulsemedia datasets. Thus, the main product generated by this project so far has been PlaySEM SER – a framework for delivering sensory effects such as wind, smell, vibration, and light to heterogeneous systems. It deals with variability of scenarios in mulsemedia delivery systems, particularly with the variation of end-user applications and sensory effect devices through the support and reuse of even unforeseen protocols of communication and connectivity, and sensory effects metadata.

Partners:
Brunel University London and the University of Kent (UK).

Funding:
– Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Brasília)
– European Union’s Horizon 2020 Research and Innovation Programme

PlaySEM SER’s Github repository:
https://github.com/estevaosaleme/PlaySEM_SERenderer

Related PhD thesis:
Title: A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems
Author: Estêvão Bissoli Saleme
Supervisors:
Dr. Celso Alberto Saibel Santos, Federal University of Espírito Santo
Dr. George Ghinea, Brunel University London (UK)
Committee members:
Dr. José Gonçalves Pereira Filho, Federal University of Espírito Santo
Dr. Magnos Martinello, Federal University of Espírito Santo
Dr. Débora Christina Muchaluat Saade, Fluminense Federal University
Dr. Niall Murray, Athlone Institute of Technology (Ireland)

Alternative link to the thesis is available at ACM SIGMM Records.

Related publications:
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., A Mulsemedia Framework for Delivering Sensory Effects to Heterogeneous SystemsMultimedia Systems, Springer, 2019.
– SALEME, E. B.; COVACI, A.; MESFIN, G.; SANTOS, C. A. S.; GHINEA, G., Mulsemedia DIY: A Survey of Devices and a Tutorial for Building your own Mulsemedia EnvironmentACM Computing Surveys, 2019.
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., Coping with the Challenges of Delivering Multiple Sensorial MediaIEEE MultiMedia, 2019.
– SALEME, E. B.; SANTOS, C. A. S.; GHINEA, G., Improving Response Time Interval in Networked Event-Based Mulsemedia SystemsACM MMSys’18.
– SALEME, E. B.; CELESTRINI, J. R.; SANTOS, C. A. S., Time Evaluation for the Integration of a Gestural Interactive Application with a Distributed Mulsemedia PlatformACM MMSys’17.
– SALEME, E. B.; SANTOS, C. A. S., PlaySEM: a Platform for Rendering MulSeMedia Compatible with MPEG-VWebMedia’15.

Photo gallery: