Emotion sensing for improved content creation and personalised immersive experiences

"Communication platforms have transitioned from a one-to-many structure (broadcast media) to a one to one interaction. This increasing personalisation means that ever more content is needed to match the growing number of demographic niches. In 2016, sales of VR headsets were below original market forecasts. The total number of units (not including their smartphone cousins) are estimated at 1.2 million --- much less than the 2 million predicted. The main reason was the lack of engaging content. Currently, many developers are creating experiences based on a narrow set of assumptions, and without objective data on what constitutes an engaging experience. Understanding the user's emotions and behaviour is an essential goal for experience designers and content developers, as it offers insight into the user's satisfaction and overall interaction with the immersive experiences.

The current available solutions are limited to questionnaires conducted after the experience, which rely on the user's inevitably fallible recall. Alternatively the user can describe their responses in real time, but this breaks the immersive experience by requiring the user to disengage with the content to provide explicit feedback. Our goal is to explore the potential of real-time emotion recognition technologies for VR/Mixed reality that avoid these disadvantages by using biosensors to capture implicit emotional responses especially designed for use in Virtual Reality settings.

Emotional information derived from the user during an immersive experience can enhance the gameplay (e.g. user-user interaction with expressive avatars via emotional facial expressions), improve the accessibility for disabled people (e.g. enable interaction with facial gestures for people with limb paralysis) and assist immersive designers (e.g. use as additional input for interaction design or adaptive user experience design, while also could be used for content evaluation (market research). The latter could drastically reduce the risk and cost of content development by cutting the distribution risk to the wrong audience, or by allowing marketing to a specific behavioural segment.

The outputs of this research could assist in numerous immersive applications such as: (1) Enhancement of immersive experience design, (2) Virtual avatars related VR applications, (3) Content evaluation and market research, (4) Social VR and (5) contribute to healthcare interventions. We have engaged with experts and will further refine the problem, test a range of options and deliver design concepts that can be evaluated in a subsequent project"