10127335Development of an Immersive Control System for Extended Reality Experiences & PerformancesClosedCollaborative R&DInnovate UKImagine being in a virtual world where, in addition to seeing and hearing the action you can feel real sensations too! A gust of wind from a dragon's wing as it flies over, the heat as it breathes fire and the smell of burning ash. Feel the floor shake as it walks towards you. Our work supporting extended reality (XR) in the arts has seen a desire for technology solutions to deliver new audience experiences. However, this tends to follow traditional thinking and rely purely on visual experiences, due in part to a lack of understanding of multisensory delivery mechanisms which provide these enhanced audience experiences. We aim to combine virtual worlds and the real world together in one immersive multisensory experience. Our extended reality controller (HERC) can integrate physical multisensory effects seamlessly into virtual or augmented environments. The HERC system consists of two main parts - there is a hardware controller that handles the physical switches of devices, (such as turning a wind effect on at the right time), and there is a software system that talks to virtual environments (such as those widely used by the creative industries, Unity or Unreal engine) which determines when these effects occur. The system interacts with Unity or Unreal to determine where a person is, or what object they have just interacted with, and interprets this to trigger synchronised effects in the real world. These effects will complement the virtual action that the user is experiencing and can be from devices such as wind machines, heat lamps or scent dispensers, and will change the experience from being a virtual one to one that is fully immersive. The virtual experiences created can be experienced in several ways. Using Virtual Reality (VR) headsets, the user is given a full visual representation of an entire world. With Augmented Reality (AR) headsets, the user can see their surrounding environment, but with virtual elements such as people or creatures overlaid and interacted with. HERC can also be used with projectors or LED screens to create a fully immersive environment within a room that does not require headsets. Our project will make this level of immersion available on a much wider scale by making it easy for creatives to use rather than requiring significant technical expertise. The experiences can be anything from story narratives to interactive immersive art installations to interactive theatre.