SADIE: Spatial Audio for Domestic Interactive Entertainment

Interactive media systems such as games consoles have become commonplace in UK domestic environments offering increased connectivity and media integration. Whilst many systems support 3-D visuals and Ultra-High Definition video, most do not address the importance of 3-D sound for immersive experiences. 5.1 surround is the most common format supported by such systems, but its immersive capability is limited to a two-dimensional plane. New formats incorporating height channels have been targeted at cinema but could also naturally complement the virtual reality experience of gaming by extending the soundfield to three dimensions. The benefits not only include new audio features for enhanced gameplay, but also enable the design of immersive sound-centric games with significant social, cultural, educational and healthcare gains. However, sound immersion is difficult to achieve in the home as it is impractical to have dozens of transducers placed around the living room. SADIE pioneers new approaches for rendering interactive spatial sound in the home that includes sound source rendering with height.

For gaming, there is also the complex issue of listener position. For example, in games based on kinetic motion tracking the listener moves in reaction to the game play and the formation of a stable soundfields becomes difficult as they are no longer located at the acoustic sweet-spot. Real-time soundfield compensation through tracking of the listener can be used to counteract this, but the listener movement also makes digital equalisation of the room acoustics a considerable challenge. The result is a flawed virtual environment in which visual and auditory cues are not spatially coincident.

SADIE will address the improvement of spatial audio quality for immersive interactive media experiences in the home. It will undertake novel science to improve soundfield immersion and the formation of 3-D sound sources beyond the horizontal plane, lifting the constraints of loudspeaker placement and dynamic source-listener movements whilst conserving good sound reproduction quality. The research will pioneer new methods for soundfield rendering formed through characterisation of the cues required for perception of sources with height in dynamic listening. The work is poised to have significant transformative impact on sound reproduction in homes in the UK, whilst also addressing wider questions on auditory perception and acoustic signal processing that serve other research disciplines outside of interactive media.

The SADIE programme of research will impact upon:

The Interactive Media Industry:
The scientific outcomes have the potential for commercialization and knowledge transfer with (i) Hardware and Software audio systems producers (E.g. Headphone/Loudspeaker producers, Virtual Studio Technology producers) and Games Console manufacturers, (ii) Middleware game audio producers, and (iii) Games studios. Adoption of the research outcomes by the gaming industry will transform immersive sound capability in games. Whilst this project focuses on sound reproduction in the home, it also facilitates the development of new games which ultilise full 3-D surround sound for tracked players. This enables new sound features which can be exploited for market potential such as improved situational awareness, use of 3-D audio game cues to reduce user interface 'clutter' and improved inter-player dialogue via spatial positioning in networked games. The financial cost of creating a start-up company or employing knowledge transfer of the research into games companies is low in comparison to the revenue generated by successful games enterprises.

The Broadcast Industry:
Due to the logic of media convergence, games consoles offer increased media integration allowing viewers to not only play interactive games with local and remote players, but also to watch movies and television and use internet services. Consequently the broadcast industry is undergoing a significant change from video-based content delivery to multi-platform content. This means that immersive technology which benefits the games industry can also be used to enhance broadcast facilities in the home. SADIE will enable new 3-D sound formats aimed at cinematic reproduction which include height channels, such as 22.2 surround sound or Dolby Atmos, to now be presented to domestic viewers. The broadcast and cinematic industries are actively researching new methodologies for creating immersive media content for both theatres and the living room. Consequently, the British Broadcasting Corporation are a partner on this project, in an effort to advance spatial sound reproduction to support broadcast of immersive content to UK homes.

Game Players and Society:
Improved soundfield immersion and the discrimination of 3-D sound source locations beyond the horizontal plane will lead to enriched gaming experiences and creates the potential for new sound reproduction cues to enhance gameplay, not just for entertainment value, but also to support serious games with learning outcomes. The average child up to 21 years of age spends over 10,000 hours playing games meaning there is a 'cognitive surplus' which can be tapped into to increase the learning potential of digital games. Indeed, digital games have the capability to go beyond mere entertainment value and touch on deeper societal issues. Immersive 3-D audio can therefore help in the development of games aimed at providing deep cultural (e.g. interactive concert experiences), social (e.g. e-Learning games between local and remote classrooms) and even therapeutic learning outcomes (e.g. 3-D auditory stimulation for listening and learning disabilities). Immersive 3-D sound also enables the creation of enhanced audio-only games designed for the vision impaired.

Academics in other disciplines:
As outlined in the Academic Impact section, the proposed psychoacoustic and signal processing research will also have significant impact in other academic disciplines that utilise interactive audio technologies. These include psychology, computer science, music, and biology amongst many others. The psychoacoustic research fits into the EPSRC area Vision, Hearing and Other Senses and the scientific outcomes are well placed to make a larger impact to the wider ICT research portfolio.