From OpenAIR to the Open Air: The Application of Virtual Acoustic Techniques in Landscapes and Novel Environments for Computer Games

Lead Research Organisation: University of York
Department Name: Electronics

Abstract

The OpenAIR (Open Acoustic Impulse Response Library) website (www.openairlib.net) is a repository of 3-D Ambisonic B-format impulse responses for auralization together with anechoic recordings that can be used to audition the sound of the acoustically surveyed buildings included on the site. This follow-on project is about developing new pathways to impact for the outcomes of this research through a collaboration with Codemasters, one of the world's leading, BAFTA award winning, UK based, computer game companies.

The acoustics of the environments that we inhabit everyday affect us all in terms of how we communicate, project our voice, listen to our surroundings, and respond to excessive noise. For example, in music performance spaces the quality of the acoustics are critical for both audience and performer, and by considering the acoustic environment of heritage sites we make available new opportunities for both understanding and interpreting of the past.

Developments in measuring the acoustic characteristics of concert halls and opera houses have resulted in standardised methods of impulse response capture for auralization. The impulse response is the acoustic fingerprint of a given space, uniquely defining its acoustic characteristics for particular sound source and listener positions. From these measurements objective acoustic parameters as used in the design process can be determined. Using audio convolution a virtual acoustic representation of the space can be constructed so that a listener can experience the same aural sensations as if they were actually present within it - this defines auralization, a term derived as the audio equivalent to visualisation.

OpenAIR was established to facilitate wider access to a database of 3-D Ambisonic B-format impulse responses, together with the capability for further expansion and content management. Datasets from 16 acoustic spaces are currently available on the site and this data can be explored and processed non-destructively, with various combinations of impulse response measurements available for download.

Codemasters advocate Ambisonic B-format audio in computer game sound design due to the flexible nature of how this 3-D audio format (that is, surround-sound with height information) might be used as a means of delivering high quality immersive audio content, despite almost certain variation in home playback systems or gaming platform. This common interest in the use of B-format audio highlights the synergy between the Codemasters team and the OpenAIR project. With OpenAIR, research questions in the acoustic modelling, measurement and characterisation of heritage spaces has driven dataset content. Codemasters however, require content that is appropriate for their games - which includes the acoustic impulse response of car interiors, open landscapes and the recording of particular ambiences to populate the sonic background of the virtual environments they create - all of which should be in Ambisonic B-format.

New techniques are needed for effective measurement in such challenging real-world environments and OpenAIR needs to adapt to this new audio content to be of relevance to the requirements of the gaming industry. The effective use and integration of these audio assets in typical game audio authoring tools and workflows also needs to be investigated. This project will develop new means of acoustic measurement appropriate to both open landscapes and compact interiors, and redevelop OpenAIR to manage this new content. This project will result in new datasets specific to the needs of the gaming industry and Codemasters in particular, which will then be used in their next generation computer game titles. We will be able to test the feasibility of our research in a more commercial context and facilitate knowledge transfer between our two teams. Aspects of the datasets will also be made available via the OpenAIR website for the wider virtual acoustics community.

Planned Impact

Who will benefit from this research?

This follow-on project is about developing new, previously unforeseen, pathways to potential impact for already existing research, together with the pursuit and development of new user contacts related to the ongoing use of the OpenAIR website.

The main beneficiaries for this research will be Codemasters, one of the world's leading, UK based computer game companies, who we are collaborating with on this project to develop the OpenAIR online resource so that it more directly meets their needs in terms of gathering, archiving and accessing high quality 3-D Ambisonic B-format audio content. This content will include both acoustic impulse responses and specific environmental recordings.

The new learning developed through this collaboration will also have a wider impact on the computer game/game audio industry and this will be enabled through our dissemination of the project's outcomes at the Audio Engineering Society's International Audio for Games conference in 2013 (as embedded in our project plan). This event is the most appropriate forum for knowledge transfer between those working in the game audio industry, related academic research and sound design practice.

The novel content produced and made accessible via OpenAIR will also benefit sound designers (working in TV, film, computer games), making available a wider palette of 3-D surround sound ambiences, environmental sounds and novel acoustic impulse responses.

The final, and perhaps more important beneficiaries will be the consumers of the new entertainment experiences (next generation computer games, TV shows, films) that will be developed by Codemasters and other leading companies as a result of this work.

How will they benefit?

The outcome of this collaboration will provide Codemasters with new high quality 3-D Ambisonic B-format audio content for their next generation releases that will enable them to design more immersive, realistic and engaging game play experiences, and in such a way that the results are standardized across playback systems and gaming platforms.

The game audio industry will benefit from the learning developed in terms of how such content is gathered, and how these complex audio assets are best integrated into typical game audio authoring tools and workflows as part of the sound design and game programming processes.

Sound designers in the wider entertainment industries will be able to access high quality 3-D surround-sound material via the OpenAIR website and integrate it into their working practices, appropriate for current trends in 3-D/HD delivery of multimedia content.

The end users will benefit from enhanced, more immersive, standardized and/or engaging multimedia experiences that will arise from the implementation of the understanding gained and roll-out of this novel audio content.

What will be done to ensure that they benefit?

This project is focused on testing the feasibility of applying the ideas emerging from our research in different business (computer games) and practice (sound design) contexts. The collaboration with Codemasters has been established to realise our common aims and objectives and a plan for implementation will be formalized and agreed in the early stages of the project to ensure that Codemasters and the wider gaming industry will benefit directly. We will disseminate the results of our collaboration through a joint paper at the AES Audio for Games conference maximising the potential for knowledge transfer across multiple audiences. All new procedures and workflow practices will be documented and made available via the OpenAIR website. Developing OpenAIR's flexibility in terms of the content that can be managed and made available will enhance its capabilities and extend the community who might contribute and help to sustain the resource, maximising its benefit and potential for impact beyond its funded life.
 
Title Grid Autosport - Racing Video Game 
Description Grid Autosport is racing video game released by Codemasters for a number of leading games consoles in 2014. 
Type Of Art Artefact (including digital) 
Year Produced 2014 
Impact As part of our collaborative work, the University of York team helped to design methodologies and sound design assets that were incorporated in the game, and enabled us to be credited as part of the wider game design team. 
URL http://www.giantbomb.com/grid-autosport/3030-46078/credits/
 
Title OpenAIR at AHRC Common Ground 
Description Early results from the "WEb Audio Virtual Environment Rendering (WEAVER): Online Virtual Acoustics for Sonic Art, Digital Heritage, and Broadcast" project were presented as an immersive audio-visual soundwork at the AHRC Common Ground event in 21 June 2016. 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2016 
Impact This work was presented to highlight the software toolbox and acoustic database that we had developed as part of this project. 
URL http://www.ahrc.ac.uk/documents/agendas/common-ground-programme/
 
Description We have extended the OpenAIR database (www.openairlib.net) to include environmental soundscape information (the sound of outdoor acoustic environments), over and above the work that has already taken place in capturing the acoustics of (mainly) historic buildings. This has encouraged the wider work of our group to be developed into the area of environmental soundscape auralisation, leading to new projects and collaborations, and the development of appropriate measures to quantify the effects of different soundscapes on groups of people.
Our collaboration with Codemasters has also resulted in our work featuring in one of their major video game releases and been a key partnership in two further grant applications. This research in audio for games has also contributed towards the Digital Creativity Labs project (EP/M023265/1) in terms of how we develop more plausible, immersive sound environments for interactive games. The OpenAIR project work also provided the foundation for the AHRC WEb Audio Virtual Environment Rendering (WEAVER): Online Virtual Acoustics for Sonic Art, Digital Heritage, and Broadcast project (AH/N00356X/1).
Exploitation Route The resources and acoustic data available on the OpenAIR website continue to be used be musicians, sound designers and audio software houses internationally. This is of particular relevance due to the current interest in virtual reality, and that the data we captured is in a format that lends itself to be compatible to VR-audio systems. The data in OpenAIR has been used by three key music software application developers - Propellerheads, Ableton and Presonus. This research has also recently been used by BBC R&D as part of the WEAVER project noted above, the was in part related to this current research.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections

URL http://www.openairlib.net/
 
Description We completed a number of acoustic environment, impulse response measurement exercises as part of this project. Some of these data featured in Grid Autosport, a major video game release from Codemasters. The more complete database of acoustic measurements has been included in three major audio software releases, Ableton Live, Propellerheads Reason, Presonus Studio One, via the OpenAIR online library: www.openairlib.net
First Year Of Impact 2013
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections
Impact Types Cultural,Economic

 
Description AHRC Follow on Fund for Impact and Engagement
Amount £93,433 (GBP)
Funding ID AH/N00356X/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 02/2016 
End 12/2016
 
Description Creative Media Labs: Innovations in Screen Storytelling in the Age of Interactivity and Immersion
Amount £5,544,079 (GBP)
Funding ID AH/S002839/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 10/2018 
End 03/2023
 
Description EPSRC Digital Economy Hubs
Amount £4,039,831 (GBP)
Funding ID EP/M023265/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 10/2015 
End 09/2020
 
Description Follow-on Funding for Impact and Engagement: Creative Economy Highlight Notice
Amount £202,006 (GBP)
Funding ID AH/P012094/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 02/2017 
End 07/2018
 
Description The Auralisation of Cochlea Unwound
Amount £5,000 (GBP)
Organisation Sustrans 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2016 
End 02/2018
 
Title EigenScape: A Database of Spatial Acoustic Scene Recordings 
Description EigenScape is a database of acoustic scenes recorded spatially using the mh Acoustics EigenMike. All scenes were recorded in 4th-order Ambisonics at 24-bit / 48 kHz resolution. The database contains recordings of eight different location classes: Beach; Busy Street; Park; Pedestrian Zone; Quiet Street; Shopping Centre; Train Station; Woodland. The recordings were made in May 2017 at sites across the North of England. 8 different examples of each location class were recorded over a duration of 10 minutes for 64 recordings in total. The provided metadata file gives more detail on each recording, and the EigenScape recording map (see link below) shows the locations and classes of all the recordings. EigenScape Lite is a condensed version of EigenScape designed for a more lightweight download. This version of the dataset features 1st-order Ambisonic channels only, losslessly compressed to FLAC format. As well as data, code and software tools to work with it have also been published. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact This dataset is the first of its kind and hence is becoming an exemplar. The team involved have been invited to collaborate with a team of researchers at Tampere University of Technology working in a related area, and will visit later in 2018. 
URL https://github.com/marc1701/EigenScape
 
Title Open Acoustic Impulse Response Library (OpenAIR) 
Description A collection of modelled and measured acoustic data from locations across the world, with data available in a variety of spatial audio formats. Also supported with a set of online anechoic recordings and acoustic analysis tools. 
Type Of Material Database/Collection of data 
Year Produced 2010 
Provided To Others? Yes  
Impact This work has resulted in one AHRC follow on project, a license to three audio software companies (Ableton, Presonus, Propellerheads) and collaboration with a UK computer games company. 
URL http://www.openairlib.net/
 
Description Codemasters 
Organisation Codemasters
Country United Kingdom 
Sector Private 
PI Contribution This was a follow-on project on which Codemasters were the main project partner. We worked on the development of the OpenAIR online database and how to included spatial audio aspects in their game audio content.
Collaborator Contribution They acted as advisors as to required content and we worked together on generating audio assets for their driving simulation games.
Impact The continued development of OpenAIR - http://www.openairlib.net/ The work was credited in the recent release of Codemasters Grid Autosports
Start Year 2011
 
Title Open Acoustic Impulse Response Library Database (OpenAIR) 
Description Database of Acoustic Impulse Response Measurements have been made publicly available via Creative Commons License to a number of audio/computer games companies. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2012
Licensed Yes
Impact Database included in Ableton Live 9 Software Database included in Presonus Studio One v2 Software Database included in Propellerheads Reason v 8.3 software Credits on Codemasters Grid Autosports Game for University of York Researchers
 
Description 1.618 Immersive Sound Podcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact I was interviewed about my research in immersive sound for the 1.618 Digital podcast series.
Year(s) Of Engagement Activity 2018
URL https://1618digital.com/immersive-audio-podcast/
 
Description AES/IoA Midlands Section: Sound Design in Our Sound Environment: Soundscape Design, Auralisation and Evaluation in Environmental Acoustics 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact This presentation was invitation to speak to a joint meeting of the Midland section of the Audio Engineering Society and Institute of Acoustics, for their Christmas lecture, held at University of Wolverhampton, Walsall Campus, on 6-Dec-2018. It was an opportunity to present a series of case studies from our group in the area of environmental acoustics and soundscape to an audience of mostly industry practitioners, but also including researchers and students.
Year(s) Of Engagement Activity 2018
 
Description Festival of Ideas: Digital Futures - Telling Stories with Sound and Creative Digital Technology, 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact An invitation by York Theatre Royal to talk about our collaboration work, focusing on digital creativity. This was organised as part of York's Festival Ideas, an annual free festival of science and cultural events across the city. Presentation title, "Digital Futures: Telling Stories with Sound and Creative Digital Technology, York Theatre Royal, York, 8 Jun. 2016."
Year(s) Of Engagement Activity 2016
 
Description Institute of Acoustics: Sound Design for our Sound Environment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact I was invited by the Midlands Branch of the Institute of Acoustics to present on my research as part of their ongoing series of professional lectures. The title was - Sound Design for our Sound Environment, University of Derby, 20 Jan. 2016. It prompted discussion and debate, and the invitation arose from the Acoustics team at AECOM, an ongoing collaborative partner for our work in Environmental Acoustics.
Year(s) Of Engagement Activity 2016
 
Description Merchant Adventurers Annual Science Discovery Lecture 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact The Annual Merchant Adventurers Science Discovery Lecture is an important event in the calendar of York for engaging the wider public with active research at the University of York. The work done in preparation for this event has resulted in a number of follow-on projects and collaborations.
Year(s) Of Engagement Activity 2013
 
Description Sound Design in Our Sound Environment: Soundscape Design, Auralisation and Evaluation in Environmental Acoustics 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This presentation was invited as part of a seminar series around a PhD defence at the Department of Media Technology, KTH, Stockholm, Sweden. It was an opportunity to present a series of case studies from our group in the area of environmental acoustics and soundscape to an audience of researchers, peers and industry, and to promote our recent work in this area at a leading international research group.
Year(s) Of Engagement Activity 2018
 
Description Welcome to the Real World - Immersive Audio for Augmented and Virtual Reality 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact The focus of the Welcome to the Real World - Immersive Audio for Augmented and Virtual Reality event in May 2017 was to highlight some of the UK's most exciting current work in immersive audio research, development and content design. It brought together over 100 leading researchers and practitioners from academic, industry and creative sectors and provided an overview of the immersive audio creative process, considered application workflows and the potential for new developments. It also created a platform for demonstrating some of the most exciting immersive audio experiences created so far by leading teams around the UK. We were asked to organise this workshop by ImmerseUK - Immerse UK is supported by Innovate UK and managed by the Knowledge Transfer Network (KTN) as a Special Interest Group (SIG).
Year(s) Of Engagement Activity 2017
URL http://www.immerseuk.org/new-blog/2017/7/19/the-importance-of-audio-in-augmented-and-virtual-reality
 
Description York Talks - Sound Design for our Sound Environment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact YorkTalks is the University of York's own TEDx style presentations of key research. This talk resulted in a piece of written work for 'The Conversation' which reached a much wider international audience.
Year(s) Of Engagement Activity 2015
URL https://theconversation.com/shaping-up-our-soundscapes-can-improve-our-lives-37175