Rendering object-based audio in a domestic environment
Lead Research Organisation:
University of Surrey
Department Name: Sound Recording
Abstract
Creating immersive media experiences in the home is a topic popular in both academic and mainstream publications. The positive effects of spatialised audio on listener preference have been described in the literature, and technologies for producing 3D spatial audio experiences are well established, with channel-based systems being most commonly used. Whilst the effectiveness of these systems has been well demonstrated in laboratory and cinema-type environments, implementation in the domestic environment faces multiple obstacles including the seemingly poor tradeoff between the cost of additional loudspeakers and improvement in listening experience, a lack of knowledge and confidence in the setup of such a system, and a negative perceived effect on room aesthetics.
A recent survey, performed as part of this project, the results of which have been submitted for publishing, has determined that approximately 11% of UK households own some sort of multi-channel surround sound system and that more households own wireless and smart devices, despite their comparative youth as a technology. As such, a new type of system, recently described in a journal paper by the S3A Project as Media Device Orchestration (MDO), seeks to combine installed loudspeakers with these wireless loudspeakers, as well as any other potentially audio-capable devices, to create a new type of system in the domestic environment controlled by an intelligent renderer. This system would receive object-based audio, as opposed to channel-based audio, enabling the re-rendering of the sound scene specifically for the unique reproduction setup in any particular living room.
The focus of this project is on the development of rules for the intelligent renderer, incorporating factors including those in the following list: position, quality, number and type of loudspeakers; type of media content including audio-only and audio-visual; narrative intent; and needs of individuals and groups of listeners. To create these rules, listening experiments have been performed to compare various potential ways of re-rendering audio and the interaction of these different factors, using an MDO-type system installed in a simulated domestic living room environment. This has included textual response, individual rating and direct comparison type experiments.
A recent survey, performed as part of this project, the results of which have been submitted for publishing, has determined that approximately 11% of UK households own some sort of multi-channel surround sound system and that more households own wireless and smart devices, despite their comparative youth as a technology. As such, a new type of system, recently described in a journal paper by the S3A Project as Media Device Orchestration (MDO), seeks to combine installed loudspeakers with these wireless loudspeakers, as well as any other potentially audio-capable devices, to create a new type of system in the domestic environment controlled by an intelligent renderer. This system would receive object-based audio, as opposed to channel-based audio, enabling the re-rendering of the sound scene specifically for the unique reproduction setup in any particular living room.
The focus of this project is on the development of rules for the intelligent renderer, incorporating factors including those in the following list: position, quality, number and type of loudspeakers; type of media content including audio-only and audio-visual; narrative intent; and needs of individuals and groups of listeners. To create these rules, listening experiments have been performed to compare various potential ways of re-rendering audio and the interaction of these different factors, using an MDO-type system installed in a simulated domestic living room environment. This has included textual response, individual rating and direct comparison type experiments.
People |
ORCID iD |
Russell Mason (Primary Supervisor) | |
Craig Cieciura (Student) |
Studentship Projects
Project Reference | Relationship | Related To | Start | End | Student Name |
---|---|---|---|---|---|
EP/N509383/1 | 30/09/2015 | 30/03/2021 | |||
1806025 | Studentship | EP/N509383/1 | 02/10/2016 | 28/02/2021 | Craig Cieciura |
Title | A library of audio and audio-visual object-based audio content for testing and demonstration |
Description | Four audio-only and two audio-visual programme items were obtained, mixed and then captured in an object-based audio format. Alternate versions of these items were then created for demonstration and testing of a Media Device Orchestration type system. |
Type Of Art | Film/Video/Animation |
Year Produced | 2018 |
Impact | This content has been demonstrated to other researchers, industry and the general public on multiple occasions, sparking discussion and potential future implementations by industry. |
Title | A dataset of wireless loudspeaker placements recorded in real living rooms |
Description | A study was conducted in participants' homes to ascertain how they would position one to eight compact wireless loudspeakers, with the goal of enhancing their existing system. Consistencies were observed between the locations which can be used to estimate device positions for a given living-room layout. A database of the positions measured during the experiment was created for future use. |
Type Of Material | Database/Collection of data |
Year Produced | 2019 |
Provided To Others? | No |
Impact | None yet. |
Title | Dataset of object-based audio content including audio-only and audio-visual media |
Description | For use in developing, testing and demonstrating new object-based audio systems, a dataset of broadcast quality audio-only and audio-video content was produced with the assistance of a professional mixing engineer. |
Type Of Material | Database/Collection of data |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | Dataset used by postgraduate students at other universities. |
URL | https://zenodo.org/record/1404797 |
Description | Demonstration of audio and audio-visual content to attendees of an AES meeting at Southampton University |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Professional Practitioners |
Results and Impact | A regional Audio Engineering Society meeting was hosted by Southampton University. I was asked to give a brief presentation and provide a demo of content to attendees. |
Year(s) Of Engagement Activity | 2018 |
Description | Demonstration of audio and audio-visual content to delegates from multinational conglomerate company |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Introduction to project aims and demonstration of content followed by discussion of some technical aspects and potential applications. |
Year(s) Of Engagement Activity | 2018 |
Description | Demonstration of audio and audio-visual content to media installation company director |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Industry/Business |
Results and Impact | Demonstration of produced content to director of business operating in this area, as a means of introducing and discussing the research project. |
Year(s) Of Engagement Activity | 2018 |
Description | Demonstration to research associate from other department at Surrey University |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Other audiences |
Results and Impact | Demonstration of content and introduction to project aims to one associate researcher from other University department. |
Year(s) Of Engagement Activity | 2019 |
Description | Poster presentation at event organised by The Institute of Directors |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Public/other audiences |
Results and Impact | At an event organised by The Institute of Directors, University of Surrey student section, work was presented to members of the public, representatives from local industry and other postgraduate students. |
Year(s) Of Engagement Activity | 2020 |
Description | Presentation used by researcher in related project to introduce participants of related study to aims of a workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Study participants or study members |
Results and Impact | Presentation was used by an associated researcher to introduce study participants to ideas used in workshop for audience members and producers. |
Year(s) Of Engagement Activity | 2019 |