3D Audio Interface for Exploration of Audio Collections

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Electronic Eng & Computer Science

Abstract

A large number of people own a portable digital audio player - in a recent market report by In-Stat the worldwide unit sales were predicted to be 286 million by 2010. Music collections on these devices are also sizeable, typically with many 1000s of songs. The songs are grouped together in various ways: historically as albums, but more recently in playlists either generated automatically by software, or generated by the user. The challenge with large groups of songs is to navigate effectively though the group, or playlist, to select the song, or songs of choice.Currently users interact with playlists through a visual interface - the screen on the front of the device. Proposed in this application is a purely audio means of interacting with a playlist. The user browses through the songs in the playlist, listening briefly to each and selecting the songs of choice. This is achieved by using 3D audio technology, which plays several songs simultaneously as if they were originating from different spatial locations. From trials of an early-stage prototype of the technology it has been established that 4 songs playing at the same time, located using virtual acoustics in different virtual positions around the listener's head, is the optimal set up. The user can then easily navigate around the 3D virtual sound space, either using buttons on the device, or by pointing and tilting the device appropriately, if it contains an accelerometer (or similar), as increasingly many devices do, including the iPhone and iPod Touch.The funding will be used to further develop the technology, ultimately so that it runs on a stand-alone hand-held device. This can then be demonstrated to potential licensees. We also intend to release a limited functionality version free from our isophonics web portal, which currently offers our SoundBite automatic playlisting tool. This limited functionality release will enable potential customers and end-users to trial the software before purchase. Additionally, it will enable us to engage with end-users, seeking their feedback for improving the software.The technology will be commercially exploited by licensing the software as an API. One of the commercial issues addressed by the work plan presented in this proposal is whether a spin-out company should be established as a licencing vehicle, or whether direct exclusive licensing to an major mobile handset manufacturer, such as Nokia is more appropriate.

Publications

10 25 50
publication icon
Stewart R (2011) Exploring Music Contents

publication icon
Stewart R (2011) An Auditory Display in Playlist Generation in IEEE Signal Processing Magazine

 
Description The project demonstrated the utilities of spacial audio rendering techniques such as Ambisonics in audio content management and navigation. It was found that positioning audio items organised by similarity in a three dimensional sound scape provides an immersive environments that facilitates access to audio collections. This effect was demonstrated using listening tests for music collection navigation. It was also found that there is an upper bound on the number of sound sources listeners can successfully localise and use in a 3D navigation environment.
Exploitation Route The project outcomes may be utilised in audio content management and delivery platforms, audio archives and libraries, immersive interactive environments, 3D multimedia production and delivery, mobile platforms, as well as applications that require hands-free interaction, or applications that require audio only interaction, for instance, audio content management tools for the visually impaired. The amblr, a mobile spatial audio music browser was created to exploit research findings in further research or commercial applications including mobile devices. This was demonstrated at the 2011 IEEE International Conference on Multimedia and Expo. A software library facilitates the exploitation of the outcomes in future applications.
Sectors Communities and Social Services/Policy,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Healthcare

 
Description An evaluation licence was taken out by Rocket Pictures. This is for the software library developed under the project, which renders multiple audio streams in 3D virtual space over headphones. Their intended application is confidential to Queen Mary.
First Year Of Impact 2015
Sector Creative Economy
Impact Types Economic

 
Title Amblr: C++ library for rendering multiple sound sources over headphones with "out of head" localisation 
Description The library takes in several audio streams (typically songs) and renders them in 3D audio space using binaural technology. Up to 4 simultaneous streams are allowed (the upper limit is imposed by human hearing constraints on paying attention to multiple simultaneous sources) and these can be moved around in virtual 3D space, e.g. with external controls such as with a mobile computing platform. 
Type Of Technology Software 
Year Produced 2012 
Impact It has been licensed for evaluation to Rocket Pictures.