Augmented Sonic Reality
Lead Research Organisation:
University of Nottingham
Department Name: School of Computer Science
Abstract
The sounds and noises of our urban landscapes provide the artist with a fertile resource with which to reflect society back at itself. By listening to these noises with greater scrutiny we can gain a better understanding of our environment and insights into possible futures.
In addition, urban landscapes are awash with hidden noises, such as network and communication data, subsonic and ultrasonic frequencies, radio signals, electromagnetic fields and the transitory data noise of its inhabitants.
This PhD will involve artist-led, and practice-based research and development into ways in which listeners can create personal, interactive and immersive mobile sound art experiences, or interactive soundscapes. It is proposed that these interactive soundscapes could enhance perceptions of interior and exterior urban environments and provide insights into, and a more accurate document of, our contemporary urban landscapes.
Detailed research will be undertaken into ways of effectively capturing and processing environmental and personal data sources for the purpose of its sonification and inclusion within the soundscape.
It is also envisaged that this project will involve the research and development of a wearable audio and data capturing interface that would facilitate an accessible and mobile audio experience. Such an interface could incorporate binaural recording and monitoring, along with environmental sensing and smartphone technologies.
Increasing the granularity of the data and the fidelity of the audio captured would build significantly on previous work in this area, expanding options relating to data mapping, processing and sonification, and resulting in a more detailed soundscape.
The personal sound art creation and performance experience that this project potentially offers, also offers exciting opportunities for the generation of new musical encounters. The empowerment of the listener as performer and the embracing of ambient noise are considered key compositional elements in the realisation of such new musical experiences.
In addition, urban landscapes are awash with hidden noises, such as network and communication data, subsonic and ultrasonic frequencies, radio signals, electromagnetic fields and the transitory data noise of its inhabitants.
This PhD will involve artist-led, and practice-based research and development into ways in which listeners can create personal, interactive and immersive mobile sound art experiences, or interactive soundscapes. It is proposed that these interactive soundscapes could enhance perceptions of interior and exterior urban environments and provide insights into, and a more accurate document of, our contemporary urban landscapes.
Detailed research will be undertaken into ways of effectively capturing and processing environmental and personal data sources for the purpose of its sonification and inclusion within the soundscape.
It is also envisaged that this project will involve the research and development of a wearable audio and data capturing interface that would facilitate an accessible and mobile audio experience. Such an interface could incorporate binaural recording and monitoring, along with environmental sensing and smartphone technologies.
Increasing the granularity of the data and the fidelity of the audio captured would build significantly on previous work in this area, expanding options relating to data mapping, processing and sonification, and resulting in a more detailed soundscape.
The personal sound art creation and performance experience that this project potentially offers, also offers exciting opportunities for the generation of new musical encounters. The empowerment of the listener as performer and the embracing of ambient noise are considered key compositional elements in the realisation of such new musical experiences.
Planned Impact
We intend the Horizon CDT to be the place where partners come to find their future employees and to engage with the opportunities and challenges of digital identity and personal data. The key beneficiaries of our research will be:
- Commercial private sector companies that will engage with our CDT students during their research and/or employ them after graduation. Our partners include companies developing digital identity technologies as well as user companies across a range of sectors (consumer goods, entertainment, transportation, energy and others).
- Public sector and third sector organisations that are concerned with the use of digital identities to support civil society including broadcasters, healthcare providers and campaign groups.
- The public whose personal data forms the focus of their research and who will ultimately use and come to depend upon digital identities.
- Research communities spanning computer science, engineering, psychology, sociology, business and humanities.
These will benefit in various ways.
- Commercial, public and third sector companies will benefit from being able to recruit from a pool of talented PhD graduates who bring an in-depth understanding of digital identity and a proven ability to work in interdisciplinary teams. They will also benefit from being able to participate in co-creation of PhD research to ensure focus on relevant challenges and be able to exploit results of this PhD research.
- The public will benefit through a greater understanding of the opportunities and challenges of digital identity.
- Research communities will benefit by opening up promising new interdisciplinary fields.
Our Impact activities will be driven by Professor Derek McAuley, the Director of Horizon, who has a track record of establishing industry labs, spinning our companies and who is currently acting CIO of the TSB funded Connected Digital Economy Hub. Key activities will be:
- All Horizon PhDs will be carried out in collaboration with an external partner who will be involved in drawing up the initial topic, recruiting students, shaping the PhD proposal, supervision, and hosting at least one internship
- We will continue to organise knowledge exchange events within Horizon that are open to our network of over 100 external partners, including our annual Horizon Research Conference.
- We will encourage the release of applications, open source software, and open datasets wherever collaboration agreements allow.
- We will actively encourage our students to spin-out new ventures, including providing seedcorn funding through Horizon.
- We will engage our students with our two partner catapults, the Connected Digital Economy Catapult and the Satellite Applications Catapult.
- We will actively encourage industry visits through guest lectures on our "Broadening Horizons" core taught programme.
- We will also encourage companies to define, steer and sponsor the first year interdisciplinary team projects.
- Our students will complete a module on Public and External Engagement and are encouraged to engage in public events and exhibitions.
- Horizon's journalist-in-residence will help expose students research to the wider world through regular blogposts, while the University's marketing and communications team will help them develop press releases.
- We will provide training in research publication as part of the Professional Skills module and mentor publications through the Practice Led Project and the annual writing retreat (where students present and critique draft papers).
These impact activities will be supported by a professional online presence with posters, demos and podcasts made available through our website and associated YouTube channel and twitter feed, and with individual PhD profiles being posted on our own site and on external networking portals such as LinkedIn and ResearchGate.
- Commercial private sector companies that will engage with our CDT students during their research and/or employ them after graduation. Our partners include companies developing digital identity technologies as well as user companies across a range of sectors (consumer goods, entertainment, transportation, energy and others).
- Public sector and third sector organisations that are concerned with the use of digital identities to support civil society including broadcasters, healthcare providers and campaign groups.
- The public whose personal data forms the focus of their research and who will ultimately use and come to depend upon digital identities.
- Research communities spanning computer science, engineering, psychology, sociology, business and humanities.
These will benefit in various ways.
- Commercial, public and third sector companies will benefit from being able to recruit from a pool of talented PhD graduates who bring an in-depth understanding of digital identity and a proven ability to work in interdisciplinary teams. They will also benefit from being able to participate in co-creation of PhD research to ensure focus on relevant challenges and be able to exploit results of this PhD research.
- The public will benefit through a greater understanding of the opportunities and challenges of digital identity.
- Research communities will benefit by opening up promising new interdisciplinary fields.
Our Impact activities will be driven by Professor Derek McAuley, the Director of Horizon, who has a track record of establishing industry labs, spinning our companies and who is currently acting CIO of the TSB funded Connected Digital Economy Hub. Key activities will be:
- All Horizon PhDs will be carried out in collaboration with an external partner who will be involved in drawing up the initial topic, recruiting students, shaping the PhD proposal, supervision, and hosting at least one internship
- We will continue to organise knowledge exchange events within Horizon that are open to our network of over 100 external partners, including our annual Horizon Research Conference.
- We will encourage the release of applications, open source software, and open datasets wherever collaboration agreements allow.
- We will actively encourage our students to spin-out new ventures, including providing seedcorn funding through Horizon.
- We will engage our students with our two partner catapults, the Connected Digital Economy Catapult and the Satellite Applications Catapult.
- We will actively encourage industry visits through guest lectures on our "Broadening Horizons" core taught programme.
- We will also encourage companies to define, steer and sponsor the first year interdisciplinary team projects.
- Our students will complete a module on Public and External Engagement and are encouraged to engage in public events and exhibitions.
- Horizon's journalist-in-residence will help expose students research to the wider world through regular blogposts, while the University's marketing and communications team will help them develop press releases.
- We will provide training in research publication as part of the Professional Skills module and mentor publications through the Practice Led Project and the annual writing retreat (where students present and critique draft papers).
These impact activities will be supported by a professional online presence with posters, demos and podcasts made available through our website and associated YouTube channel and twitter feed, and with individual PhD profiles being posted on our own site and on external networking portals such as LinkedIn and ResearchGate.
People |
ORCID iD |
Chris Greenhalgh (Primary Supervisor) | |
Laurence Cliffe (Student) |
Title | Horror-Fi Me |
Description | Horror-Fi Me is an audio augmented reality (AAR) iOS mobile application. It enables users to author and experience an AAR environment from within any indoor location. It enables the creation of an explorable and interactive cinematic soundscape. |
Type Of Art | Artefact (including digital) |
Year Produced | 2020 |
Impact | As of November 2020, this mobile application has logged over 60 downloads and installations. |
URL | https://horror-fi.me |
Description | An augmented reality software application has been developed, tested and analysed which demonstrates the potential for increasing visitor engagement with physical exhibition content and digital audio archival content by realising audio augmented objects. Audio augmented objects are physical objects that have had virtual audio content attached to them, and a collection of audio augmented objects can realise a virtual and physically explorable soundscape. Furthermore, audio augmented objects demonstrate themselves as useful additions to audio augmented reality experiences by helping to ground virtual audio content in physical reality and realising a heightened augmented reality experience. |
Exploitation Route | Insights, software and best practices could potentially be utilised by curatorial and creative practitioners to increase visitor exploration of, and engagement with, museum, gallery and heritage sites, their existing collections, spaces and artefacts therein. As well as providing potential interfaces to digital audio archive material. |
Sectors | Creative Economy,Culture, Heritage, Museums and Collections |
URL | https://doi.org/10.1007/s00779-020-01405-3 |
Description | Two public-facing sound art installations and a publicly available augmented reality mobile application have so far been realised using the findings and software developed as a part of this research. |
First Year Of Impact | 2020 |
Sector | Culture, Heritage, Museums and Collections |
Impact Types | Cultural |
Description | Audible Artefacts |
Organisation | Science Museum Group |
Country | United Kingdom |
Sector | Public |
PI Contribution | Creative and research based collaboration between MRL (University of Nottingham), Department of Culture and visual studies (University of Nottingham) and Nation Science and Media Museum (The Science Museum Group). This collaboration contributed to the research and development of an Audio Augmented Reality Mobile Application, a related visitor focus group study, a public sited sound installation, and a technical demonstration and presentation. |
Collaborator Contribution | The Science Museum Group supported the public engagement activities by hosting the related public engagement events, providing access to museum artefacts and curatorial and collections staff. UoN Department of Culture also provided assistance with public engagement activities. |
Impact | Science Museum Listening Session, Public and professional engagement/focus group demonstration, Public sound installation. |
Start Year | 2019 |
Description | Psychical Laboratory Sound Installation |
Organisation | University of Stirling |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Design and implementation of an audio augmented reality sound installation. |
Collaborator Contribution | The partners provided a platform and environment (a sound installation at a cultural festival event) to publicly deploy a recently developed interactive and immersive mobile audio application experience. |
Impact | This was a multi-disciplinary project across Computer Science (mobile and interactive audio application development) and cultural studies (researching and provision of audio archival content for use within the developed application). |
Start Year | 2021 |
Title | SoundScaper |
Description | A smartphone mobile application to situate virtual sound sources within physical space, through which participants can interact within an Audio Augmented Reality (AAR) experience, and with which curatorial and artistic practitioners can also author their own experiences. |
Type Of Technology | Webtool/Application |
Year Produced | 2019 |
Impact | It was used in two public engagement experiences at Nation Science & Media Museum, Bradford. |
Description | DMRN+ 12: Digital Music Research Network One-day Workshop 2017 |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Poster presentation at a digital music workshop, discussed ideas with both national and international academics and industry practitioners. |
Year(s) Of Engagement Activity | 2017 |
URL | https://www.researchgate.net/profile/Panos_Kudumakis/publication/323942840_DMRN12_Digital_Music_Rese... |
Description | Listening Session |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Study participants or study members |
Results and Impact | Focus group and study workshop on virtual soundscapes and Audio Augmented Reality held at the Science and Media Museum, Bradford, UK. Participation in the study prompted discussions around the technology's effective deployment within cultural spaces amongst participants. |
Year(s) Of Engagement Activity | 2019 |
Description | Presentation given at Live Cinema III Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | A presentation given on the subject of audio augmented reality at Live Cinema III conference. The presentation was accompanied by a demonstrative mobile application, which was made available for attendees to try prior to and after the conference event. |
Year(s) Of Engagement Activity | 2020 |
URL | http://livecinema.org.uk/live-cinema-conference-iii-the-reopening/ |
Description | Science Museum Blog Article |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Blog post for the Science and Media Museum website (guest author) about how the design of analogue music equipment influenced the online interactive experiments in the museum's Sonic Futures project. |
Year(s) Of Engagement Activity | 2019 |
URL | https://blog.scienceandmediamuseum.org.uk/listening-to-digital-dust/ |
Description | Science Museum Lates |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Public/other audiences |
Results and Impact | Public sound installation at National Science & Media Museum, Bradford. Experienced by 45 members of the public within the context of a free, open invitation, late night open gallery/museum event. The overall event was designed to engage public audiences with new research and technologies, the installation prompted discussions with many participants around the subject of Audio Augmented Reality. |
Year(s) Of Engagement Activity | 2019 |
Description | Talk and presentation given to Society of British Theatre Designers (SBTD) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | A talk delivered to approx 20 members of Society of British Theatre Designers (SBTD) on the subject (and recent research findings) relating to audio augmented reality. |
Year(s) Of Engagement Activity | 2020 |
URL | http://www.theatredesign.org.uk/news-story/launch-of-the-sbtd-new-collaborations-working-group/ |
Description | TechFast Talk |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Industry/Business |
Results and Impact | Morning presentation and subsequent discussion around the subject of Audio Augmented Reality (AAR) and presentation of recent research findings around virtual soundscape authoring and production. Presentation took place at a local design consultancy business, and was attended by their employees, along with others from other local businesses. Audience discussion was largely around possible commercial applications of the research presented. |
Year(s) Of Engagement Activity | 2020 |
URL | https://www.meetup.com/Notts-Techfast/events/268502129/ |