City Immersive Sound and Movement Environment (CISME)
Lead Research Organisation:
City St George’s, University of London
Department Name: School of Arts
Abstract
This project will lead to the construction of an innovative, modular performance system - City Immersive Sound and Movement Environment - which will allow us to undertake research in audio-visual performance involving ambisonic sound, extended reality, and immersive projection. The use of digital techniques such as avatars, movement tracking and various forms of extended/virtual reality are becoming increasingly widespread in theatre and dance performance. In this system these will become allied to, and extend, our existing research strengths in sound practice and the manipulation of sound within different spaces.
The system will be portable, so that it can be deployed not only in City's internal spaces, but also externally, where it will underpin performance events in a variety of different venues. While previous research has made advances separately in the fields of 3D imaging and movement or immersive sound, the core motivation of CISME to explore the synergies of 3D movement, image, and audio is a novel goal, and one for which City and Urdang are uniquely qualified.
In addition to the cutting-edge research that the system will allow, we shall also use it as part of the training and development of our students in music, dance and musical theatre, so that they can gain experience of working with some of the latest digital techniques that they will increasingly find to be part of the professional environments to which they aspire in their future careers.
The system will be portable, so that it can be deployed not only in City's internal spaces, but also externally, where it will underpin performance events in a variety of different venues. While previous research has made advances separately in the fields of 3D imaging and movement or immersive sound, the core motivation of CISME to explore the synergies of 3D movement, image, and audio is a novel goal, and one for which City and Urdang are uniquely qualified.
In addition to the cutting-edge research that the system will allow, we shall also use it as part of the training and development of our students in music, dance and musical theatre, so that they can gain experience of working with some of the latest digital techniques that they will increasingly find to be part of the professional environments to which they aspire in their future careers.
People |
ORCID iD |
| Aaron Einbond (Principal Investigator) |
Publications
Carpentier T
(2023)
Spherical correlation as a similarity measure for 3-D radiation patterns of musical instruments
in Acta Acustica
Einbond A
(2022)
Embodying Spatial Sound Synthesis with AI in Two Compositions for Instruments and 3-D Electronics
in Computer Music Journal
Gioti A
(2022)
Composing the Assemblage: Probing Aesthetic and Technical Dimensions of Artistic Creation with Machine Learning
in Computer Music Journal
| Title | Cosmologies |
| Description | Cosmologies is a CD and digital audio recording released by record label all that dust in June, 2023. The techniques for spatial sound and binaural rendering can be attributed to our CResCa grant. |
| Type Of Art | Performance (Music, Dance, Drama, etc) |
| Year Produced | 2023 |
| Impact | The album release event welcomed a capacity audience at London's notable venue IKLECTIK and the album received a positive review from the blog "Boring like a drill" by Ben Harper, who described "essay of supreme restraint, producing the smallest possible swatches of attenuated sounds in their most muted colours." |
| URL | https://allthatdust.com/releases/cosmologies/ |
| Title | Prestidigitation II |
| Description | Prestidigitation II is a composition for Ensemble L'Instant Donné and Maxime Echardour, percussion soloist, composed by Aaron Einbond in 2023-24. Commissioned by Ircam-Centre Pompidou and Radio France for Festival Présences, it was premiered in Paris in 2024 and broadcast on radio station France Musique. The composition relied on research enabled by the CResCa grant. Radio France has released a video and 3-D binaural audio rendering of the work on their website and YouTube channel: https://youtu.be/eL6TooEwtHY |
| Type Of Art | Composition/Score |
| Year Produced | 2024 |
| Impact | The work was premiered in a sold-out performance at Ircam-Centre Pompidou and received a positive review in the online journal ResMusica: "Prestidigitation II by New Yorker Aaron Einbond is an extension for ensemble of Prestidigitation (2022) developed with Maxime Echardour around percussion instruments. The writing articulates sound figures and morphologies that allow us to appreciate the work of sound projection and spatialization emerging from artificial intelligence and the presence of a unique loudspeaker with 3-D radiation placed above the public's heads." (translation by Aaron Einbond) |
| URL | https://brahms.ircam.fr/en/works/work/61818/ |
| Title | Prestidigitation II video release |
| Description | Prestidigitation II is a composition for Ensemble L'Instant Donné and Maxime Echardour, percussion soloist, composed by Aaron Einbond in 2023-24. Commissioned by Ircam-Centre Pompidou and Radio France for Festival Présences, it was premiered in Paris in 2024 and broadcast on radio station France Musique. The composition relied on research enabled by the CResCa grant. |
| Type Of Art | Film/Video/Animation |
| Year Produced | 2024 |
| Impact | Following the sold-out premiere performance off Prestidigitation II, and broadcast by radio station France Musique, Radio France produced a high-definition video and 3-D audio recordings (available in binaural and 5.1 surround renderings) released publicly on the France Musique website and YouTube channel with 412K subscribers. |
| URL | https://youtu.be/eL6TooEwtHY |
| Description | This award led to the development and implementation of an innovative facility for research, teaching, and practice-based performance of immersive artwork. We have named the facility "SPARC Lab" in honor of the SPARC (Sound Practice and Research @City) research centre in which it is hosted in the Department of Performing Arts at City St George's, University of London. While these objectives were met during the project's short timeline, the longer-term outcomes are still ongoing, as researchers, students, and general audiences take advantage of the facilities to produce and experience future research outputs. |
| Exploitation Route | We have ambitious plans in the coming years to welcome academic researchers and students as well as non-academic artists and professionals to take advantage of SPARC Lab to produce new practice-based outputs, and to introduce non-academic audiences to the facilities through public concerts, workshops, and short courses. |
| Sectors | Creative Economy Digital/Communication/Information Technologies (including Software) Education Electronics Culture Heritage Museums and Collections |
| URL | https://sparc.london/sparc-lab |
| Description | The CResCa project enabled the launch of SPARC Lab for research in immersive sound and movement at City St George's, University of London. Since completion in 2023, SPARC Lab has hosted public concerts, workshops, seminars, an open house, a research symposium, and has supported staff to participate in further external conferences and festivals. SPARC Lab Launch in May 2024 was a highlight: six days of events animated by international immersive sound artists. These free events filled our facilities to capacity, with audience members citing "an incredible experience" and "hope for more similar events". Guest artists Natasha Barrett and Gerriet K. Sharma demonstrated their software in combination with SPARC Lab's unique equipment. We will follow up with SPARC Symposium 2025 with an ambitious schedule of events and international guests. Based on these events we can identify non-academic and academic impacts of audiences being introduced to new ways of experiencing 3-D immersive artwork through the project. We can also identify academic impact of researchers and students producing new musical work facilitated by the project. We expect to expand this impact over the course of this and coming years through our public concerts, workshops and symposia planned for 2025 and beyond. SPARC Lab contributes to the UK Government Mission to "break down barriers to opportunity at every stage". Participants from outside of HEI, including professional and non-professional artists, audiences, young people, and other community members, rarely have access to innovative equipment for 3-D immersive artwork such as that hosted by SPARC Lab. This is especially true for participants from less privileged backgrounds, including those from our local community in London. By engaging with these beneficiaries through free public workshops, concerts, and open houses we are expanding access to cutting-edge facilities for artistic experience and expression. This is helping to break down barriers and cultivate wider access for future generations of UK artists and audiences. |
| First Year Of Impact | 2023 |
| Sector | Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Culture, Heritage, Museums and Collections |
| Impact Types | Cultural |
| Description | Enhancing Research Culture (ERC) Fund |
| Amount | £5,000 (GBP) |
| Organisation | City, University of London |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 09/2024 |
| End | 07/2025 |
| Description | Higher Education Impact Fund (HEIF) for Knowledge Exchange and Impact |
| Amount | £1,800 (GBP) |
| Organisation | City, University of London |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 01/2025 |
| End | 07/2025 |
| Description | Research Impact Fund |
| Amount | £3,000 (GBP) |
| Funding ID | SPARC Lab launch |
| Organisation | City, University of London |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 02/2024 |
| End | 07/2024 |
| Description | Parnership with IKLECTIK Art Lab |
| Organisation | IKLECTIK |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | The CResCa project enabled the launch of SPARC Lab for research in immersive sound and movement at City St George's, University of London. Since completion in 2023, SPARC Lab has hosted public concerts, workshops, seminars, an open house, a research symposium, and has supported staff to participate in further external conferences and festivals. We have partnered with nonprofit organisation IKLECTIK Art Lab to curate, publicise, and produce several of these events. |
| Collaborator Contribution | Among the new partners and collaborators generated by the CResCa project is nonprofit organisation IKLECTIK Art Lab. Their in-kind contributions of publicity, technical expertise, and panel discussion participation at our SPARC Lab Launch in May 2024 generated a palpable energy by attracting full houses of non-academic listeners, representing a significant increase in the reach and diversity of our audiences. They will continue in their partnership relationship to co-produce concerts and workshops as part of the next SPARC Symposium in June 2025. They have continued to partner on several external grant applications which, if successful, would lead to further in-kind contributions in the future. |
| Impact | Outcomes include public concerts and workshops at SPARC Lab Launch in May 2024 and planned events in SPARC Symposium in June 2025. |
| Start Year | 2024 |
| Title | SPARCL |
| Description | The SPARCLab (or SPARCL) repository is a forum for users to share software for use with SPARC Lab https://sparc.london/sparc-lab for immersive sound, image, and movement based in the Department of Performing Arts at City St George's, University London and its research centre SPARC https://sparc.london/ funded by an AHRC CResCa (Creative Research Capability) grant. The repository, developed primarily by principal investigator Aaron Einbond, includes tools for Higher-Order Ambisonic (HOA) spatialization with a 24.4-channel Ambisonic loudspeaker dome, 9.1.6 Dolby Atmos system, IKO 20-channel spherical loudspeaker array, and EM64 Eigenmike. |
| Type Of Technology | Software |
| Year Produced | 2025 |
| Open Source License? | Yes |
| Impact | The SPARCL software repository has been central to producing public events with the SPARC Lab CResCa facilities. SPARCL software enables developing, testing, and producing live concert events with immersive sound and movement. SPARCL was made public in May 2024 so that community members both locally and internationally can benefit from the software and also make their own collaborative contributions. Significant development has continued on the software in academic year 2024-25 led by PI Aaron Einbond. |
| URL | https://github.com/einbond/SPARCL |
| Title | SPARCL |
| Description | This software is developed by PI Aaron Einbond for immersive audio applications with the CResCa project. While there are plans to release the software with an open source license through GitHub, it is currently still private while it is under development. |
| Type Of Technology | Software |
| Year Produced | 2024 |
| Impact | SPARCL will be used for future performances and practice-based artistic outputs taking advantage of the CResCa project. |
| URL | https://github.com/einbond/SPARCL |
| Description | From the City Studios 2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | The research organization hosted a public concert "from the City Studios" featuring the first public event with the equipment acquired through the CResCa project. Students and staff performed original practice-based research work enabled by the project. 50 guests attended including members of the general public and students and staff from the university. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.city.ac.uk/news-and-events/events/2023/may/from-the-city-studios |
| Description | SPARC Lab Launch |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | May 21-26, 2024 SPARCL Lab hosted a weeklong festival including "launch" events for the outcomes of the CResCa project. The festival, entitled "Improbably Networks" included concerts featuring practice-based creative work by students and staff, two concerts with world-leading international guest artists who are specialists in the research area of the project, and a symposium with international visitors on the related theme of improbable networks. Concerts and workshops over five days were sold-out audience with audiences of 50 guests each. The week included two workshops for non-HEI beneficiaries including young people from the local area and professional practitioners with little previous experience of the research area. These workshops serve as pilot studies for future ongoing partnerships to deliver further research impact. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://www.citystgeorges.ac.uk/news-and-events/events/2024/may/sparc-symposium-2024 |
| Description | SPARC Lab: 3D sound from the City Studios |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | Our postgraduate students presented new electronic music and sound, projected on CResCa funded SPARC Lab facilities including our 24.4-loudspeaker Ambisonic dome and the 3-D IKO loudspeaker array. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://www.citystgeorges.ac.uk/news-and-events/events/2024/december/sparc-lab |
| Description | School of Communication and Creativity: Community Open House |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | Sound Demo: Extended reality for the ears. Aaron Einbond, PI of SPARC Lab, led a demonstration situating listeners in the midst of a virtual acoustic environment. The demo featured SPARC Lab, our immersive 3-D sound and movement system hosted by the SPARC (Sound Practice and Research @ City) research centre in the Department of Performing Arts and generously supported by AHRC CResCa funding. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://www.citystgeorges.ac.uk/news-and-events/events/2024/may/school-of-communication-and-creativi... |