ENSEMBLE Performing Together Apart: Enhancing Immersive Multi-Location Co-Performance in Real Time

Summary
The feeling of being immersed within a live performance by an ensemble of world class musicians can be a deeply engaging and highly valued cultural experience for audiences, as well as for the performers themselves. Operas, orchestral performances and musical theatre are popular and attract large followings, but they are costly and highly resource intensive to produce, particularly when the most talented performers are often located in different locations across the globe. Advanced audio-visual streaming technology can potentially provide solutions and new creative opportunities, and Edinburgh Napier University is at the forefront of this experimental work. However, the ENSEMBLE project will provide a complementary, human-centred focus on relatively under explored experiential aspects.

The ENSEMBLE project will look at how musicians can perform live together, seamlessly as a group even though they are separated by distance, mediated through cutting-edge connection technologies. ENSEMBLE will examine the subjective, experiential and contextual factors that support high quality performance experiences, primarily from the perspective of the performers themselves, using qualitative and analytical methods.
With the advent of super low latency network connections, the possibility has emerged for musicians, singers and actors to perform together in multiple remote locations in real time. The latency aspect has been a particular challenge for musicians where even a lag of 30 milliseconds may have an impact resulting in a significant deterioration in performance. With the resolution of some of these latency technical hurdles, additional more experiential challenges have emerged regarding the nuance of representation of remote performance partners of both the co-performer(s) and their audience.

In its simplest terms, the question is how best should remote participants in a performance, be they co-performers (e.g. conductors and orchestras) or audience members, be represented to one another (visually, aurally and kinetically) to result in the most engaging, joyful and immersive experience for all involved?

The ENSEMBLE project seeks to answer this question by exploring the potential of emerging technologies such as multi-person immersive dome environments (IDEs) to enable, enhance, extend and immerse performers and/or audiences in real-time during multi-location and distributed musical and theatrical performances. This in turn will provide insight into the parameters of influence of remote co-performance on both the performer and audience experience, in turn enabling the establishment of UX (user experience) and technological frameworks informing best practice when designing, developing and deploying such technologies into temporary and permanent physical performance spaces and scenarios.

The research to-date shows that real-time performance is unquestionably possible over academic research networks and that, equally significantly, there is an appetite for this amongst the artists and organisations who would ultimately use it for economic or societal benefit.

While current knowledge might be satisfactory for masterclasses between academic institutions, ENSEMBLE's engagement with representatives from various areas of the creative industries will enable us to roadmap the technical and performance/place issues that need to be addressed if we are work towards commercial exploitation of real-time remote performance.

The identified users and beneficiaries of real-time remote performance can be grouped into two categories: artists/cultural organisations and technology innovation companies.This project will draw upon its existing industry networking links with these groups to form the core membership of the proposed ENSEMBLE workshops with additional membership through invitation during the project lifetime.

Who might benefit?
Edinburgh Napier University has a strong research relationship with the JISC Audio/Video Streaming team and was the remote partner in the UK's first demonstration of the LoLa audio/video streaming technology in 2012. Edinburgh Napier then became one of the founder members of the JISC Arts and Humanities Special Interest Group established that year. The membership of that group now reaches beyond education into industry and includes representatives from UK theatre and music groups. The list below identifies artists and cultural organisations that have either been involved with Edinburgh Napier's research or consultancy or have indicated their willingness to collaborate:

Royal Scottish National Orchestra, BBC Scottish Symphony Orchestra, Scottish Opera, The Rogue Orchestra. Q-Strings, Hebrides Ensemble with Sir James MacMillan, Steve Cradock (Paul Weller), Connect Theatre, Bristol Old Vic, Abbey Road Studios. Creative Scotland, Creative Carbon. Edinburgh International Festivals.

How might they benefit?
Short-term impact - The Rogue Orchestra approached Edinburgh Napier with a view to creating a competitive UK remote recording orchestra for music and film projects by international clients. Following positive market research amongst potential clients in the UK and USA feasibility testing has been undertaken with producer/guitarist Steve Cradock for projects by PP Arnold and Paul Weller. The ENSEMBLE project's collaboration with the technology innovation companies described below will help the orchestra and other UK artists to compete remotely in a global marketplace.

Mid to long-term impact - Companies such as the BBC, Scottish Opera and Abbey Road Studios have expressed interest in Edinburgh Napier's research and their participation in this project will allow us to roadmap the steps required before they can evaluate and commercially exploit remote real-time immersive performance.

The ENSEMBLE project will ensure impact through industry engagement workshops held at the mid and end points of the project lifespan. These multidisciplinary events will engage with artists, live and broadcast events practitioners and technology innovation companies to disseminate results of the project work and enable discussion and feedback.