Hearing History: Bringing to life the sounds of the past through Virtual Reality

Imagine experiencing a musical moment from the past; a rich, sensory experience that combines the flickering light from candles, the exquisite performance of centuries-old music, and the warm acoustics of a performance space that now no longer exists. That is what our initial project, Space, Place, Sound, and Memory, set out to do.

Working closely with game developers, musicologists, architectural historians, and acousticians, we set out to recreate the sounds and sensations of early music performance in virtual reality. We began by painstakingly piecing together two concerts of music from fragmentary sources and records, and then recorded historically-informed performances in an anechoic chamber, a space that is specifically designed to have no natural acoustic. Once complete, we constructed two virtual auditoria, digitally rebuilding St. Cecilia's Hall and the Chapel at Linlithgow Palace from detailed laser scans taken on site. Then, we brought the recordings and virtual spaces together using acoustic ray-tracing to recreate the sensations of hearing a historic performance.

The key outcome of that work was the production of virtual models and software which allows users to don VR headsets and step into the past. For our follow-on project, we would like to capitalise on our partnership with Historic Environment Scotland (HES) and St. Cecilia's Hall to create three dedicated public access points, situated at St. Cecilia's Hall, Linlithgow Palace, and the Engine Shed, HES's digital innovation centre. By partnering with HES, we get the greatest possible reach for our VR software: in 2017/18 Linlithgow Palace attracted 87,254 visitors from across the world. We plan to work closely with the organisation to measure impact in terms of visitor engagement using HES's established KPIs.

In addition to exploring the application of our acoustic modelling software in the heritage sector, we would like to use the follow-on project to explore the viability of our models and experience of anechoic recording in a commercial music production environment. Working with the Binchois Consort and the classical label Hyperion, we will arrange, record, and produce music linked, through detailed research, to Linlithgow Palalce, including lesser-known pieces from Scotland's famous Carvor Choirbook. Building on our VR production workflow, we will work with Hyperion's recording engineers to develop a commercial music production pipeline based around anechoic recordings, using the virtual acoustic models as a tool during production and post-production to situate those anechoic recordings within a historical acoustic space. This will be the first instance of a classical album recorded and produced entirely within VR.

We will develop a companion VR app that allows end-users to download additional musical content and VR models to remix and audition the recordings in different historic spaces and at different points in time. In terms of impact, we will capture data about Hyperion's ability to reach and engage its audiences in new ways and to grow new audiences, and the commercial viability of the app as a new revenue stream through creating and packaging additional paid-for content.

By the end of this phase of the project, we expect the following outputs:

1. Three public-facing VR installations located at high footfall cultural sites across the Scottish Central Belt;
2. A new commercial album, to be released on the Hyperion label, featuring the Binchois Consort performing the music of the Carvor Choirbook;
3. New commercial music workflows and production pipelines based around production and post-production using VR tools;
4. A user-friendly mobile app that allows end-users to remix audio in different spaces and at different points in time;
5. Datasets relating to the impact of the underlying research work on visitor engagement to cultural sites, and on the commercial operation of a record label specialising in early music production.

There are five key beneficiaries to this project:

1. Heritage organisations (St. Cecilia's Hall and Historic Environment Scotland (HES))
One of the key drivers of HES is in increasing the impact and value of the historic environment as a part of Scotland's economy, primarily through heritage tourism, but also through education. This is concretised in HES's organisational KPIs, which include:
- Increase the overall contribution of heritage tourism to the Scottish economy
- Increase knowledge and understanding of the historic environment
- Enhance engagement with the historic environment
- Provide excellent service to our visitors and service users
By creating a dedicated public installation that allows both the public and HES's specialist digital conservators to access these historic environments and their acoustics in new and engaging ways, our project will enable HES measurably to deliver on these KPIs. We will combine HES's own data gathering with user analytics and on-site interviews to capture data on visitor numbers and engagement, and we will work with specialists at HES to capture impact on knowledge and understanding.

2. Visitors to historic sites
VR and the attendant gamification of space and performance offers new modes of access and engagement for the public, who will further benefit from a better and more nuanced understanding of historic cultures and places. By locating our VR installations in high-footfall visitor attractions with international reach, we will ensure that we capture the broadest and most representative visitor engagement data possible.

3. Independent recording engineers and classical labels, like Hyperion
Traditionally, early music has been produced on-location in modern performance venues or churches. This is problematic because such spaces are not optimised for recording and production and are susceptible to high levels of external ambient noise, and such venues are often subject to restricted access during the day. This can lead to lengthy recording sessions and out-of-hours fees, increasing the cost of production. By carrying out a process analysis and practitioner interviews with Hyperion's sound engineers, we aim to measure the impact of our virtual production process on Hyperion's commercial workflow. We anticipate that it will lead to time and cost savings in production, and greater flexibility during post-production. In addition, we aim to measure the commercial impact of our companion app and downloadable content by carrying out a comparative analysis of revenues and profits of this release against previous CD releases by the Binchois Consort.

4. Performing musicians, like the Binchois Consort
Performing live ensemble vocal music in an anechoic chamber requires a very specific set of performance skills, since the recording environment exposes issues around timing, diction, intonation and tuning. We will train the Consort to adapt their technique specifically for performance and recording in anechoic spaces, and will follow up to investigate the impact, over time, of this training and experience on their performance practice in other contexts. Using comparative analysis of recordings and interviews with singers, we anticipate that the experience will result in a measurable improvement in the ensemble's technique.

5. Audiences of early music
All of the outcomes of this project offer new opportunities and new mechanisms for early music audiences to experience and explore music, and for this audience to be widened. Previously, these audiences have only been able to access early music through CD releases or live concerts. Our VR applications, particularly the companion app and downloadable content, will allow audiences to explore space as a dimension of performance in a way never before possible. We will use user analytics and audience interviews to measure the impact on audiences and how they interact with and understand the music they love.