vRSP Follow-on

The existing vRSP project explores new, responsive, immersive, and interactive methods of experiencing a recorded performance, as well as allowing users to explore the unique heritage places for which these performances have been created. The existing vRSP network has created a prototype demonstrator of a new form of immersive site-specific music performance. In the existing prototype, a solo singer, moves through the space. Currently, the audience can only rotate around to watch and listen to this singer. If audience members move too close to the singer, the illusion breaks down, due to the quality and resolution of the capture. This follow-on bid proposes to work with a new industry partner, Dimension Studios, to polish this demonstrator to the highest possible standard in order to allow audiences to experience the ability to physically explore a virtual space and walk around inside a live music performance, giving audience a more direct experience than currently possible

One of the most experimental aspects of the existing vRSP project has been the integration of 360 video, mapped environment capture, performance, sound and volumetric multi-dimensional (aka holographic) video. The latter aspect has only recently become a viable 'audience proposition', as the recreation of a realistic yet synthesised human performer needs to adequately bridge the 'uncanny valley' to create a believable and acceptable experience for the viewer/listener/interactor. Indeed, one of the few examples of this integration so far is the Hold the World project for Sky/Natural History Museum featuring a virtually recreated David Attenborough demonstrating interactive animated exhibits in person and in vivo. Successful volumetric video capture could only be achieved in this project with the full power of a large consortium of professional production partners, plus the full technological might of Microsoft's volumetric capture studios in Redmond and their UK incarnation at Dimension Studios in Wimbledon. To do this process to an general audience-ready standard is time-consuming, complicated and resource intensive. For the initial phase of VRSP, we needed to experiment with the mix of immersive media and see if the overall impact was as hoped, particularly in terms of the harmonisation of imageries (and sound). We therefore adopted a 'proof of concept' approach to 4D volumetric capture using resources within University of Surrey's CVSSP to create a low resolution proxy of the final result. Having done this, we saw that the overall effect was absolutely worth pursuing, but only with the higher 'professional' resolution version of the capture, which was at that point beyond our means and scope. The partners have agreed that to make this mass audience-ready, we need to recreate the initial capture using the kind of technological resources which were available to the Hold the World project, which are fortunately relatively close at hand within Dimension Studios, who we have now signed up as a project contributor. The network can now massively benefit from their expertise, and with their input we an ensure that the outputs are a veritable worthy match of Price's amazing musical tapestries, and a great, high production value audience experience.

VR is a major opportunity for the UK's world leading creative industry --- already worth £84.1 billion and employing 2.8M people --- with investment of $4bn in immersive displays and a predicted global market size of $150bn for AR/VR content by 2020. vRSP collaborators, including creative leaders in Film, Broadcast, Video Games and Music, are ideally placed to exploit research advances for commercial, economic and cultural impact. Technologies for the creation of engaging digital actors with autonomous interactive behaviour will have widespread impact beyond VR/AR for video games, film production, education, training and simulation. To realise impact on the creative industries vRSP has established a strong stakeholder partner group with leading expertise from live performance, 3D Audio to VR production and creative tools.

As an intrinsically collaborative, co-created research network, vRSP anticipates developing a series of outputs which - individually and as a whole - will generate significant wider impact. Our impact strategy, has initially identified independent impact opportunities spanning societal, commercial and academic sectors, with our project partner helping to maximise opportunities. Our impact strategy will be re-assessed at the beginning and end of the project to ensure it meets both our research objectives and partner/wider industry need. Impacts and successes will be reported to the AHRC and vRSP partners as they develop. Our strategy will focus on 1) how best to identify and realise accessible benefit for vRSP partners from the creative tools techniques introduced, 2) implementation of experimental co-production of 3 prototypes with vRSP partners to demonstrate first-hand the step change in capability for VR/AR and immersive audio content, and 3) to disseminate these prototype through the partner's academic and industry networks to gather feedback and identify opportunities for future research. These core goals will be facilitated through a hands-on workshop with stakeholders, presentations to AHRC and at the culminating G3 Futures games conference. The prototypes will also be submitted submitted to major film/VR festivals (Caanes Marché du Film NEXT, Sundance NEXT, TriBeca Film Festival, Ars Electronica) and creative industry forums (SIGGRAPH, FMX, IBC). These presentations as well as network discussions will be used to gather evaluation feedback for inclusion in final publications, which will be submitted to music, digital arts, and VR publications.

The inclusion of the new partner, Dimesion Studios will extend the existing vRSP network focused on Audio research, into the visual domain, and create new links into the Digital Catapult network. This will greatly extend the UK reach and visibility of the project. The new volumetric video assets will be integrated into the existing prototype. This will require Dimension Studio's engineers to work directly with the vRSP researchers in order to exchange image and sound experience and techniques. This will benefit Dimension Studios as much as the vRSP network.

The new version of the experience will be presented to industry partners and invited guests at Abbey Road Studios in London. This is a professional setting, and one of the most highly respected recording studios in the world. Presenting the work at Abbey Road will have a much greater impact than presenting it on a university campus, and is more likely to reach future partners and distributors.


One of the core societal impacts of vRSP will be to initiate a rethinking of our means of conducting research between Academia and the Creative Economy. Rather than focusing solely on developing new technologies, vRSP aims to demonstrate that valuable insights can be gathered through close integration of Arts and Science.