Immersive Pipeline: Production pipelines and translators for the authoring, sharing, and touring of immersive media performance works

Lead Research Organisation: Goldsmiths College
Department Name: Computing Department

Abstract

Immersive media and virtual reality are not the same thing. While VR brings with it the image of experiencing virtual worlds through 3D headset displays, the actual use of such displays can be physically disorienting and socially isolating. Immersive audiovisual experiences can be created in public space and in performance by the seamless projection of data, graphics, and video on all surfaces of a performance space, and by the use of multichannel surround sound. Immersion can take place in real space and be shared with others as an embodied collective experience. Goldsmiths' Sonics Immersive Media Labs (SIML) is part of an international network of immersive projection venues including rectangular surround video facilities like the RML CineChamber (San Francisco), domes such as the SATosphere (Montreal), or new formats such as the ISM Hexadome (Berlin). Whilst these facilities exist around the world, there has yet to be established a standard working method for the creation of new cultural works for them. Artists find themselves dedicating an inordinate proportion of their creative time understanding how to configure their work to fit an immersive space, often reinventing the wheel. Once a work has premiered, they find it difficult to tour it out of lack of compatibility across these venues, resulting in inadvertently site-specific work.

We are experiencing a teething period in immersive media not dissimilar to the early days of cinema, before a lingua franca of authoring method and technical process was established. The Immersive Pipeline project will partner with these leading cultural sector institutions above, as well as with immersive technology companies Derivative and NSC Creative, to form a partnership to design and develop an artist-facing immersive media production workflow. The project will engage with artists, scientists, presentation venues, and technology companies in participatory design sessions to define best practice for creative immersive media production. We will produce an immersive content authoring pipeline and make it available to artists as a method and process with which to create new works for multiscreen spaces. By partnering with the Elizabeth Garrett Anderson Institute for Women's Health, the project will demonstrate the pertinence of artistic immersive visualization and sonification in the health and medical sector.

Finally, we will work with our partner venues to define the specification for, and prototype, a translation and format conversion technology allowing work created for one venue (for example a dome) to be automatically translated for projection in another (a rectangular venue). The results of the project will:
1.) Empower artists to produce works for a range of immersive venues,
2.) Advance the state of the art of projection mapping technology,
3.) Bring together a diverse group of cultural institutions through shared vision and technical interoperability, and
4.) Bootstrap an international consortium for future large research projects.
The results of the Immersive Pipeline project will be showcased in a public performance/screening event.

Planned Impact

As a pilot project, the impact of the 6 month project will be preliminary and indicative. As the artist and cultural venue beneficiaries described above gain the knowledge to produce and present work for immersive spaces, the potential for broader impact on industry and culture will become apparent. This includes impact in mainstream cinema, responding to interest expressed by large organisations like the British Film Institute (BFI) and industry companies like IMAX (https://www.wired.com/2017/01/imax-vr-theaters/). This has the potential to trigger a sequence of impact from practitioner and industry bodies leading to influencing policy at the governmental level. This upward arch starts with Immerse UK (www.immerseuk.org), followed by the Creative Industries Federation (https://www.creativeindustriesfederation.com/) to future policy briefs following on from NESTA's 2016 report, "The Geography of Creativity in the UK" and fulfillingthe creative economy elements in HM Government's 2017 green paper, "Building our Industrial Strategy."

Our work with the EGA Institute for Women's Health's Sonic Womb project will have an impact in society beyond the academic benefits described in the previous section, and point out the potential for immersive audiovisual projection to aid in the public communication of science leading to greater public understanding of health and well being.

A specific process of identifying and fulfilling impact is described in the Pathways to Impact section.

Publications

10 25 50
 
Title Le Loup, Lifting, and Myogram 
Description A gestural sound/image performance by Atau Tanaka and visual artist Lillevan 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2018 
Impact This was performed at the IX Symposium in Montreal and in SIML Goldsmiths 
URL https://ix.sat.qc.ca/node/879?language=en
 
Description There is no universal immersive format, it depends on the nature of the artists' project. Artists need access to facilities.
Exploitation Route Our tools and methods are openly available to communities of practitioners.
Sectors Creative Economy,Culture, Heritage, Museums and Collections

URL http://sonics.goldsmithsdigital.com/immersive-pipeline/
 
Description We created tools for immersive audiovisual performance that are used by artists performing internationally
Sector Creative Economy,Culture, Heritage, Museums and Collections
Impact Types Cultural

 
Description Immersive Pipeline 2: Out of the lab and into the wild
Amount £32,099 (GBP)
Funding ID AH/S010637/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 01/2019 
End 12/2019
 
Description Immersive Pipeline at Splice Festival 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact We presented the Immersive Pipeline project at the Splice Festival
Year(s) Of Engagement Activity 2018
URL http://www.splicefestival.com/line-up/immersive-pipeline/
 
Description Immersive Pipeline performance event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Performance event from the Immersive Pipeline project in the Sonics Immersive Media Labs (SIML) at Goldsmiths
Year(s) Of Engagement Activity 2018
URL http://sonics.goldsmithsdigital.com/immersive-pipeline/