Immersive Pipeline: Production pipelines and translators for the authoring, sharing, and touring of immersive media performance works
Lead Research Organisation:
Goldsmiths University of London
Department Name: Computing Department
Abstract
Immersive media and virtual reality are not the same thing. While VR brings with it the image of experiencing virtual worlds through 3D headset displays, the actual use of such displays can be physically disorienting and socially isolating. Immersive audiovisual experiences can be created in public space and in performance by the seamless projection of data, graphics, and video on all surfaces of a performance space, and by the use of multichannel surround sound. Immersion can take place in real space and be shared with others as an embodied collective experience. Goldsmiths' Sonics Immersive Media Labs (SIML) is part of an international network of immersive projection venues including rectangular surround video facilities like the RML CineChamber (San Francisco), domes such as the SATosphere (Montreal), or new formats such as the ISM Hexadome (Berlin). Whilst these facilities exist around the world, there has yet to be established a standard working method for the creation of new cultural works for them. Artists find themselves dedicating an inordinate proportion of their creative time understanding how to configure their work to fit an immersive space, often reinventing the wheel. Once a work has premiered, they find it difficult to tour it out of lack of compatibility across these venues, resulting in inadvertently site-specific work.
We are experiencing a teething period in immersive media not dissimilar to the early days of cinema, before a lingua franca of authoring method and technical process was established. The Immersive Pipeline project will partner with these leading cultural sector institutions above, as well as with immersive technology companies Derivative and NSC Creative, to form a partnership to design and develop an artist-facing immersive media production workflow. The project will engage with artists, scientists, presentation venues, and technology companies in participatory design sessions to define best practice for creative immersive media production. We will produce an immersive content authoring pipeline and make it available to artists as a method and process with which to create new works for multiscreen spaces. By partnering with the Elizabeth Garrett Anderson Institute for Women's Health, the project will demonstrate the pertinence of artistic immersive visualization and sonification in the health and medical sector.
Finally, we will work with our partner venues to define the specification for, and prototype, a translation and format conversion technology allowing work created for one venue (for example a dome) to be automatically translated for projection in another (a rectangular venue). The results of the project will:
1.) Empower artists to produce works for a range of immersive venues,
2.) Advance the state of the art of projection mapping technology,
3.) Bring together a diverse group of cultural institutions through shared vision and technical interoperability, and
4.) Bootstrap an international consortium for future large research projects.
The results of the Immersive Pipeline project will be showcased in a public performance/screening event.
We are experiencing a teething period in immersive media not dissimilar to the early days of cinema, before a lingua franca of authoring method and technical process was established. The Immersive Pipeline project will partner with these leading cultural sector institutions above, as well as with immersive technology companies Derivative and NSC Creative, to form a partnership to design and develop an artist-facing immersive media production workflow. The project will engage with artists, scientists, presentation venues, and technology companies in participatory design sessions to define best practice for creative immersive media production. We will produce an immersive content authoring pipeline and make it available to artists as a method and process with which to create new works for multiscreen spaces. By partnering with the Elizabeth Garrett Anderson Institute for Women's Health, the project will demonstrate the pertinence of artistic immersive visualization and sonification in the health and medical sector.
Finally, we will work with our partner venues to define the specification for, and prototype, a translation and format conversion technology allowing work created for one venue (for example a dome) to be automatically translated for projection in another (a rectangular venue). The results of the project will:
1.) Empower artists to produce works for a range of immersive venues,
2.) Advance the state of the art of projection mapping technology,
3.) Bring together a diverse group of cultural institutions through shared vision and technical interoperability, and
4.) Bootstrap an international consortium for future large research projects.
The results of the Immersive Pipeline project will be showcased in a public performance/screening event.
Planned Impact
As a pilot project, the impact of the 6 month project will be preliminary and indicative. As the artist and cultural venue beneficiaries described above gain the knowledge to produce and present work for immersive spaces, the potential for broader impact on industry and culture will become apparent. This includes impact in mainstream cinema, responding to interest expressed by large organisations like the British Film Institute (BFI) and industry companies like IMAX (https://www.wired.com/2017/01/imax-vr-theaters/). This has the potential to trigger a sequence of impact from practitioner and industry bodies leading to influencing policy at the governmental level. This upward arch starts with Immerse UK (www.immerseuk.org), followed by the Creative Industries Federation (https://www.creativeindustriesfederation.com/) to future policy briefs following on from NESTA's 2016 report, "The Geography of Creativity in the UK" and fulfillingthe creative economy elements in HM Government's 2017 green paper, "Building our Industrial Strategy."
Our work with the EGA Institute for Women's Health's Sonic Womb project will have an impact in society beyond the academic benefits described in the previous section, and point out the potential for immersive audiovisual projection to aid in the public communication of science leading to greater public understanding of health and well being.
A specific process of identifying and fulfilling impact is described in the Pathways to Impact section.
Our work with the EGA Institute for Women's Health's Sonic Womb project will have an impact in society beyond the academic benefits described in the previous section, and point out the potential for immersive audiovisual projection to aid in the public communication of science leading to greater public understanding of health and well being.
A specific process of identifying and fulfilling impact is described in the Pathways to Impact section.
Organisations
- Goldsmiths University of London (Lead Research Organisation)
- Engineering and Physical Sciences Research Council (Co-funder)
- Society for Arts and Technology (Project Partner)
- Recombinant Media Labs (Project Partner)
- University College London (Project Partner)
- National Space Centre (Project Partner)
- Derivative Visual Thinking (Project Partner)
- Institute for Sound & Music (Project Partner)
Publications
Peng Y
(2019)
Body and Embodiment in Dance Performance
Gibson S
(2022)
Live Visuals - History, Theory, Practice
Title | Cave Music, with Uta Kogelsberger |
Description | Cave Music was the 3rd incarnation of this 8 channel sound installation of free standing, simulated "rock-speakers" spreading mixes of Kögelsberger's recordings captured in the mouth of a disused mine, creating a trancelike, rhythmic and ritualistic soundscape hovering between abstraction and dance track. Cave Music - a playful evocation of man's often futile attempt to control nature - took its cue from the increase of rainfall in the Lake District due to climate change. It followed on from this video installation Playing the Cave and Orchestra of Rocks not a Rock Concert. During an evening of live performances, experimental sound artists Atau Tanaka, Shelley Parker, Dane Law, and Chooc Ly created their interpretations of the original sounds. In the Battle of the Rocks, devised by Kögelsberger, the beat boxing champion Marv Radio challenged the Rocks to a battle between voice and environmental sound. |
Type Of Art | Artistic/Creative Exhibition |
Year Produced | 2019 |
Impact | Working with community arts organisation, Ugly Duck. http://uglyduck.org.uk/portfolio_page/rockconcert/ |
URL | https://utakogelsberger.net/rock-concert-cave-music/ |
Title | Gone Gone Beyond, by People Like Us |
Description | Gone, Gone Beyond is an immersive a/v spatial cinema work by People Like Us, which breaks the rectangle, smashing the thin screen into tiny fragments, looking beyond the frame, climbing through to see what's behind. Commissioned by Naut Humon, the founder of immersive theatre project CineChamber, Gone, Gone Beyond is a 10 screen/8 speaker work by Vicki Bennett with seamless wrap around projection and surround sound, where the audience sit inside. It uses edited collage sewn together in a giant patchwork. Pull on a thread and watch whole new narratives expand and unravel all at once on a 360 palette. The project has been a work in progress since 2017, and is currently 50 minutes long. History: the initial in-process tester movie screened in San Francisco in October 2017 at RML's own Recombinant Festival at Gray Area Foundation for the Arts. Since then we have been continuing to work on content, and staged a private event in April 2019 at Goldsmiths SIML to encourage some partner support in the UK/Europe. |
Type Of Art | Film/Video/Animation |
Year Produced | 2019 |
Impact | Working with the cultural sector: Sound & Music in the UK, and the Recombinant Media Labs in San Francisco USA |
URL | http://peoplelikeus.org/2019/gone-gone-beyond/ |
Title | Le Loup, Lifting, and Myogram |
Description | A gestural sound/image performance by Atau Tanaka and visual artist Lillevan |
Type Of Art | Performance (Music, Dance, Drama, etc) |
Year Produced | 2018 |
Impact | This was performed at the IX Symposium in Montreal and in SIML Goldsmiths |
URL | https://ix.sat.qc.ca/node/879?language=en |
Title | Rock Concert II with Uta Kogelsberger |
Description | Loosely based on the notion of a ritualistic "stop the rain dance" Rock Concert II is a visceral response to the increase of rainfall in the Lake District due to climate change. Recordings of individual raindrops captured by Uta Kögelsberger during heavy rainfall in the mouth of a disused mine, with experimental recording techniques are handed over to sound artists including Atau Tanaka, Will Edmondes, The Noise Choir and Mariam Rezaei. An 8 channel sound installation of free standing, simulated "rock-speakers" spread live mixes creating trancelike, rhythmic and ritualistic soundscapes hovering between abstraction and dance track. The work takes its cue from the previous video installation 'Playing the Cave'. |
Type Of Art | Artistic/Creative Exhibition |
Year Produced | 2018 |
Impact | Taking the work to northeast England. Working with local artists. |
URL | https://utakogelsberger.net/rock-concert-ii/ |
Title | Skin Awareness II by Youhong "Friendred" Peng |
Description | Dancer's body communicates with the immersive space extending the dancer's sensory. Skin-awareness explore the relationship between the body and space, and experiment with the capacity of the body while interfering with the digital technologies that change the surrounding space. The body is not only an entity, but also an action, an experience and an embodied perception. The immersive space morphs and alters with light and becomes solid, its pressure composing and decomposing the self-awareness of skin. The dancer's body is extended and manipulated as a conscious entity, exceeding the physiological object. In Skin-awareness, the embodied, symbiotic relationship of light, sound, and space is reciprocal. This work crosses the intersection of technological interventions, architectural fabrication, and contemporary dance performance. Forty laser beams controlled by mechanical robots read and encode choreography. Synergies of futuristic lighting and surrounding screens create a brief detachment from reality. Computer vision is utilized with computational precision along with the control of body movement. |
Type Of Art | Performance (Music, Dance, Drama, etc) |
Year Produced | 2019 |
Impact | Working with community arts organisation, Ugly Duck http://uglyduck.org.uk/portfolio_page/skin/ |
URL | http://friendred.studio/2019/04/28/skin-awareness-london-ugly-duck/ |
Description | There is no universal immersive format, it depends on the nature of the artists' project. Artists need access to facilities. |
Exploitation Route | Our tools and methods are openly available to communities of practitioners. |
Sectors | Creative Economy Culture Heritage Museums and Collections |
URL | http://sonics.goldsmithsdigital.com/immersive-pipeline/ |
Description | We created tools for immersive audiovisual performance that are used by artists performing internationally. The artists, People Like Us were able to use our immersive facilities at Goldsmiths to prepare their international US premier of Gone Gone Beyond in San Francisco USA at the Recombinant Festival in October 2019. This award enabled a collaboration with the music charity, Sound & Music, to support People Like Us' "New Voices" residency. http://peoplelikeus.org/tag/gone-gone-beyond/ |
First Year Of Impact | 2019 |
Sector | Creative Economy,Culture, Heritage, Museums and Collections |
Impact Types | Cultural |
Description | Immersive Pipeline 2: Out of the lab and into the wild |
Amount | £32,099 (GBP) |
Funding ID | AH/S010637/1 |
Organisation | Arts & Humanities Research Council (AHRC) |
Sector | Public |
Country | United Kingdom |
Start | 02/2019 |
End | 12/2020 |
Title | Immersive Pipeline whitepaper |
Description | This report examines immersive experiences in the context of social shared spaces. It presents the Immersive Pipeline research project led by Prof. Atau Tanaka and funded by the AHRC/EPSRC Research and Partnership Development call for the Next Generation of Immersive Experiences from January to June 2018. It covers a historical and technological overview of state of the art on this field, cases of study and interviews in depth with some of the participants. |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2019 |
Provided To Others? | Yes |
Impact | making available to other audiovisual and immersive practitioners |
URL | http://research.gold.ac.uk/id/eprint/28259 |
Description | Immersive Pipeline at Splice Festival |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | We presented the Immersive Pipeline project at the Splice Festival |
Year(s) Of Engagement Activity | 2018 |
URL | http://www.splicefestival.com/line-up/immersive-pipeline/ |
Description | Immersive Pipeline performance event |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Public/other audiences |
Results and Impact | Performance event from the Immersive Pipeline project in the Sonics Immersive Media Labs (SIML) at Goldsmiths |
Year(s) Of Engagement Activity | 2018 |
URL | http://sonics.goldsmithsdigital.com/immersive-pipeline/ |