Reality Remix

Lead Research Organisation: Coventry University
Department Name: Ctr for Dance Research

Abstract

The 'Reality Remix' project brings together an interdisciplinary team of experts - a fashion designer, artists, researchers, and a choreographer to address challenges and opportunities that emergent technologies bring to content creation and interaction methods in Mixed, Augmented and Virtual Reality. The participants will debate, create and share understandings to reveal insights into the new arena of immersion. The project will explore how these 'New Realities' (Huff Post, 2016) can enrich cultural and commercial space, shedding light on ideas of interface design, display methods, and understandings of how we may inhabit, move in and around these new spaces.

The team will host three Research Salons under the call themes of Memory, Place & Performance. The aim is to enhance user experience and produce new understandings and insights through the creation of proof-of-concept prototypes in the development of next generation immersive experiences. The project will test a variety of prototypes culminating in dissemination events at Ravensbourne University and Siobhan Davies Dance where the research findings will be shared with the public. The aim is to enhance the interaction experience for audiences to produce new experiences which are intellectually and socially engaging and can, through a re-engagement with the complexities of encountering 'new spaces' generate new understandings about our relationship with our perceptions within the world and how the physical body is reconceptualised and re-encountered in mixed reality environments. These interactions will be accessible, reaching parts of the world where VR and AR have not yet been experienced (through connections and international contacts and links of the collaborators).

The interdisciplinary project will be of benefit to researchers and practitioners in a wide range of academic disciplines from specialists in movement - both performance and media arts based to those working with digital technologies, simulation and games, to social scientists interesting in understanding modes of behaviour in virtual environments, serious gaming and to cultural theorists interested in the creation of new spaces.

The research questions are:

How do we guide audience expectations for interactions in (mixed reality, VR, AR) space?

How is the figure represented, what behaviours surface from modes of interaction?

Can we elicit visceral and physical responses from our players, visitors and performers?

How can physically present and offsite viewers and performers be represented and acknowledged in shared virtual space?

Is the presence of other viewers or users of benefit in a live event?

Can we affect preconceived notions from audiences used to scripted theatre, video games or escape rooms?

What information do performers need to guide the audience?

What are the politics and aesthetics of cultural expression in VR from a global point of view?

Can an auditory, locative project augment one's sense of time, place and memory?

Can abstract virtual environments give the user useful data through visualisation or sonification?

Planned Impact

The project will benefit a range of external users. It builds on longstanding relationships between the participants. The project's cultural, social and educational impact will be delivered through two dissemination events and the project website. The project beneficiaries for non-academics include:

Dance Community
- A dissemination event at Siobhan Davies Dance (SDD) exposes the project to the dance professional community, addressing themes of performance, memory and place in relation to immersion. An investigative arts organisation, Siobhan Davies Dance stimulates new thinking based on a contemporary definition of choreography this event will give an opportunity guests to try the prototypes and will nicely follow on from Gibson's Virtual Reality Workshop at SDD. Gibson is also regularly invited to speak to the dance/art community on the subject of performance and new media both in the UK and abroad, lectures which span academia, creative economies and industry.

Creative Economy in the UK and beyond
- Project Partners include Paul Callaghan, Programme Manager, Games & Interactive, at the British Council. He develops new collaborations, innovation and enterprise with rapidly-changing creative and cultural industries worldwide. The insights generated during the project could feed into new ways of collaborating, working and commissioning for the communities of future leaders that the British Council nurtures

- Project Partners include Lauren Wright at Siobhan Davies Dance -a forward looking dance and theatre companies may wish to expand their practice into these new performance 'spaces' opened up by the project.

Invited to the dissemination events are also curators and arts programmers at galleries and festivals who wish to gain further understanding of next generation immersive environments and how they can be used to create artistic experiences and engage a widening audience - for example invited will be both Gazelli Art House and Gossamer Fog galleries who have recently begun exhibiting virtual reality artworks.

Commercial
Project Partners include Ads Reality, who develop immersive and augmented reality experiences and are looking for fresh ideas and techniques that can be developed into future commercial projects.

Medical
- Invited guests to the dissemination events will be Caroline Moore and her Great Ormond Street Hospital (GOSH) Arts Programme colleagues, who have been actively seeking interactive and immersive experiences to use with long term, or isolated, young patients in the Hospital. (Gibson/Martelli have recently been artist researchers-in-residence at GOSH developing an augmented reality project for the patients to examine the benefits of performance in a clinical settings: www.gosh.nhs.uk/parents-and-visitors/gosh-arts/news/gibsonmartelli-great-ormond-street-hospital)

General Public
- Members of the public who will be able to engage in new artistic immersive experiences, experience new forms of performance - it is clear from the media attention that there is widespread public interest in this field.

Academia and Creative Economy
A dissemination event at Ravensbourne contextualises the project in relation to new media, research and contributes to their overall research culture. Project Partner Nick Lambert is Head of Research at Ravensbourne in the VASARI Research Centre. Outside Ravensbourne, as chair of the Computer Arts Society (CAS), Nick is well placed to both foster new collaborations in the short and medium term whilst whilst developing opportunities to disseminate the project at future events, the Electronic Visualisation in the Arts conference to give one example.

- Gibson has presented at both of the VR UK conferences held at Ravensbourne, which are aimed at technologists, artists and content creators. She will introduce initial project at the 2018 VRUK conference and present the results in 2019.

Publications

10 25 50
 
Title Drawing Levels 
Description Virtual Environments A set of 4 linked environments rendered in a non-photorealistic manner from 3d models produced by a VR drawing package by a dancer using her feet to draw. The foot sketches resemble brain-shaped tumbleweed, skeletal forms which became superstructures in virtual environments, colourised using translucent shaders which are programmed to animate individual vertices, creating undulating movement giving usually rigid models a softer organic feel. The drawings experienced at different scales, small and jellyfish-like or gigantic scale, the human action creating a horizonless landscape, Discarding conventional 'landmarks' creates a scaling effect for the user who may feel miniaturised in a petri dish of seaweed-like forms - the set of 4 linked environments stylistically rendered in a non-photorealistic manner which adds another layer of abstraction. The pilot project, as yet unexhibited has had many rounds of testing with users leading to a simple control scheme where the user is piloting a submarine or spaceship-like craft that glides through a viscous medium. 
Type Of Art Artefact (including digital) 
Year Produced 2018 
Impact Has not been exhibited yet 
 
Description Reality Remix explored how we move in and around the new spaces that emergent technologies afford, through the development and examination of a group of proofs-of-concept and prototypes initiated from notions of Memory, Place and Performance. A team of artists, computer programmers, fashion and game designers, uncovered the mysteries of these new encounters, focussing on human behaviour, modes of moving, and kinaesthetic response. 'Reality Remix' took a unique dance perspective in the form of somatic enquiry to question concepts of embodiment, sensory awareness, performance strategy, choreographic patterning and value of touch in virtual worlds.

I held three Salons, the first at Abertay University in Dundee, the second at the Centre for Dance Research, Coventry University and the third at Studio Gibson/ Martelli in East End of London. We developed the beginnings of a kinetography for Virtual Reality (VR) through an expanded drawing practice, observations of restricting dancers, using the controllers in unexpected ways, by others and on the feet stepping into other bodies and characters, navigating data spaces. In the summer of 2018 two showcase events brought the project to the public for the first time at Ravensbourne University and Siobhan Davies Studios. In winter 2018 the work was demonstrated at The Creative Economy's Next Generation Immersive Experiences Showcase at University of York.
Exploitation Route The team propose to stage a Mixed Reality version of the 'Dazzle Ball'. Held in 1919 by the Chelsea Arts Club, first world war dazzle-ship patterns inspired this costume ball. Black-and-white patterns were applied to the costumes and set design that played with the audience's vision and perception. By embracing participatory forms of theatre and the inclusion of Mixed Reality/VR/AR, we are situating the 2019 Dazzle Ball in the 21st century, to bring to live colourful net-art and post-internet aesthetics. Audiences will be able to playfully experience enhanced perception in changing virtual landscapes: Able to lend costumes, they partake in dance performances that would allow them to explore and celebrate the dazzling and stimulating effect of new technologies in a performance and dance spree. A handful of professional performers will aim to transmit their movement onto the audience and collaborative action will diffuse the separation of stage and spectator. This uncommon set-up of VR multi-player interaction was tested in a first workshop at Siobhan Davies Dance in July 2018 and showed great first successes. With Dazzle Ball 2020 our aim is to provide inclusive VR experiences, catering not only for able bodies but by designing an environment that challenges perception and use of technology for all. Our proposal is a playful and needed rendering of the Mixed Reality playspace towards a more body-centric and multiplayer experience that celebrates technology for its magical world-opening possibilities that are changed by interaction and input rather than mere consumption. Dazzle Ball 2020 will be a coming together of the latest technology, dance performance and an active audience -a truly ambitious project taking us into the next generation of immersive experience.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections,Other

URL http://gibsonmartelli.com/SpacePlace/category/reality-remix/
 
Description One of the team members has established an artist residency programme - 'The Xenoform Labs Residency is an invitation-only art residency program for new media artists from outside of the (San Francisco) Bay Area.'
First Year Of Impact 2018
Sector Creative Economy,Other
Impact Types Cultural

 
Description Alex Woolner - Creative Lead Ads Reality 
Organisation Ads Reality Ltd
Country United Kingdom 
Sector Private 
PI Contribution Networking
Collaborator Contribution Submit ideas and prototypes, assessing and guidance towards commercialisation routes
Impact Reality Remix Salons
Start Year 2018
 
Description Lauren Wright - Programme Director Siobhan Davies Dance 
Organisation Siobhan Davies Dance
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Networking
Collaborator Contribution Studio space and marketing support
Impact Reality Remix Salons and showcase events
Start Year 2018
 
Description Nick Lambert - Head of Research Ravensbourne 
Organisation Ravensbourne College of Design and Communication
Country United Kingdom 
Sector Academic/University 
PI Contribution Networking
Collaborator Contribution Studio space, advice and support
Impact Reality Remix Salons and showcase events
Start Year 2018
 
Description Paul Callaghan - Programme Manager Games and Interactive British Council 
Organisation British Council
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Networking
Collaborator Contribution Advice and support
Impact Reality Remix Salon events
Start Year 2018
 
Description 'Altered States, Alternate Landscapes' - Virtual Reality day 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact As part of the Lightmoves Screendance Festival, Ruth Gibson and Bruno Martelli share their experimental Virtual and Mixed reality works with undergraduate students at Irish World Academy of Music and Dance, University of Limerick - Thursday 15 November
Year(s) Of Engagement Activity 2018
 
Description 'Altered States, Alternate Landscapes' Doing Day: Dance/Digital 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact From Siobhan Davies Studios website
'This day-long workshop invites you to actively engage in artists' research to explore the intersection between the body, movement and the digital.

In the morning's workshop, choreographer Janine Harrington will introduce digital concepts that she uses in making her work like coding, tracking, zooming and navigating. Through games and movement activities, you'll develop and use a set of tools for creating movement drawing on our culture's growing digital literacy.

In the afternoon, artists Gibson / Martelli will introduce tools for engagement with movement in augmented and virtual reality. You will be able to experience and experiment with performance technologies to discover how we respond to ourselves, others and environments in these 'new spaces'.'
Year(s) Of Engagement Activity 2018
URL http://www.siobhandavies.com/whats-on/workshops/doing-day-dance-digital/
 
Description 'Automated Away' Human - Computer Interaction Seminar (Seminar on People, Computers, And Design) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact 'Automated Away' Human-Computer Interaction Seminar (Seminar on People, Computers, And Design): William Gates Computer Science Building, Stanford University, Palo Alto

'Our engagement with digital technologies we disrupt, augment and interfere uncannily with our co-presence in virtual and real space causing us to constantly reassess our perceptions of space, time & place. Our recent artworks consider how we experience our bodies meeting virtual worlds.

By exploring how we move in and around the new spaces that emergent technologies afford we are engaged in new creative practices and models of performance that are emerging and transforming in this arena.

Virtual reality enables us to experience the body in different ways, on a different somatic plane inhabiting places and spaces as if they were physical, corporeal encounters. Automated Away unveils process in interdisciplinary art collaborations - exploring virtual travel through bodily navigations of real and simulated environments and performing data encryption resulting in rematerialisations of bodily data.

Since Terry Winograd began the seminar in 1990, speakers in the series have included Marissa Meyer, Mitch Kapor, Ted Nelson, Alan Kay, Jim Hollan, Hiroshi Ishii, Brewster Kahle, Bill Buxton, Brenda Laurel, Don Norman, Ben Shneiderman, Lucy Suchman, Andy Hertzfeld, Adele Goldberg, Randy Pausch, Joy Mountford, Pattie Maes, Bruce Tognazzini, Marc Andreessen, Judith Donath, Jakob Nielsen, Larry Page, and Will Wright.'
Year(s) Of Engagement Activity 2018
URL https://hci.stanford.edu/courses/cs547/speaker.php?date=2018-10-19
 
Description 'Reality Remix Presentation and Showcase' AHRC/EPSRC Next Generation of Immersive Experiences Showcase Event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact I was asked to make a 5 min presentation about my Reality Remix project at the AHRC's Next Generation Immersive Experiences showcase at the University of York, Department of Theatre, Film and Television in December. Alongside the other 32 award winners, I had the opportunity to show off some of the virtual experiences and make this 5 min slideshow and talk.
Year(s) Of Engagement Activity 2018
URL http://www.gibsonmartelli.com/SpacePlace/2018/12/11/reality-remix-at-the-ahrc-next-generation-immers...
 
Description ///SPACE///PLACE///INTERFACE/// 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Research Blog presenting information, news and current developments for projects including REALITY REMIX and Kinaesthetic Navigations. Covering areas including but not limiting to: Movement, Virtual, Augmented and Mixed Reality, Tracking, Mapping, Mocap, Game Engines, Tutorials, Immersive Experience.
Year(s) Of Engagement Activity 2010,2011,2012,2013,2014,2015,2016,2017,2018,2019
URL http://www.gibsonmartelli.com/SpacePlace/category/reality-remix/
 
Description Altered states, Alternate Landscapes' Professional Screen Dance Lab, Light Moves Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The Light Moves Screendance Lab is a three-day workshop that invites participants to experience the exciting potential of immersive technologies that are becoming ever more available across the arts and society. The Lab is open to dancers, choreographers, media artists and film-makers. It will introduce these technologies and open the door to the many possibilities that are emerging across and between these disciplines. 'Altered States, Alternate Landscapes' shares artists Ruth Gibson research and process in works such as Falling Upwards and MAN A VR. The workshop opens kinaesthetic awareness through self-exploration and active engagement in Motion Capture & Virtual Realities channelling the teaching and philosophy of Skinner Releasing Dance Technique. Participants will experience and experiment with performance technologies to discover how we respond to ourselves, others and environments in these 'new spaces'.
At Irish World Academy of Music and Dance, University of Limerick Monday 12 - Wednesday 14 November 2018
Year(s) Of Engagement Activity 2018
URL http://www.lightmoves.ie/light-moves-lab
 
Description An embodied turn: game studies across worlds and bodies 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact "An embodied turn: game studies across worlds and bodies" Darshana Jayemanne's Panel at 2018 Digital Games Research Association (DiGRA) International Conference, Turin, Italy.
Year(s) Of Engagement Activity 2018
URL http://www.digra.org/digital-library/forums/digra-18/
 
Description Reality Remix: Prototype Play 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Reality Remix: Prototype Play at The Roof Studio, Siobhan Davies Studios, St Georges Road, London- Saturday 28 July 2-5pm

'Mobile technology makes possible the mixing of real and virtual worlds, but who will colonise this space? Join a team of artists, designers and a choreographer to explore prototype technologies in mixed, virtual and augmented reality for experimentation with movement and performance.'
Year(s) Of Engagement Activity 2018
URL http://www.siobhandavies.com/whats-on/workshops/reality-remix-prototype-play/
 
Description Reality Remix: Showcase 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Ravensbourne University, Greenwich Digital Peninsula, London - Friday 27 July 2-5pm
Workshop and showcase to present to results of the Reality Remix Project.

'Mobile technology makes possible the mixing of real and virtual worlds, but who will colonise this space? Join a team of artists, designers and a choreographer to explore prototype technologies in mixed, virtual and augmented reality for experimentation with movement and performance. '
Year(s) Of Engagement Activity 2018
 
Description VR Salon at Xenoform Labs 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Xenoform Labs: 3338 23rd Street San Francisco, Saturday 20 October 12 - 5pm.
Xenoform Labs is a studio, a workshop space and art residency program. It is a new experiment run by Scott Kildall and showcases new media work in electronics, virtual reality, digital fabrication and more. The VR salon was an event to allow visitors to experience Ruth Gibson's experimental virtual reality works, share the process and discussion.
Year(s) Of Engagement Activity 2018
URL https://xenoformlabs.com/art-residency/