Immersive, Innovative, and Interactive Experience (IIIE): A Technology Driven London-Shanghai Partnership in Mixed Realities and Performing Arts

Lead Research Organisation: Goldsmiths University of London
Department Name: Computing Department

Abstract

This is an ambitious interdisciplinary collaborative project aiming to facilitate UK-Shanghai business in the theatre and performance space, focusing on immersive and mixed reality experiences, via curated networking and partnership building events involving creative IP companies, academics and technical partners in the UK and Shanghai, including a series of Investor Showcases to fund development of concepts/IP surfaced by project partners. Our UK-Shanghai Consortium will be led by Goldsmiths, University of London, and will include:
- leading UK arts, cultural, immersive and digital content partners
- leading Shanghai arts, tech, immersive and academic partners

The project will result in sustainable model boosting economic impact of immersive and strong legacy collaborative R&D space, where project partners will share interest in commercial distribution company to be set up in Shanghai. Other setting up one or more companies in Shanghai, we will also build:
- Goldsmiths Shanghai Research Centre
- Social VR and online digital platform for knowledge exchange and collaboration, incorporating immersive technology

Planned Impact

This project is built around creating direct impact for UK and Chinese creative industries. The primary beneficiaries will be the substantial list of project partners who participate in this project and their networks of collaborators. These will include the partners who have contributed letters of support to this project, but also further partners that we anticipate will take part in the project activities.

These creative industry partners will benefit through improved cross-national collaboration. In particular they will gain:
- A better understanding of the other nation's creative industry and market. This will be through presentations and networking at events.
- A greater understanding of how to do business and collaborate with the other country, also through presentation and networking
- Access to new collaborators in the other country. This will happen directly through networking opportunities at our events, in particular there will be frequent break-out group work that will facilitate working relationships
- Access to investment from both UK and China. This will be achieved through investor involvement in events and specific investor showcases.
- Access to markets in the other country. This will be through collaborations developed during the project, investment gained and better understanding of the other country. We will also set up a linked commercial vehicle to enable commercialisation of IP across nations.

These impacts can be measured in terms of financial value of investments (our minimum target is £10 million), number of collaborative ventures formed and the audience size for the creative outputs of those ventures.

While the immediate benefits will be to the participants, we do foresee a number of broader benefits.

The creative sectors of both countries will benefit more broadly. We aim to achieve a large enough scale of collaboration that its impact will spread to other companies. This will be due to creating a sufficient critical mass of individuals who have experience of cross-national collaboration and a more widely diffused understanding of cross-cultural working in both sectors. We expect the knowledge and collaborative relationships to spread through existing networks within the creative sectors of both UK and Shanghai (we have a number of partners with very strong networks). The project also includes the creation of durable institutions that will support further collaboration beyond the length of the project, including linked commercial vehicles in Shanghai and London to facilitate cooperation and IP exploitation and a joint research centre.

All of these impacts will have a positive economic impact on the UK creative sector and economy more broadly and strengthen the development of the Shanghai creative sector.

There will also be a positive impact on audiences in both countries through the creation of new creative work, and the greater creative innovation that comes from the cross-fertilization of ideas across cultures.
 
Description Research and teaching collaboration 
Organisation China National Academy of Arts
Country China 
Sector Academic/University 
PI Contribution Our project supported the administrative side of this collaboration.
Collaborator Contribution Professor Jonny Freeman's time was paid by CAA at a negotiated preferential rate.
Impact We collaborated with CAA for a joint course focusing on creative topics. A first pilot online course was conducted on Human-Computer Behaviour and Psychology at CAA, run by both their lead Lecturer and one of our project co-Is (Prof Jonny Freeman). The course included a group presentation with various topics such as unconscious design-affordance and behaviour, diversified representation of image symbols. Students participated to showcase their work as the outcome of the course. The goal of this collaboration is to provide students with a comprehensive understanding of the various creative topics through a combination of instruction and hands-on experience.
Start Year 2022
 
Description Research and teaching collaboration 
Organisation Shanghai Jiao Tong University
Country China 
Sector Academic/University 
PI Contribution The AHRC seeding fund was used to organise a trip to Shanghai, which led to multiple meetings with Shanghai Jiaotong University.
Collaborator Contribution They hosted our meeting in China, and supported our further AHRC grant application. SJTU has offered a visiting Professorship to Prof William Latham from Goldsmiths. Dr Pearl Wang from SJTU has collaborated with us on market research, studying immersive content consumption by Chinese audience.
Impact We have signed an MoU with Shanghai Jiaotong University as a direct outcome of the AHRC seeding grant.
Start Year 2019
 
Description AWE Asia Online Talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact PI Sylvia Pan and Co-I Marco Gillies gave an invited online talk at AWE Asia, a well-attended conference in China.
Year(s) Of Engagement Activity 2020
URL https://www.awexr.com/asia-2020
 
Description Beyond Conference - Creative China presentation and session MC 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact For the first time BEYOND, with the support of the AHRC's UK-China Creative Partnerships Programme, introduced Creative China, an exciting new track that explored research and industry partnerships between the creative industries in the UK and China.

Creative China featured conference sessions including live Q&A plus a showcase of projects collaboratively developed with UK and Chinese creative organisations. Delegates discovered Shaun the Sheep in Shanghai, how to Create Shared Experiences for Multicultural Audiences with speakers from the Science Museum, Aardman, the Shanghai Theatre Company, HTC amongst many others.

The IIIE project not only presented our research, the PI (Sylvia Pan) was the MC of the Creative China session. We estimate ~100 audience attended the Creative China session, and the video recordings from the session would reach to hundreds more.
Year(s) Of Engagement Activity 2020
URL https://beyondconference.org/creative-china
 
Description Frontiers Panel Discussion - XR for bringing people together in times of lockdown and post-COVID 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I hosted a panel discussion as part of Frontiers in VR online talk series.

One of them was particular related to China as we have invited the CEO of NReal glasses to join us discussing remote working. The video itself has been watched over 1,600 times as of March 2021
Year(s) Of Engagement Activity 2020
URL https://www.youtube.com/watch?v=nJ6ylZLwPkE
 
Description Immersive Assembly 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Held between October - December 2021, Immersive Assembly is a multidisciplinary, virtual residency bringing together an international cohort of artists and technologists

Immersive Assembly is a programme for artists and technologists to learn, share, network and play. The cohort consists of 9 emerging practitioners from Argentina, China, Ghana, Mexico, United Kingdom and Zimbabwe, curated in partnership with our project partners, British Council and Goldsmiths, University of London. The 3 month residency programme supported the artists in developing new ideas for immersive experiences that invite us to collectively share in rituals and connections.

This residency programme includes a series of sessions hosted by leading UK digital artists in Mediale's community, including art collective Marshmallow Laser Feast, artist and film-maker Kit Monkman, and Alexa Pollmann, director of performance design collective Peut-Porter. These sessions will inspire the group to further explore the theme of rituals and connection before arriving at their final idea, which they will pitch to a panel of experts from across the immersive arts industry in December.
Year(s) Of Engagement Activity 2021
URL https://yorkmediale.com/immersive-assembly/
 
Description Interview for national news - BBC Radio 3 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact I was interviewed at the BBC Radio 3, talking about using VR and AI to make technology more inclusive, also about the future of UK-China partnership. It was a 45 minutes interview recently podcasted, already resulting in emails asking me to author a book in this area and some keynote speak invitations.
Year(s) Of Engagement Activity 2020
URL https://www.bbc.co.uk/programmes/p085mn2k
 
Description Tedx talk: Social Interaction in the Era of Virtual Reality (also a version with Chinese subtitle) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The PI of the IIIE project, Sylvia Pan, has given a TEDx talk about social interaction in the time of virtual reality, where she promotes the value of research in VR, and her view of how VR is going to change our real lives. The talk received nearly 1000 views on YouTube.

Subsequently, supported by contacts developed as part of the IIIE project, we were able to produce a Chinese subtitled version of this talk and distributed on various Chinese social platforms. It has been viewed over 10,000 times in China. As a result, we have received very positive comments and invitations to collaborate.
Year(s) Of Engagement Activity 2021,2022
URL https://www.youtube.com/watch?v=rh6e-mW_y5E