Immersive VR and Interactive Machine learning: a London-Shanghai Collaboration in art, education, & AI

Lead Research Organisation: Goldsmiths College
Department Name: Computing Department

Abstract

Three activities proposed will be led by Goldsmiths experts in the area of virtual reality, immersive media, AI, art, education, and computer games, with the supports from other UK creative industry and Chinese organisations.

First, a 360 Video in Chinese will be produced by a team of UK-based and Chinese directors, led by UK academics and facilitated by a local Chinese school. The production of a proof-of-concept 360-degree video promotes the use of VR in language learning with Chinese students in London, and explores new ways in using VR in education.

Second, a one day London-Shanghai Art and Tech Partnership Development Event will be hosted at Goldsmiths this June. Supported by keynotes and demos (e.g., immersive VR demos, 360 videos), we will identify and invite 30 key partners to attend this event to develop a close conversation around London-Shanghai partnership and network development through a panel discussion and a workshop. One-to-one meetings will be organised for the following day to explore concrete future collaborations.

Third, a one week Shanghai trip will be co-organised with the Chinese partners, when UK academics will visit current our Chinese partners and any other shanghai- based partners developed throughout this project. PI and co-Is will prepare a presentation summarising themes and challenges identified in the Goldsmiths event and will also prepare separate talks closely related to their own research area (immersive art, VR, interactive ML and AI) where they see potentials for future collaboration.

Publications

10 25 50
 
Title 360 Film in Chinese Language Learning 
Description A 10 minutes 360 VR movie teaching where users can learn Chinese with other children. 
Type Of Art Film/Video/Animation 
Year Produced 2019 
Impact This piece was shown at our exhibition in Shanghai, and was well received by the audience. 
URL https://youtu.be/W4aSOujFDho
 
Description This grant has enabled us to make the important initial steps to develop a Goldsmiths based creative industry partnership network between Shanghai and London.
First of all, we have established a London-based network with Goldsmiths academics and the creative industries interested in working in China, some of them with existing experience and connections (UKIE, EF, etc.). We have also reached out and made initial connections to the head office of several top Chinese tech companies (Tencent, Alibaba, Huawei).
Secondly, through a 9-day Shanghai visit, with 6 people travelling, we established partnerships with three top Universities in Shanghai:
- Jiaotong University (ICCI, Department of Computer Science and Engineering)
- Fudan University (media department)
- Shanghai Theatre Academy
We met the following companies on site at their offices:
- Disney Shanghai (Shanghai International Tourism Resort, received by government officials)
- Mirror Pictures
- Hirain (http://www.hirain.com.cn/)
- Tencent Shanghai
- Education First Shanghai
- Lilac International
In total, during our China trip we have organised several workshop, meetings, and exhibitions:
- Meeting with Shanghai International Resort Tourism
- Workshop, meeting, and demo with ICCI, Jiaotong University
- Workshop, meeting, and demo with Department of Computer Science and Engineering, Jiaotong University
- Workshop and demo with Education First
- Workshop with Tencent Shanghai office (gaming department)
- Workshop at the Lilac International Commercial Centre on Financial data and music visualisations
- Visit and meeting with Hirain Company about collaborative VR project in theme park
- Visit and meeting with STA culture development Ltd.
- VR /AR exhibition open to the public (attended by over 800 people over 5 days at Lilacs Towers Shopping Centre. This event was featured on the two main Shanghai TV Channels)
We have identified a few ways to establish future collaborations:
- joint degrees and student and teaching exchange programmes
- co-productions of immersive experience
- post-production technology exchange
- co-organise workshop
Exploitation Route Academic route: as a direct result of this grant, we have won a further AHRC funding with international consortium. Planned academic pathway to impact activities include: research publications and workshops.

Non-academic route: We will be working closely with partners to explore exhibitions and business ventures.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education

URL https://www.panxueni.com/china
 
Description Immersive, Innovative, and Interactive Experience (IIIE): A Technology Driven London-Shanghai Partnership in Mixed Realities and Performing Arts
Amount £504,316 (GBP)
Funding ID AH/T011416/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 02/2020 
End 01/2023
 
Description Joint Research Centre 
Organisation MING GONG PICTURES
Country China 
Sector Private 
PI Contribution AHRC seeding grant enabled our visit to Shanghai for meetings with Mirror Pictures. We have also hosted workshop with them in London.
Collaborator Contribution Mirror Pictures hosted multiple workshop for us in China, and reimbursed our hotel.
Impact We have signed an MoU for a Goldsmiths Mirror Research Centre, based in the Disney International Resort in Shanghai.
Start Year 2019
 
Description Research and teaching collaboration 
Organisation Shanghai Jiao Tong University
Country China 
Sector Academic/University 
PI Contribution The AHRC seeding fund was used to organise a trip to Shanghai, which led to multiple meetings with Shanghai Jiaotong University.
Collaborator Contribution They hosted our meeting in China, and supported our further AHRC grant application.
Impact We have signed an MoU with Shanghai Jiaotong University as a direct outcome of the AHRC seeding grant.
Start Year 2019
 
Description Innovation in Immersive Art & Tech: Goldsmiths UK-China Workshop 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Over 20 UK and China based company attended the Workshop helped us to develop a UK-China immersive art & tech network. Many became our project partners for our AHRC UK-China grant 2020.
Year(s) Of Engagement Activity 2019
URL https://sites.google.com/view/ukchinags
 
Description Interview for national news 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact We were interviewed by multiple mainstream media during our Shanghai trip in July.
Year(s) Of Engagement Activity 2019
URL https://3g.163.com/v/video/VUJ9T000K.html
 
Description Interview for national news - BBC Radio 3 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact I was interviewed at the BBC Radio 3, talking about using VR and AI to make technology more inclusive, also about the future of UK-China partnership. It was a 45 minutes interview recently podcasted, already resulting in emails asking me to author a book in this area and some keynote speak invitations.
Year(s) Of Engagement Activity 2020
URL https://www.bbc.co.uk/programmes/p085mn2k