Immersive VR and Interactive Machine learning: a London-Shanghai Collaboration in art, education, & AI
Lead Research Organisation:
Goldsmiths University of London
Department Name: Computing Department
Abstract
Three activities proposed will be led by Goldsmiths experts in the area of virtual reality, immersive media, AI, art, education, and computer games, with the supports from other UK creative industry and Chinese organisations.
First, a 360 Video in Chinese will be produced by a team of UK-based and Chinese directors, led by UK academics and facilitated by a local Chinese school. The production of a proof-of-concept 360-degree video promotes the use of VR in language learning with Chinese students in London, and explores new ways in using VR in education.
Second, a one day London-Shanghai Art and Tech Partnership Development Event will be hosted at Goldsmiths this June. Supported by keynotes and demos (e.g., immersive VR demos, 360 videos), we will identify and invite 30 key partners to attend this event to develop a close conversation around London-Shanghai partnership and network development through a panel discussion and a workshop. One-to-one meetings will be organised for the following day to explore concrete future collaborations.
Third, a one week Shanghai trip will be co-organised with the Chinese partners, when UK academics will visit current our Chinese partners and any other shanghai- based partners developed throughout this project. PI and co-Is will prepare a presentation summarising themes and challenges identified in the Goldsmiths event and will also prepare separate talks closely related to their own research area (immersive art, VR, interactive ML and AI) where they see potentials for future collaboration.
First, a 360 Video in Chinese will be produced by a team of UK-based and Chinese directors, led by UK academics and facilitated by a local Chinese school. The production of a proof-of-concept 360-degree video promotes the use of VR in language learning with Chinese students in London, and explores new ways in using VR in education.
Second, a one day London-Shanghai Art and Tech Partnership Development Event will be hosted at Goldsmiths this June. Supported by keynotes and demos (e.g., immersive VR demos, 360 videos), we will identify and invite 30 key partners to attend this event to develop a close conversation around London-Shanghai partnership and network development through a panel discussion and a workshop. One-to-one meetings will be organised for the following day to explore concrete future collaborations.
Third, a one week Shanghai trip will be co-organised with the Chinese partners, when UK academics will visit current our Chinese partners and any other shanghai- based partners developed throughout this project. PI and co-Is will prepare a presentation summarising themes and challenges identified in the Goldsmiths event and will also prepare separate talks closely related to their own research area (immersive art, VR, interactive ML and AI) where they see potentials for future collaboration.
| Title | 360 Film in Chinese Language Learning |
| Description | A 10 minutes 360 VR movie teaching where users can learn Chinese with other children. |
| Type Of Art | Film/Video/Animation |
| Year Produced | 2019 |
| Impact | This piece was shown at our exhibition in Shanghai, and was well received by the audience. |
| URL | https://youtu.be/W4aSOujFDho |
| Description | This grant has enabled us to make the important initial steps to develop a Goldsmiths based creative industry partnership network between Shanghai and London. First of all, we have established a London-based network with Goldsmiths academics and the creative industries interested in working in China, some of them with existing experience and connections (UKIE, EF, etc.). We have also reached out and made initial connections to the head office of several top Chinese tech companies (Tencent, Alibaba, Huawei). Secondly, through a 9-day Shanghai visit, with 6 people travelling, we established partnerships with three top Universities in Shanghai: - Jiaotong University (ICCI, Department of Computer Science and Engineering) - Fudan University (media department) - Shanghai Theatre Academy We met the following companies on site at their offices: - Disney Shanghai (Shanghai International Tourism Resort, received by government officials) - Mirror Pictures - Hirain (http://www.hirain.com.cn/) - Tencent Shanghai - Education First Shanghai - Lilac International In total, during our China trip we have organised several workshop, meetings, and exhibitions: - Meeting with Shanghai International Resort Tourism - Workshop, meeting, and demo with ICCI, Jiaotong University - Workshop, meeting, and demo with Department of Computer Science and Engineering, Jiaotong University - Workshop and demo with Education First - Workshop with Tencent Shanghai office (gaming department) - Workshop at the Lilac International Commercial Centre on Financial data and music visualisations - Visit and meeting with Hirain Company about collaborative VR project in theme park - Visit and meeting with STA culture development Ltd. - VR /AR exhibition open to the public (attended by over 800 people over 5 days at Lilacs Towers Shopping Centre. This event was featured on the two main Shanghai TV Channels) We have identified a few ways to establish future collaborations: - joint degrees and student and teaching exchange programmes - co-productions of immersive experience - post-production technology exchange - co-organise workshop |
| Exploitation Route | Academic route: as a direct result of this grant, we have won a further AHRC funding with international consortium. Planned academic pathway to impact activities include: research publications and workshops. Non-academic route: We will be working closely with partners to explore exhibitions and business ventures. |
| Sectors | Creative Economy,Digital/Communication/Information Technologies (including Software),Education |
| URL | https://www.panxueni.com/china |
| Description | Immersive, Innovative, and Interactive Experience (IIIE): A Technology Driven London-Shanghai Partnership in Mixed Realities and Performing Arts |
| Amount | £504,316 (GBP) |
| Funding ID | AH/T011416/1 |
| Organisation | Arts & Humanities Research Council (AHRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 02/2020 |
| End | 01/2023 |
| Description | Joint Research Centre |
| Organisation | MING GONG PICTURES |
| Country | China |
| Sector | Private |
| PI Contribution | AHRC seeding grant enabled our visit to Shanghai for meetings with Mirror Pictures. We have also hosted workshop with them in London. |
| Collaborator Contribution | Mirror Pictures hosted multiple workshop for us in China, and reimbursed our hotel. |
| Impact | We have signed an MoU for a Goldsmiths Mirror Research Centre, based in the Disney International Resort in Shanghai. |
| Start Year | 2019 |
| Description | Research and teaching collaboration |
| Organisation | Shanghai Jiao Tong University |
| Country | China |
| Sector | Academic/University |
| PI Contribution | The AHRC seeding fund was used to organise a trip to Shanghai, which led to multiple meetings with Shanghai Jiaotong University. |
| Collaborator Contribution | They hosted our meeting in China, and supported our further AHRC grant application. SJTU has offered a visiting Professorship to Prof William Latham from Goldsmiths. Dr Pearl Wang from SJTU has collaborated with us on market research, studying immersive content consumption by Chinese audience. |
| Impact | We have signed an MoU with Shanghai Jiaotong University as a direct outcome of the AHRC seeding grant. |
| Start Year | 2019 |
| Description | Innovation in Immersive Art & Tech: Goldsmiths UK-China Workshop |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Industry/Business |
| Results and Impact | Over 20 UK and China based company attended the Workshop helped us to develop a UK-China immersive art & tech network. Many became our project partners for our AHRC UK-China grant 2020. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://sites.google.com/view/ukchinags |
| Description | Interview for national news |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | We were interviewed by multiple mainstream media during our Shanghai trip in July. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://3g.163.com/v/video/VUJ9T000K.html |
| Description | Interview for national news - BBC Radio 3 |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | I was interviewed at the BBC Radio 3, talking about using VR and AI to make technology more inclusive, also about the future of UK-China partnership. It was a 45 minutes interview recently podcasted, already resulting in emails asking me to author a book in this area and some keynote speak invitations. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.bbc.co.uk/programmes/p085mn2k |