Virtual Realities - Immersive Documentary Encounters

Lead Research Organisation: University of Bristol
Department Name: Computer Science

Abstract

This project explores the application of Virtual Reality (VR) technology to documentary content. Thirty years since the term virtual reality was coined, and nearly fifty years since Ivan Sutherland's first experiments with a "head mounted three dimensional display", developments in hardware and software now make VR viable as a mass-market consumer proposition, and 2016 sees a variety of devices coming to market. While cinema offers an optical illusion of movement, 360 experiences offer a new generation of illusion - a powerful sense of "being there" within the scene portrayed known as presence. To date, consumer VR has been associated with gaming. However, the potential audience appeal promised by presence has led to considerable interest in the video market - notably among producers of non-fiction content. 2015 saw VR projects commissioned by the New York Times, Vice News and the UN for example, the latter winning the Innovation Award at the Sheffield DocFest - one of the world's major non-fiction markets.
We hypothesise that Virtual Reality documentary has significant potential to inform public debate - offering new directions and in particular novel forms of witnessing enabled by 360 perspectives, but that the powerful sense of presence offered by immersion also gives rise to ethical challenges; does the feeling of "being there" offer genuine insight or a new form of voyeurism? do unmanned 360 camera rigs feel like surveillance to subjects? a user in an immersive headset experience is likely to be paying attention to the world depicted, but what does that mean for their presence where they are standing?
The project will approach these questions through six case studies created by industry partners - examining production and user experience. Producers will share their approaches to storytelling in this new 360 medium. With an audience group who we will follow from their first exposure to VR, we will investigate engagement, attention, understanding and emotional reaction, as experienced on contrasting VR platforms, and compare these with 2D and 3D experiences delivered without headsets. The first three case studies will address the theme of migration - one of the big global challenges that has been a notable subject of VR non-fiction to date.
The research will explore the implications of exposure to powerful documentary content within immersive experiences, probe issues around isolation and sociality that relate to the use of headsets. The project will also consider the point-of-view of the subjects of media, asking for example whether particular issues around privacy arise in the context of 3D filming and immersive display. Drawing on findings from the case studies, and from workshops with industry and community partners, we will define key themes which will be explored in the production of three path finding prototypes. These will illustrate research findings and suggest future directions for Virtual Reality documentary.
Producers are eager to engage with the potentials of VR documentary, but face challenges around language, technology and audience insight. Through a process of investigation, dialogue, and shared findings, the research will stage a timely engagement with this emerging medium, supporting the development of this new production sector, while keeping the interest of audience and subjects to the fore.

Planned Impact

The work undertaken in this proposal seeks to create both economic and societal impact.

Creative Industries are now worth £84 billion according to the UK's government economic estimates (January 2016), with significant emphasis placed on VR with a current market estimated at $6.7 billion and VR hardware sales expected to reach $70 billion by 2020 (Re/code). VR experiences are moving into the mainstream, with entry points to VR available at every level of consumer interest, from Google Cardboard to high end consumer experience such as Oculus Rift or HTC Vive.

Therefore an investigation into the opportunities and challenges of VR for documentary content creation and user experience is timely and pertinent. The project will support the digital and creative economy by providing much needed evaluation, critical reflection and guidance to this fast-growing new market including VR technology creators and documentary producers, from large corporations through to small independents. This intelligence is critical to understand the affordances of VR for documentary in order for the UK to emerge as leaders in documentary content creation and consumption of this new medium. It also has potential impact for creators of content within the wider field where Virtual Reality technology is being applied to non-fiction - museums, galleries, marketing, music videos and other forms of factual entertainment, as well as those exploring VR for journalism. While the domain of the research is VR, the findings also have relevance and potential impact more widely for creators of virtual experiences such as the emerging Augmented Reality (AR) market.

These impacts will emerge from a number of aspects of the research:
- investigation into and assessment of best creative practice to date.
- investigation into and assessment of audience experience.
- networking during the course of the project and dialogue between major industry players and SMEs at the forefront of creative work in the field.
-the advancement of a common language for this new medium between makers, creators, technologists and audiences.
- the demonstration of future directions for VR non-fiction through co-production of prototypes with SMEs.
- the development of guidelines for ethical practice.

Industry will be reached through panels and workshops at the VR World Congress, Virtual Reality Summit, Sheffield DocFest, the International Journalism Festival, Perugia, IDFA Doc Lab, Tribeca Interactive and Sundance Festival.

Dissemination on social media will be key for reach so the project will develop not only a project website but also a presence on twitter and facebook, as well as targeting and working with groups with established social media followings such as i-Docs and Women in VR.

Documentary, journalism and factual media play a significant role in informing and encouraging public debate. The project therefore also anticipates societal impact in supporting the development of meaningful and ethical forms of VR non-fiction practice which can reflect and critique "our shared world" (Nichols). It will also have potential impact on the video practices of the public as they engage with 360 media, to date as viewers but in the future increasingly also as amateur producers.
 
Description Although we are still early on within the project as a whole through the analysis of some of the work packages initial findings are being raised.

The analysis that has been done from the creation of the VR mediography we are able to detect that there are popular content themes to the pieces of VR that have been created thus far. We are also able to see how the pieces of VR non-fiction have increased in length as the producers are exploring the new medium and the time people are in the VR experience. We have now finished populating the VR mediography with a comprehensive list of non fiction VR pieces until 2018 and therefore have started to conduct a more more thorough analysis looking at different narrative and technological techniques which effect the user's experience. This has led to a paper being accepted to be presented at ACM CHI 2019.

We are coming to the end of an "in the wild" audience study involving 12 households recruited locally from as diverse a range as possible. We have provided them with an Oculus Go and a selection of VRNF pieces to view over an 8 week period. Within the study participants are being asked to reflect on the VR platforms themselves; on individual and communal experiences of VRNF; on their emotional & intellectual responses to VRNF; and on how VRNF might form part of a wider diet of NF media.

We are also coming to the end of a psychological study with over 100 participants examining how watching the Virtual Reality Nonfiction documentary "Clouds Over Sidra", which shows the life of Syrian Refugees in a refugee camp in Jordan affects participants explicit and implicit attitudes towards Arabs and pro-social behaviour. Participants viewed the documentary on either an Oculus Rift or on a tablet and either as a 360 video or from a fixed view.

We have commenced a production study which is examining the three commissions in detail, alongside interviewing a wide range of producers within Virtual Reality Nonfiction to explore current business models, language and ethics among other aspects.
Exploitation Route The VR mediography is openly accessible for both academics and VR producers to explore and give us feedback. http://vrdocumentaryencounters.co.uk/vrmediography/vrmediography/
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Leisure Activities, including Sports, Recreation and Tourism,Government, Democracy and Justice

URL http://vrdocumentaryencounters.co.uk/
 
Title Non-Fiction Documentary VR: A Mediography 
Description "Non-Fiction Documentary VR: A Mediography" is a database of non-fiction documentary VR projects, covering the period 2012-current. A web-based frontend has also been developed for the database, providing analytics and an interactive timeline. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact Since launching in mid January 2018, the Mediography website has attracted 333 page views from 90 unique visitors. 
URL http://vrdocumentaryencounters.co.uk/vrmediography/vrmediography/
 
Description Aardman Animations 
Organisation Aardman Animations Ltd
PI Contribution Aardman Animations are a partner on the project as named in the grant. Representatives of Aardman Animations have attended our workshops where we share our research expertise and findings from the project
Collaborator Contribution Representatives of Aardman Animations attend our workshops and contribute relevant projects and industry expertise which Aardman Animations have amassed working within Virtual Reality.
Impact ongoing
Start Year 2018
 
Description Archers Mark 
Organisation Archer's Mark
PI Contribution Archers Mark are a partner on the project as named in the grant. Representatives of Archers Mark have attended our workshops where we share our research expertise and findings from the project
Collaborator Contribution Representatives of Archers Mark attend our workshops and contribute relevant industry expertise built up over years of working within Virtual Reality.
Impact ongoing
Start Year 2018
 
Description BBC 
Organisation British Broadcasting Corporation (BBC)
Country United Kingdom 
Sector Public 
PI Contribution BBC are a partner on the project as named in the grant. A representative of the BBC R&D team (Maxine Glancy) attends our regular project meetings where we share our research expertise and findings from the project.
Collaborator Contribution A representative of the BBC R&D team (Maxine Glancy) attends our regular project meetings and contributes relevant findings and research that the BBC are conducting within Virtual Reality nonfiction. The BBC are hosting the Ethics workshop in November 2019 at their BBC London site.
Impact ongoing
Start Year 2018
 
Description The Guardian 
Organisation Guardian Media Group
Department Guardian News and Media
Country United Kingdom 
Sector Private 
PI Contribution The Guardian are a partner on the project as named in the grant. Representatives of the Guardian have attended our workshops where we share our research expertise and findings from the project
Collaborator Contribution Representatives of The Guardian attend our workshops and contribute relevant findings and research that the Guardian are conducting within Virtual Reality nonfiction.
Impact ongoing
Start Year 2018
 
Description The Watershed 
Organisation Watershed Media Centre
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution We are bringing our underlying research on VR nonfiction production to the commissioning process, as well as research findings.
Collaborator Contribution Watershed are contributing expertise in commissioning and production process.
Impact The Watershed and research team work together on the commissioning aspects of the project. The partnership has so far resulted in the call for commissions, shortlisting and interviewing of the shortlisted candidates and continues to support the commissioned teams while they develop their non fiction pieces via a series of commissions workshops and setting up of communication channels between all parties.
Start Year 2018
 
Title A 360-degree video playback and visualisation platform that provides frame-by-frame analysis and comparison of viewer gaze behaviour. 
Description A 360-degree video playback and visualisation platform that provides frame-by-frame analysis and comparison of viewer gaze behaviour. 
Type Of Technology Software 
Year Produced 2018 
Impact Our video player is cross-platform and capable of handling all forms of 360 degree video. It is currently supporting three collaborative user studies across our three project sites. 
 
Title A 360-degree video player with inbuilt head telemetry recording 
Description A 360-degree video player with inbuilt head telemetry recording. Developed for Oculus Rift, GO / Gear VR. Also compatible with tablet playback using the 'magic window' approach. 
Type Of Technology Software 
Year Produced 2018 
Impact This analytics suite is allowing us to examine differences in viewing behaviours across multiple films and types of VR scene. This tool is currently supporting two large collaborative research projects across our three project sites. We are developing the functionality of the tool to ultimately support VR content creators. 
 
Description A preliminary opinion survey about audiences of VR non-fiction 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact An expert opinion survey of 60+ professional directors, curators and producers of non-fiction VR, asking questions about what we need to know about audiences of non-fiction VR.
Year(s) Of Engagement Activity 2018
URL https://eu.jotform.com/73114258709357
 
Description Art, Research and Creativity Event - Tales from the Field Talk, Danae Stanton Fraser 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Danae Stanton Fraser gave a talk on her interdisciplinary work, co-designing with the creative industry. Tales from the field - Art, Research and Creativity. Hosted by the Arts Council at The Edge. This galvanized relationships with Bath Spa and others leading to an AHRC Creative Clusters bid submission.
Year(s) Of Engagement Activity 2017
 
Description Bristol Computer Science lunchtime talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact A seminar, half of which was dedicated to communicating the aims of the Virtual Realities project to an audience of peers in the academic community.
Year(s) Of Engagement Activity 2018
 
Description Commissioning Workshops for commissioned teams 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Two Commissioning Workshops have been delivered (January 2019 and March 2019) with one more to take place in May 2019. These workshops have brought together the commissioned teams with the research team and Watershed producer to discuss the work each commissioned team are doing and ensure close collaboration and focused attention to the research questions. Bringing the commissioned teams together to relate to each other so they become a cohort has also provided valuable insight and opportunities for exchange of information and ideas.
Year(s) Of Engagement Activity 2019
 
Description Commissions publicity 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact PR focusing on the announcement of the three successful commissioned teams commissioned to produce non fiction Virtual Reality documentaries as part of the project. Coordinated approach between University of Bristol press team and those of Bath University and UWE as well as Watershed PR agency Ladbury. Articles appeared on the three University web pages as well as that of Watershed and was sent to industry specific news agencies.
Year(s) Of Engagement Activity 2019
URL http://www.bristol.ac.uk/news/2019/march/vr-documentaries.html
 
Description Creation of project website and social media channels 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact A public facing website for the "Virtual Realities" project was developed and launched in Jan 2018. The website provides a portal for project related news and events, details of the team members and our external partners. An associated social media channel for the project has also been created on Twitter. As of 4/3/18, our project website has attracted 238 sessions from 170 unique users. Our Twitter handle @VRDocsProject has attracted 56 followers.
Year(s) Of Engagement Activity 2018
URL http://vrdocumentaryencounters.co.uk/
 
Description Danae Stanton Fraser Invited to speak at Facebook London as part of the UX Research Speaker Series 9 October 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Danae Stanton Fraser spoke at Facebook to an internal audience there to highlight the work that is being conducted within the project and start discussions on how they might be involved in future workshops.
Year(s) Of Engagement Activity 2018
 
Description Danae Stanton Fraser Keynote speech at VR in Education 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The VR in education conference is detailed at https://www.swansea.ac.uk/vr-in-teaching/vr-conference/ and there is a video of Danae Stanton Fraser keynote speech at the URL below.
Year(s) Of Engagement Activity 2018
URL https://www.youtube.com/watch?v=Mm0m97w2TEg
 
Description Danae Stanton Fraser Talk at Mixed Reality lab, University of Nottingham, 31 October 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Talk at Mixed Reality lab
Year(s) Of Engagement Activity 2018
URL https://www.nottingham.ac.uk/research/groups/mixedrealitylab/events/2018-10-31-school-seminar-danaƫ-...
 
Description Danae Stanton Fraser invited to attend Education Committee session on the Fourth Industrial Revolution and the role of education - Tuesday 11 December at 10am 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The inquiry is ongoing to examine the challenges posed and opportunities presented by the Fourth Industrial Revolution. Danae took part in the second evidence session as part of the inquiry to look at how best to prepare young people to take advantage of future opportunities by examining the suitability of the school curriculum. It is also looking at the role of lifelong learning and how best to help people climb the ladder of opportunity in the future.
Year(s) Of Engagement Activity 2018
URL https://www.fenews.co.uk/press-releases/23133-fw-education-committee-holds-session-on-the-fourth-ind...
 
Description Dave Green organised regular immersion reading group 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact A regular reading group to discuss developments within the field of non fiction virtual reality.
Year(s) Of Engagement Activity 2018
 
Description Interview by Mandy Rose on Radio 4 programme - "I Feel for You - Empaths and Empathy" 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Radio 4 Producer Jolyon Jenkins read my article - The Immersive Turn - in Studies in Documentary Film special edition on virtual reality. My argument significantly influenced the programme he was developing looking at the contemporary cultural preoccupation with empathy. Jenkins interviewed me and my contribution ran through a ten minute section of the programme.
Year(s) Of Engagement Activity 2019
URL https://www.bbc.co.uk/programmes/m0001xsk
 
Description Keynote at Creative Industry Conference - Berlin 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Keynote - "Challenges and Opportunities for Virtual Reality in the Creative Industries" at the 4th Conference on Virtual Reality Journalism and Documentary. 50-60 Attendees across industry, academic research and teaching
Year(s) Of Engagement Activity 2018
URL https://www.vrconference.info/
 
Description Kirsten Cater live interview on India Today discussing innovation and Virtual Reality in education 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Facebook live on India Today interview with Kirsten Cater regarding innovation and VR in education. Viewed by over 3.3k individuals across the world.
Year(s) Of Engagement Activity 2018
URL https://www.facebook.com/indiatodaygroupeducation/videos/717574701942905/
 
Description Mediography Poster at TVX 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Poster as part of the projects' presence at TVX 2018 (Note a paper was also delivered at this conference). TVX is the leading international conference into research into online video, TV interaction and user experience. It took place in Seoul and was an excellent opportunity for our team member to show the work to date on the mediography to peers and colleagues.
Year(s) Of Engagement Activity 2018
URL https://tvx.acm.org/2018/
 
Description Mediography at Sheffield Docfest, leading UK documentary festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Mediography on display allowing for wider engagement with attendees at Sheffield Docfest and interested members of public.  Non fiction pieces were able to be shown through headsets at the stand. Banner on display showing how the mediography fits with the other strands of the project and promotional materials about the mediography and the project were distributed. This was an excellent opportunity for the mediography to be showcased to its intended audience. There was lots of interest and engagement generated with attendees as well as with members of the general public.
Year(s) Of Engagement Activity 2018
URL https://sheffdocfest.com/view/aboutdocfest
 
Description Mediography on display at i-Docs 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Mediography was on display and some non fiction VR pieces able to be accessed in headsets by attendees at i-Docs during the conference. This enabled attendees to have a try at using the mediography, helping them to understand its scope and make suggestions and provide feedback to the team member present.
Year(s) Of Engagement Activity 2018
URL http://i-docs.org/
 
Description Regular Stakeholder newsletters 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Stakeholder newsletters issued to key individuals with an interest in the project keeping them updated with progress and developments as well as important dates for future engagement.
Year(s) Of Engagement Activity 2018,2019
 
Description The Language of VR - Bristol VR Lab - lunchtime seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact One of three invited 20-minute talks at a 90 minute lunchtime seminar. Attended by a mix of industry, academic and third sector attendees. Included a 30 minute public forum.
Year(s) Of Engagement Activity 2018
URL https://bristolvrlab.com/events/28th-feb-2018-bvrl-afternoon-seminars-2-0-establishing-a-vr-storytel...
 
Description Virtual Reality taster session for Refugee Women Bristol 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Virtual Reality taster session and recruitment event delivered to refugee families with the help of Refugee Women of Bristol as part of recruitment for the Household Study. Participants were a mix of ages and genders.
Year(s) Of Engagement Activity 2018
 
Description Virtual Reality taster sessions delivered at Knowle West Media Centre for local community 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Three VR taster sessions (several hours each) two daytime and one evening workshop delivered in the local community at Knowle West Media Centre as part of recruitment for Household Study.  Participants were wide ranging in age and a mixture of genders.
Year(s) Of Engagement Activity 2018
 
Description Workshop 1 - What do we need to know? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Our first project workshop What do we need to know? was held on Friday 19th January at the Watershed in Bristol. It was the first of four workshops to be held over the course of the project, providing a framework through which partners and stakeholders can co-design the research.

The workshop introduced the project and staged a dialogue about what immersive non-fiction VR content experiences mean to audiences. We shared our objectives, research questions and initial findings, including an emergent mapping of the field to date, and workshopped areas for investigation. The findings of the workshop will inform the audience study and commissioning of three new pieces of VR work.

Over 60 delegates signed up for the workshop - a mix of creative industry and academics - which ensured we had a balanced discussion from a range of voices, backgrounds and interests. We had four guest speakers throughout the day to help push discussions around the use of VR for non-fiction both the challenges and opportunities.

Dan Archer, Empathetic Media, New York USA
Dan is a thought leader in the VR/AR/interactive storytelling space and founded Empathetic Media in 2015. He is a 2016 fellow at the Tow Center at Columbia University and was a Reynolds Journalism Institute Fellow (University of Missouri, 2014) and a Knight Journalism Fellow at Stanford University (2011).

Jane Gauntlett, Pervasive Media Studio, Bristol UK
Jane is a writer for film and theatre. In 2011, Jane founded the internationally recognised empathy project In My Shoes. In My Shoes is an ever-expanding collection of first-person documentary style interactive performances, which guide participants through the beautiful, the challenging, the mundane and the surreal aspects of being human.

Dr. Kate Nash, University of Leeds
Kate is Associate Professor of Media and Communication in the School of Media and Communication at the University of Leeds. Her research focuses on the intersections between documentary culture and emerging media platforms and practices.

Prof. Chris Anderson University of Leeds
Chris is Professor of Media and Communication at the University of Leeds. Chris studies journalism, politics, and how the production of public knowledge is being transformed in the digital age.
Year(s) Of Engagement Activity 2018
URL http://vrdocumentaryencounters.co.uk/
 
Description i-Docs Panel discussion and Q and A focused on Immersive documentary encounters. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Members of the research team Mandy Rose and Danae Stanton-Fraser were part of a panel with industry colleagues Karim Ben Khalifa (photographer and artist) and Dan Archer (Director of Empathic Media) discussing Immersive documentary encounters. There were questions from the audience who were very interested in the scope of the project.
Year(s) Of Engagement Activity 2018
URL http://i-docs.org/2018/07/16/video-immersive-documentary-encounters-i-docs-2018/