Using personal data for AR driven story telling

I will first investigate the current literature concerning which stories are being told or predicted to be told in mixed reality media (MRM; used to describe both VR and AR), as well as which features of current and near future technology allow for stories to be told effectively. Meta-analyses of previous MRM, such as those by Shilkrot et al (2014) and Zhou et al (2008), will be very important at this point.

This literature review will then inform the next steps of my research namely the creating of two mixed reality experiences. One of these experiences will utilise 360 degree video to capture a live theatre performance from multiple angles, so that the audience can decide where to view a performance from outside the performance space. The other will computer generate a version of a live theatre performance, either using motion capture or free viewpoint video (detailed in Collet et al, 2015), so that audiences can move throughout the performance space while it is happening and view it from any angle.

Once these experiences have been created participants will be recruited to view them, with their behaviour during the viewing being recorded in a number of ways that current and near future headsets are able to. Where their attention is placed in the experience, and for how long, will be figured out using tracked head positioning, described in LaValle et al (2014) and eye tracking technology. If they say anything or laugh or scream during the experience this is also useful information for content creators so on board microphones will record this data. Surveys and interviews will also be used.

This audience feedback data will then be examined and used to inform the second round of content creation, where the experiences are recreated with this feedback in mind to improve them and make them more preferred and attention grabbing, or engaging. These new and improved experiences will then be reviewed by a mixture of new and previous participants and viewing behaviour data collected again to see if the audience feedback was able to inform a more engaging content creation process.

This PhD is important because MRM use brand new consumer technology for storytelling with unique features compared to other traditional media technologies, such as the increased scope for creative interactivity (Nam, 2015). Although applications of this technology are being studied quite widely the next step in researching them is developing systems that can collect data about audience behaviour and feed this data back to content creators and broadcasters to inform future programming (for example the BBC, my project partner, is keen to look at storytelling and the HCI of engagement in AR). Doing this allows me to be at the forefront of investigating how this new technology can shape stories for future audiences.

We intend the Horizon CDT to be the place where partners come to find their future employees and to engage with the opportunities and challenges of digital identity and personal data. The key beneficiaries of our research will be:

- Commercial private sector companies that will engage with our CDT students during their research and/or employ them after graduation. Our partners include companies developing digital identity technologies as well as user companies across a range of sectors (consumer goods, entertainment, transportation, energy and others).

- Public sector and third sector organisations that are concerned with the use of digital identities to support civil society including broadcasters, healthcare providers and campaign groups.

- The public whose personal data forms the focus of their research and who will ultimately use and come to depend upon digital identities.

- Research communities spanning computer science, engineering, psychology, sociology, business and humanities.

These will benefit in various ways.

- Commercial, public and third sector companies will benefit from being able to recruit from a pool of talented PhD graduates who bring an in-depth understanding of digital identity and a proven ability to work in interdisciplinary teams. They will also benefit from being able to participate in co-creation of PhD research to ensure focus on relevant challenges and be able to exploit results of this PhD research.

- The public will benefit through a greater understanding of the opportunities and challenges of digital identity.

- Research communities will benefit by opening up promising new interdisciplinary fields.

Our Impact activities will be driven by Professor Derek McAuley, the Director of Horizon, who has a track record of establishing industry labs, spinning our companies and who is currently acting CIO of the TSB funded Connected Digital Economy Hub. Key activities will be:

- All Horizon PhDs will be carried out in collaboration with an external partner who will be involved in drawing up the initial topic, recruiting students, shaping the PhD proposal, supervision, and hosting at least one internship

- We will continue to organise knowledge exchange events within Horizon that are open to our network of over 100 external partners, including our annual Horizon Research Conference.

- We will encourage the release of applications, open source software, and open datasets wherever collaboration agreements allow.

- We will actively encourage our students to spin-out new ventures, including providing seedcorn funding through Horizon.

- We will engage our students with our two partner catapults, the Connected Digital Economy Catapult and the Satellite Applications Catapult.

- We will actively encourage industry visits through guest lectures on our "Broadening Horizons" core taught programme.

- We will also encourage companies to define, steer and sponsor the first year interdisciplinary team projects.

- Our students will complete a module on Public and External Engagement and are encouraged to engage in public events and exhibitions.

- Horizon's journalist-in-residence will help expose students research to the wider world through regular blogposts, while the University's marketing and communications team will help them develop press releases.

- We will provide training in research publication as part of the Professional Skills module and mentor publications through the Practice Led Project and the annual writing retreat (where students present and critique draft papers).

These impact activities will be supported by a professional online presence with posters, demos and podcasts made available through our website and associated YouTube channel and twitter feed, and with individual PhD profiles being posted on our own site and on external networking portals such as LinkedIn and ResearchGate.