Children's Magical Realism for New Spatial Interactions: AR and Archives

Lead Research Organisation: Newcastle University
Department Name: School of Arts and Cultures

Abstract

Modern mobile phones allow new kinds of interaction which mix the real world and virtual worlds together. So called 'augmented reality' (AR) usually overlays digital images or 3D models on to video from phone cameras to present audiences with new kinds of mixed reality that they can explore almost as if it were real space. In recent months, some new developments from Google and Apple have pushed the technical boundaries of what mobile phones can do allowing much more complicated interactions with real space allowing us to cause digital objects to appear really rooted in the real world.

Although the technical possibilities of this work are exciting there is a real danger that the experiences designers create may be led by the technology and not inspired by other kinds of creativity. If we would like to explore spaces and places in new ways we should look to other forms such as film, theatre, dance and in our case books to expand our vocabulary of space.

Our project uses the work of a prominent author of children and young adults' books, David Almond to suggest new imaginative spatial interactions for AR. Almond has been describe as a magical realist author and much of his work is about confusing or ambiguous cross overs between worlds, real and imaginary, past and present, everyday or mythical. During our project we will work with specialists from our project partner, Seven Stories, the National Centre for Children's Books as target audiences and AR specialists to co-design new kinds of spatial interaction for AR and VR applications.

Our project will have two main results. The first is the most straightforward: an Android and iPhone app that will be presented as part of the Great Exhibition of the North in a creative walking trail leading visitors through one of the most exciting and creative parts of the city of Newcastle upon Tyne, the Ouseburn Valley. The app will take early sketches from Almond's work, drawn from his archive in Seven Stories, and combine them with story telling elements in key locations in the valley. The second result of our project is to present our design method, the process by which we thought together with our project partners, audience participants and industry specialists, to create the work. We believe that by doing this we can allow other museums and galleries to produce new creative projects in a way that involves others. We think that this will produce better creative work and be inherently more democratic in its use of public digital space.

Planned Impact

Our project takes place with the partnership of a popular cultural institution, Seven Stories, and builds on an existing strategic partnership with Newcastle University, Vital North. The project will not only deepen and strengthen our partnership but will have significant impact within the organisation of Seven Stories as they develop a digital strategy for the future. The research takes place with publics almost from the outset as we involve focus groups in co-design processes. The deployment of our main output takes place within a nationally significant event with tie-ins to a wide range of cultural activities as part of the Great Exhibition of the North. We identify impact for the following beneficiaries:

Seven Stories
Seven Stories does not have an in-house digital department but has a strategic objective of developing a digital strategy. Working closely with the digital team at Newcastle University, this partnership will allow them to engage with audiences and VR experts to co-design an immersive platform based around a children's literature archive, and will extend their ability to experiment digitally in presenting collections to public audiences.

Audiences in Newcastle upon Tyne and nationally
In summer 2018, Newcastle Gateshead is hosting the Great Exhibition of the North, which aims to engage with an audience of 3 million people and celebrates Northern culture, design and innovation. Seven Stories is a key cultural destination on one of the three festival walking routes and is working with festival organisers to curate a substantial section of the 'Design' trail. The app created will complement and extend this activity, and has the potential to be seen by a large public audience during the festival period. The app will also provide a digital link to Almond's original archival material, which will form the focus of Seven Stories' in-venue exhibit. The organisation is aiming to tour the exhibition from summer 2019 to summer 2021 to three venues nationally.

Researchers and Professionals in AR/VR
The core contributions of our project which explores an expanded and imaginative sense of immersion through engagement with magical realist literature have far reaching potential for adaptation in both AR and VR gaming applications now and in future. As well as finding academic beneficiaries through associated publication outlets we will engage the AR/VR creative communities. Newcastle upon Tyne is an exceptionally appropriate venue for impact of this kind due to the impact of creative industries on the local economy which provides both a backdrop of influential activity and a stage for two-way engagement. Studio complexes around the city host VR startups and more recently VRTGO Labs in our sister city, Gateshead has provided a hub for small VR enterprises since 2015 currently hosting twenty five such companies.

This impact will be achieved in the following ways across the timeline of the project.

Early prototyping and focus groups
We will conduct one focus group with local VR professionals introducing preliminary aims of our research and eliciting future engagement along the project.

Throughout Exhibition showcase
As part of the user testing phase of our project we will invite AR/VR professionals to take the tour with our artefact and convene a short series of discussions where we both explore the user experiences of other professionals engaged with our work but also provide information about our design process and present preliminary findings in the form of a short 'designers report.' This stage of impact deliberately integrates research and impact aims of the project as we seek to both present the current state of our research and learn from the considerable expertise present in the city.

End of Project
We wll present our work at at least one national event for immersive technologies professionals such as AVR360, VR World Congress, VR World 2018 and VRX Europe 2018.

Publications

10 25 50