Children's Magical Realism for New Spatial Interactions: AR and Archives
Lead Research Organisation:
Newcastle University
Department Name: Sch of Arts and Cultures
Abstract
Modern mobile phones allow new kinds of interaction which mix the real world and virtual worlds together. So called 'augmented reality' (AR) usually overlays digital images or 3D models on to video from phone cameras to present audiences with new kinds of mixed reality that they can explore almost as if it were real space. In recent months, some new developments from Google and Apple have pushed the technical boundaries of what mobile phones can do allowing much more complicated interactions with real space allowing us to cause digital objects to appear really rooted in the real world.
Although the technical possibilities of this work are exciting there is a real danger that the experiences designers create may be led by the technology and not inspired by other kinds of creativity. If we would like to explore spaces and places in new ways we should look to other forms such as film, theatre, dance and in our case books to expand our vocabulary of space.
Our project uses the work of a prominent author of children and young adults' books, David Almond to suggest new imaginative spatial interactions for AR. Almond has been describe as a magical realist author and much of his work is about confusing or ambiguous cross overs between worlds, real and imaginary, past and present, everyday or mythical. During our project we will work with specialists from our project partner, Seven Stories, the National Centre for Children's Books as target audiences and AR specialists to co-design new kinds of spatial interaction for AR and VR applications.
Our project will have two main results. The first is the most straightforward: an Android and iPhone app that will be presented as part of the Great Exhibition of the North in a creative walking trail leading visitors through one of the most exciting and creative parts of the city of Newcastle upon Tyne, the Ouseburn Valley. The app will take early sketches from Almond's work, drawn from his archive in Seven Stories, and combine them with story telling elements in key locations in the valley. The second result of our project is to present our design method, the process by which we thought together with our project partners, audience participants and industry specialists, to create the work. We believe that by doing this we can allow other museums and galleries to produce new creative projects in a way that involves others. We think that this will produce better creative work and be inherently more democratic in its use of public digital space.
Although the technical possibilities of this work are exciting there is a real danger that the experiences designers create may be led by the technology and not inspired by other kinds of creativity. If we would like to explore spaces and places in new ways we should look to other forms such as film, theatre, dance and in our case books to expand our vocabulary of space.
Our project uses the work of a prominent author of children and young adults' books, David Almond to suggest new imaginative spatial interactions for AR. Almond has been describe as a magical realist author and much of his work is about confusing or ambiguous cross overs between worlds, real and imaginary, past and present, everyday or mythical. During our project we will work with specialists from our project partner, Seven Stories, the National Centre for Children's Books as target audiences and AR specialists to co-design new kinds of spatial interaction for AR and VR applications.
Our project will have two main results. The first is the most straightforward: an Android and iPhone app that will be presented as part of the Great Exhibition of the North in a creative walking trail leading visitors through one of the most exciting and creative parts of the city of Newcastle upon Tyne, the Ouseburn Valley. The app will take early sketches from Almond's work, drawn from his archive in Seven Stories, and combine them with story telling elements in key locations in the valley. The second result of our project is to present our design method, the process by which we thought together with our project partners, audience participants and industry specialists, to create the work. We believe that by doing this we can allow other museums and galleries to produce new creative projects in a way that involves others. We think that this will produce better creative work and be inherently more democratic in its use of public digital space.
Planned Impact
Our project takes place with the partnership of a popular cultural institution, Seven Stories, and builds on an existing strategic partnership with Newcastle University, Vital North. The project will not only deepen and strengthen our partnership but will have significant impact within the organisation of Seven Stories as they develop a digital strategy for the future. The research takes place with publics almost from the outset as we involve focus groups in co-design processes. The deployment of our main output takes place within a nationally significant event with tie-ins to a wide range of cultural activities as part of the Great Exhibition of the North. We identify impact for the following beneficiaries:
Seven Stories
Seven Stories does not have an in-house digital department but has a strategic objective of developing a digital strategy. Working closely with the digital team at Newcastle University, this partnership will allow them to engage with audiences and VR experts to co-design an immersive platform based around a children's literature archive, and will extend their ability to experiment digitally in presenting collections to public audiences.
Audiences in Newcastle upon Tyne and nationally
In summer 2018, Newcastle Gateshead is hosting the Great Exhibition of the North, which aims to engage with an audience of 3 million people and celebrates Northern culture, design and innovation. Seven Stories is a key cultural destination on one of the three festival walking routes and is working with festival organisers to curate a substantial section of the 'Design' trail. The app created will complement and extend this activity, and has the potential to be seen by a large public audience during the festival period. The app will also provide a digital link to Almond's original archival material, which will form the focus of Seven Stories' in-venue exhibit. The organisation is aiming to tour the exhibition from summer 2019 to summer 2021 to three venues nationally.
Researchers and Professionals in AR/VR
The core contributions of our project which explores an expanded and imaginative sense of immersion through engagement with magical realist literature have far reaching potential for adaptation in both AR and VR gaming applications now and in future. As well as finding academic beneficiaries through associated publication outlets we will engage the AR/VR creative communities. Newcastle upon Tyne is an exceptionally appropriate venue for impact of this kind due to the impact of creative industries on the local economy which provides both a backdrop of influential activity and a stage for two-way engagement. Studio complexes around the city host VR startups and more recently VRTGO Labs in our sister city, Gateshead has provided a hub for small VR enterprises since 2015 currently hosting twenty five such companies.
This impact will be achieved in the following ways across the timeline of the project.
Early prototyping and focus groups
We will conduct one focus group with local VR professionals introducing preliminary aims of our research and eliciting future engagement along the project.
Throughout Exhibition showcase
As part of the user testing phase of our project we will invite AR/VR professionals to take the tour with our artefact and convene a short series of discussions where we both explore the user experiences of other professionals engaged with our work but also provide information about our design process and present preliminary findings in the form of a short 'designers report.' This stage of impact deliberately integrates research and impact aims of the project as we seek to both present the current state of our research and learn from the considerable expertise present in the city.
End of Project
We wll present our work at at least one national event for immersive technologies professionals such as AVR360, VR World Congress, VR World 2018 and VRX Europe 2018.
Seven Stories
Seven Stories does not have an in-house digital department but has a strategic objective of developing a digital strategy. Working closely with the digital team at Newcastle University, this partnership will allow them to engage with audiences and VR experts to co-design an immersive platform based around a children's literature archive, and will extend their ability to experiment digitally in presenting collections to public audiences.
Audiences in Newcastle upon Tyne and nationally
In summer 2018, Newcastle Gateshead is hosting the Great Exhibition of the North, which aims to engage with an audience of 3 million people and celebrates Northern culture, design and innovation. Seven Stories is a key cultural destination on one of the three festival walking routes and is working with festival organisers to curate a substantial section of the 'Design' trail. The app created will complement and extend this activity, and has the potential to be seen by a large public audience during the festival period. The app will also provide a digital link to Almond's original archival material, which will form the focus of Seven Stories' in-venue exhibit. The organisation is aiming to tour the exhibition from summer 2019 to summer 2021 to three venues nationally.
Researchers and Professionals in AR/VR
The core contributions of our project which explores an expanded and imaginative sense of immersion through engagement with magical realist literature have far reaching potential for adaptation in both AR and VR gaming applications now and in future. As well as finding academic beneficiaries through associated publication outlets we will engage the AR/VR creative communities. Newcastle upon Tyne is an exceptionally appropriate venue for impact of this kind due to the impact of creative industries on the local economy which provides both a backdrop of influential activity and a stage for two-way engagement. Studio complexes around the city host VR startups and more recently VRTGO Labs in our sister city, Gateshead has provided a hub for small VR enterprises since 2015 currently hosting twenty five such companies.
This impact will be achieved in the following ways across the timeline of the project.
Early prototyping and focus groups
We will conduct one focus group with local VR professionals introducing preliminary aims of our research and eliciting future engagement along the project.
Throughout Exhibition showcase
As part of the user testing phase of our project we will invite AR/VR professionals to take the tour with our artefact and convene a short series of discussions where we both explore the user experiences of other professionals engaged with our work but also provide information about our design process and present preliminary findings in the form of a short 'designers report.' This stage of impact deliberately integrates research and impact aims of the project as we seek to both present the current state of our research and learn from the considerable expertise present in the city.
End of Project
We wll present our work at at least one national event for immersive technologies professionals such as AVR360, VR World Congress, VR World 2018 and VRX Europe 2018.
Publications
Reynolds K
(2019)
Children's Magical Realism for New Spatial Interactions: Augmented Reality and the David Almond Archives
in Children's Literature in Education
Arrigoni G
(2019)
Framing collaborative processes of digital transformation in cultural organisations: from literary archives to augmented reality
in Museum Management and Curatorship
Schofield T
(2019)
Magical Realism and Augmented Reality
Schofield T
(2020)
Magical Realist Design
| Description | Our findings fit into two main categories: 1. applying magical realist ideas to AR through workshopping. After conducted a number of session with children taking archival materials and imagining new forms of interaction with them we were able to identify a number of areas in wchih to imagine new interaction and to develop specific designs. These included a. the adoption of forms of 'horrific' atmosphere in app design b. the dramatisation of search for AR objects through UI design c. how to select archival items for development into AR designs in ways interesting to children 2. collaborative methods with cultural organisations Our project trialled the workshops as a way of conducting R&D for and with cultural organisations. We demonstrated the feasibility of using such processes for cultural organisations without being overly costly and indeed in ways of generating income by integrating exploratory research and public events |
| Exploitation Route | No work so far has considered magical realism and AR together. We believe that our findings provoke more research into this area exploring areas of adult magical realist fiction or indeed other literary genres. We believe that our findings in terms of the specifics of AR designs with magical realist influences can be directly adopted in an actionable way by designers. Our findings regarding collaboration with cultural organisations is already influencing our partners Seven Stories' digital strategy for commissioning new work. We are continuing to trial these methods as part of our follow on funding for impact and engagement. We have produced a booklet describing our methods and continue to distribute this to other cultural organisations seeking to conduct innovative digital work. |
| Sectors | Creative Economy,Culture, Heritage, Museums and Collections |
| URL | https://digitalcultures.ncl.ac.uk/projects/not_just_digital_knowledge.pdf |
| Description | Our partners, Seven Stories are using aspects of our workshopping methods in their outreach work with their collections material in particular developing activities which see archival materials as the basis for speculation and creativity. To support this work and others in the sector we produced a handbook/toolkit describing our collaborative process. This can be found at https://digitalcultures/projects/not_just_digital_knowledge.pdf This handbook has been distributed to other cultural partners and will form the centre piece of a series of collaborative workshops with the cultural sector later this year. We are trialling and extending these methods as part of our follow on funding for impact and engagement which extends the reach of the impact to economically/socially disadvantaged groups in Newcastle upon Tyne. Seven stories recognised the impact value of our research activities conducted with publics and the follow-on activities represent a further opportunity to develop their own capacity in this area. Since the award we used the context of our follow on grant for impact and engagement to extend the work with seven stories as described in the paragraph above. We worked to repackage the workshops into a repeatable activity and developed session plans and other resources. We engaged around 150 children over the two projects more than half of whom were from areas of Newcastle upon Tyne with low socio economic indicators. We are continuing to work with Seven Stories to identify avenues through which to continue to produce and develop this work. |
| First Year Of Impact | 2018 |
| Sector | Culture, Heritage, Museums and Collections |
| Impact Types | Cultural |
| Description | Follow on Funding for Impact and Engagement |
| Amount | £31,988 (GBP) |
| Funding ID | AH/S010661/1 |
| Organisation | Arts & Humanities Research Council (AHRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2019 |
| End | 07/2019 |
| Description | Working with Seven Stories |
| Organisation | The National Centre for Children's Books |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | We produced a series of public workshops leading to the release of a smartphone app in collaboration with our partners. We informed the organisations digital strategy through demonstrating the integration of design workshopping with public facing events for children. |
| Collaborator Contribution | Seven Stories staff contributed to the design and implementation of our workshops as described in our bid proposal. They were responsible for identifying audiences and publicising the events. They also provided access to the archive of the author David Almond who's work was the focus of our project. |
| Impact | We produced a series of 8 workshops applying magical realist ideas to the development of AR technology on smartphones. This involved -computer science -interaction design -literature We produced a smartphone app which implicated the same mix of disciplines |
| Start Year | 2017 |
| Title | Magical Reality |
| Description | Magical Reality is smartphone app for iOS and Android that embeds archival items in geolocated AR space. |
| Type Of Technology | Software |
| Year Produced | 2018 |
| Open Source License? | Yes |
| Impact | Alpha versions of the app were deployed during our workshops and used by participants in helping us imagine new forms of AR interaction. |
| URL | https://github.com/Digital-Cultures/magicalReality-Android |
| Description | Creative workshops with children |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | We conducted a series of 8 workshops with children and young people with between 10-25 participants. 7 of these took place at Seven Stories while one was conducted as part of Manchester Science Festival. In total more than 80 children and young people were engaged. We encountered very strong interest from participants with many returning to several workshops in our series. One session was conducted with a local school and staff reported high levels of interest in the children some of whom enquired about related careers after the event. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Love Among the Post-Industrial Ruins' at Emotional Discourses in Children's Literature symposium |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | This was a conference paper given at University of Western Australia which stimulated a lively academic discussion in the following seminar. |
| Year(s) Of Engagement Activity | 2018 |
| Description | workshops with professionals |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | We conducted a research on narrative in Augmented Reality and children's literature with colleagues from Seven Stories and with designers developers to trial new strategies for collaboration in the creative industries. |
| Year(s) Of Engagement Activity | 2018 |