A framework for designing Mixed Reality situated learning experiences in cultural heritage sites using AR glasses

Walking through a cultural heritage site is an immersive educative experience. The physical space itself and the artefacts that are laid out in it tell many stories of the place's history through audio guides, labels, videos, and interactive touch screens. However, the site remains a static display, which is only revived through the imagination of those who visit it through the things they learn about it. Past research has shown that immersive Mixed Reality (MR) technologies such as mobile Augmented Reality (AR) have the potential to bridge the gap between the past and the present during the visitors' walkthrough, and enhance their learning experience. However, mobile AR relies on handheld devices, poor tracking quality of the user, and the digital content is only superficially super-imposed on the physical world not interacting with the physical objects in a realistic way. AR glass technology, also called direct AR, has the potential to break these barriers and provide significantly increased immersion and a plethora of interaction mechanisms to the visitors, both of which can considerably enhance their learning experience.

The proposed fellowship seeks to explore a design framework for developing engaging, situated learning experiences in cultural heritage places for teenage and adult visitors (in the 12+ age group) with the use of AR glasses. AR glasses are a pair of wearable glasses with the ability to extend the visible physical space using superimposed digital content, including holograms, which is so well positioned that it blends successfully with the physical surroundings. They are an emerging MR technology, which has not yet been adequately researched within the context of cultural heritage education. In collaboration with professionals from heritage education, design, and MR development the project will explore the challenges and opportunities in relation to the use, adoption of and engagement with the AR glasses by visitors of diverse demographics. It will then seek to understand what is lost, gained, transformed, and confused in the visitor's learning experience during the transition from a simple to an augmented walkthrough, and build a design framework based on these insights.

The research outputs will highlight implications for future design approaches for MR experiences within the heritage context. The research will demonstrate in what ways AR glasses increase the immersion as opposed to other MR technologies, and explore their educational challenges and opportunities to create compelling educative MR walkthroughs of a heritage site. Accordingly, the programme of research will investigate the type of MR content that can be built upon the physical and educational capabilities of the device. The collaboration has the potential to promote good practices in using emerging technologies, create value for the cultural organisation that will facilitate its design and the MR developer company that will develop it, and highlight emerging skills in the MR sector.

The proposed fellowship comprises a 3-month period of preliminary research, which will bring heritage professionals, particularly educators, and MR developers together to identify the current landscape in terms of educational experiences using immersive technologies for visitors of the specific age group, discuss how MR can engage visitors through meaningful educational interactions, and create a document of considerations. The MR developers will then use this document to design the content for the MR application. This will be followed by a 3-month period of carrying out user evaluation with the public, iteratively adjusting the MR content based on the received feedback. Finally, a 3-month period of reflection, writing, and dissemination will produce a document introducing the design framework, a Mixed Reality application, two journal articles, one conference paper, a website, and a symposium.

The non-academic user-groups who will benefit from the research include heritage educators, Mixed Reality developers, interaction designers, and the general public.

The proposed fellowship will benefit heritage educators by enabling them to develop design skills to improve the ways they record, preserve and make available cultural heritage. Through the project's activities heritage educators will gain knowledge in the opportunities that Mixed Reality (MR) presents for situated learning, and develop skills to work with it to design meaningful, fun learning activities. With the predictions that AR glasses will become a prominent immersive technology in the future, such skills will be valuable for creating the future digital museums and bringing in new audiences. The proposed activities will seek to break silos between disciplines, enabling curators, educators, archaeologists, historians, and directors to create a vocabulary for sharing their vision with technologists.

By participating in the co-design process the MR developers will work closely with a multidisciplinary group of people who bring an arts and humanities approach to the design of MR applications, and they will practise iterative design informed by the feedback of the public. AR glass technology is a novel field that is normally driven by engineering rather than design approaches. This experience will have an impact on the industry's perspective in developing future projects for the heritage sector, and other sectors within or outside the creative industries. It will also inspire them to engage with the general public and allow them to develop skills to do so. The symposium at the end of the project, my online posts along the process in related online media, as well as the company's blog posts about the project in their website will extend the impact to the wider MR developer community. In addition, the project is using MR technology and at the same time explores potential designs that appropriate it as a tool for situated learning on a heritage site. The project's outcomes will relate to both the hardware and software development of MR applications. The proposed activities have knowledge exchange at their core and aim to strengthen the links between academic and industrial research and development with direct applications for the benefit of both.

The general public will have access to a new application that provides them with new methods of learning about the history of the site they visit. The final version of the MR application will be available to those who own a Hololens device and wish to experience the past in an immersive way during their visit to Sutton House. The project's methodology and insights will be transferable and applicable to other heritage sites in the UK and globally so the impact on the audience engagement can be scaled considerably. In addition, public engagement is at the heart of the MR application's design methodology to ensure the output is of high quality and a meaningful experience for its end-users, the public. Members of the public, as visitors to the house, will be consulted on the design of the experience, reflect on the design decisions of the project team, and their feedback will be analysed informing further adjustments of the MR content. Through the feedback process they will learn about the project and its research questions and discuss their views and ideas of what they would like to experience. Their participation will help them demystify the working mechanisms behind the design of interactive content while their interaction with the project team on site can inspire the next generation of researchers.

The symposium will help the project develop a network of academic and non-academic audiences that can drive future collaborative research. The project's outcomes will strengthen co-design paradigms for interdisciplinary research and development and exemplify the importance of end-user engagement in the process.