Sound and Vision Scapes (SViS)

Lead Research Organisation: University of Greenwich
Department Name: Educational Development Unit

Abstract

Sound and Vision Scapes (SViS) integrates the social Internet of Things software platform Tales of Things with the BBC media archive in order to link video and audio content to places and objects via mobile devices. Project SViS will augment a combination of 100 objects and places objects throughout the UK to enable the public to engage with the BBC archive in novel ways that increase access and provide new opportunities to participate in the replay and social recording of objects, places and spaces. For this purpose we will utilize 'broadcast beacons' to allow read/write access to the archive directly linked to the object or place. These beacons, based on Bluetooth Low Energy (BLE) technologies, will broadcast the availability of provenance information to BLE enabled devices (phones, smart watches) in the proximity (up to 25m distances). The project builds on the next generation of the Internet of Things and utilises the provenance/read write platform of Tales of Things to allow a seamless interaction with media and objects/places of cultural heritage significance as a means of gathering and displaying information.

Talesofthings.com is part of the EPSRC funded Tales of Things and electronic Memory (TOTeM) project. It represents the worlds largest archive of Internet of Things connected second hand objects with over 10,000 'things' connected to stories, tales and information about their provenance. The project has received over 40 articles in the international press, was named as part of Wired's Top Technology 2011, been utilised by Oxfam as part of their ongoing Shelf Life project and won the cross research council impact award 2011. The linking of the project with the BBC archive, along with the inclusion of geographic location and use of broadcast beacons, will provide a unique roadmap to opening up a wealth of media and audio data to location. An example of this would be a broadcast beacon located on a bench in Lyme Regis on the Dorset Coast, broadcasting a 10 minute short from the BBC Series 'Coast' with information on the Jurassic Coast. The viewer would be able to contribute comments, photos or videos of their time at the location, viewable to the next person who viewed the archive from the beacon and thus adding a social network aspect to not only the archive but also the object, place and location. Another example would be the tagging of a tree in a public park, linking to media resources applicable to wide range of age groups from 'Trees and Tall Things' from the Show Me Show Me CBBC series to Gardeners World. This aim is to link information about objects, places and spaces directly to the BBC archive while also combining the crowd sourced/social network nature of Tales of Things using known systems and techniques.

It is to be expected that the proposed project will generate new types of data about interactions with cultural heritage. This data can be used for the creation of insightful visualisations, it can inform the development of new data analysis tools and methods that support research in the social sciences, arts and humanities. The project proposes the creation of a social cultural heritage platform that allows researchers to explore how people perceive cultural heritage. We aim to investigate new social practises of creating and managing social memories linked to places. Consequently our research also explores novel forms of digital citizenship that can help to democratise how cultural heritage is recorded. In order to study these new phenomena the project will create new tools and methods to study and analyse the generated heritage data.

The use of established and proven technologies enables the project partners to carefully manage risks that are inherent to projects that involve software development. By applying existing technologies the associated risks are low and software development will be restricted to customization and integration of the different backend systems.

Planned Impact

There are a number of impact beneficiaries of the proposed intervention. First of all SViS will increase opportunities for engagement with cultural heritage resources to the general public. This will encourage people to learn about the identity of places by broadcasting information about the provenance of a place and past events linked to it. Hence the expected impacts of the project amongst others are:

- Improved access to affordable and widely available tools and services for releasing the economic potential of cultural heritage in digital form and for adding value to cultural content in educational, scientific and leisure contexts;

- Engagement of a wide range of users with cultural resources in diverse real and virtual contexts and considerably altered ways to experience culture heritage as a personalized and adaptive user experience

The proposed social software system that emerges as a result of integrating Tales of Things and the BBC broadcasting archive will enable people to participate in the discussion about cultural heritage and its preservation. Hence the intervention provides learning opportunities for members of the public and it also enables novel forms of participation and citizenship that inform decision-making and initiatives from local authorities. Consequently an outcome of this project will be a new type of social software system for the digital cultural heritage sector. The resulting software artifact will provide insights into possible avenues for commercialization, the design of socio-technical systems for cultural heritage and add-on services with economical viability.

Enabling people to add their own voice also encourages the democratization of recording cultural heritage and social memory. This is important as it will help marginalised groups having their voices heard in a sector that is otherwise dominated by the cultural industry. There is then a risk that hyperlocal heritage and provenance gets forgotten due to a potential bias for settling for a mainstream perspective. Citizens are rarely involved in curating cultural heritage experiences. While the recent uptake of digital technologies has led to improved opportunities for audience feedback in general, and to a certain extent for social interpretation, these approaches tend to only reach an already interested audience. Unfortunately too often cultural minority groups and less affluent or culturally less important regions are under-represented in heritage applications. The Internet of Things-approach that we are putting forward enables new opportunities for people to connect with the culture of place in everyday activities.

The proposed project also provides a stimulus for the economy and in particular the creative industries and tourism sectors. The project will provide pointers for new tourism applications that integrate tailored media access when interacting in-situ with places in the real world through the use of Internet of Things technologies. The use of low-cost ubiquitous mobile technologies will allow greater accessibility to informal learning in groups that may not normally visit museums or cultural sites/institutions. We also envision that our intervention will support novel forms of inquiry learning where students can probe and question their perceptions through interactions with places that have been augmented.

The intervention also unlocks new user-generated data resources that researchers can tap into, visualise and analyse. Hence it enables researchers to work with new data sets and explore novel research questions relating to cultural heritage. Peoples' perceptions and attitudes towards cultural heritage artefacts that are currently largely hidden will be made visible and thus can be used for discussion and mediation of meaning.

Publications

10 25 50
 
Title A map that shows potential locations for IoT enabled interactive objects 
Description This was a second a map we created that shows were on site interactive IoT enabled objects could be located. 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact This map was instrumental in planning fieldwork and in exploring the logistsics of having IoT enabled interactive objects on site. 
URL https://www.dropbox.com/s/ej665dmfsm85db1/230429_map.png?dl=0
 
Title All recorded audio files 
Description All the sounds and voiceovers that were recorded for the project. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/sh/9bje9m6n2tfllkt/AAAyIKfGdUYGJvAk6FgSNQfYa?dl=0
 
Title Audio story 1 
Description Edited version of a sound recording of an actress telling on-site stories based on a script from a researcher. We created six different stories. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/enfc3dqb185xsny/150717_story2_v3.wav?dl=0
 
Title Design Concept IoT Augmented Bench 
Description Detailed rendering of a prototype device for the augmentation of a bench on-site. 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact The rendering was used as a blueprint for prototyping and 3D printing of a physical prototype 
URL https://www.dropbox.com/s/qh4ihfyo4ymhfmo/150826_exploded%20persp_DIAGRAM_FINAL.png?dl=0
 
Title Final rendering of a prototype device 
Description Renderign of the final version of a prototype device 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact Contains all the parts and details for 3D print and assembly 
URL https://www.dropbox.com/s/2jwcg7kcmr23zw0/150428_augmented%20bench_FINAL.png?dl=0
 
Title First photo of the bench augmentation kit 
Description Details of the 3d printed prototype of a bench augmentation kit 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/ng8iyrfoncytx11/2a.jpg?dl=0
 
Title Second photo of the bench augmentation kit 
Description Details of the 3D printed prototype 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/wmx2g38754n58rh/3a.jpg?dl=0
 
Title Sound List 
Description List of the sounds that were recorded to augment the storytelling 
Type Of Art Creative Writing 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/qza2venvdpk6amw/150514_Sound%20List.pdf?dl=0
 
Title Sound script 
Description Script for the voice recordings of on site stories. 
Type Of Art Creative Writing 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/0tlfje7wjtb33p5/150315_SOUND%20SCRIPT.pdf?dl=0
 
Title Third photo of the bench augmentation kit 
Description Details of the 3D printed and assembled bench augmentation kit 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact Was used to elicit feedback from people that experienced the device in our research. 
URL https://www.dropbox.com/s/wzkkie77p7o6m6r/4a.jpg?dl=0
 
Title University of Greenwich Campus with film locations and locations of significant historic events 
Description As part of our research we created a map of the University of Greenwich Maritime campus overlayed with the location were films were shot on site and their genre. The map also shows significant historic events that took place on site. 
Type Of Art Artefact (including digital) 
Year Produced 2015 
Impact The research that informed the creation of this map supported us envision how the campus could be augmented with interactive IoT-enabled on-site objects. 
URL https://www.dropbox.com/s/ax50a8zr59r4se3/150212_GreenwichCampus.jpg?dl=0
 
Description Through this grant we developed a new approach with a low entry barrier to participation in digital cultural heritage through interactions with IoT (Internet of Things) enabled objects. Many IoT technologies such as Bluetooth low energy or NFC require smartphones and bespoke apps for consuming locative media and content. We found that these relatively high technical requirements restrict uptake and audiences for digital cultural heritage applications and public engagement. This led us to reconsider our approach to employing IoT in cultural heritage and urban spaces which are the primary contexts of our research. Initially we planned to emply IoT as being enabled by wearable and ubiquitous computing devices that people carry whilst interacting with hidden tags and technologies (e.g. Bluetooth Low Energy) in-situ. However, it emerged through our research that creating visible, aesthetic and evocative objects in the built environment with which people can interact without the need of sensor equipment smartphones is a much more desirable, accessible and suitable approach to engage with cultural heritage and informal learning.

During the duration of the project the research team developed and improved skills to research this interdisciplinary space and we found design research and design fiction to be a valuable approach in this context. We developed new concepts, artefacts, performative media, visual representations and prototypes which helped to balance different perspectives and to start conversations with different potential stakeholders in employing emerging technologies such as IoT to explore cultural heritage.
Exploitation Route Our research has led us to reconsider how people can access Internet of Things technologies and how participation in memory creation and social interpretation can be achieved with a low barrier to entry via using social media (e.g. Twitter) and modes of interaction (text messaging) with IoT objects in the built environment. We are currently exploring the commercialisation of new types of IoT enabled information mediators, that are informed by the research of Sound and Vision Scapes, in collaboration with a commercial partner. This project is funded through and University of Greenwich proof of concept award. This follow-on work leads to a marketable concept and a technical product that is of interest to the museum sector and we expect this work to inform similar offerings that heritage institutions might use for visitor engagement.

The research outputs and challenges are of interest to an interdisciplinary audience and our design research provides interesting insights how design fiction and traditional social science research methods can be combined to design valuable activities with Internet of Things technologies in a digital heritage context. Our prototypes and the location map of films are example artefacts of this approach.
Sectors Digital/Communication/Information Technologies (including Software),Education,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections

URL http://svis.uk
 
Description Outcomes from our research have been instrumental in informing a new project (information mediators) that is currently underway. The "Information Mediators" project aims to create a new half digital, half physical experiential environment on the University of Greenwich Campus through enhancing the built environment with physical, interactive IoT enabled site objects. The main goal of the project is to create an opportunity for students and visitors to explore and discover provenance information about the site in a creative and engaging way, through an interaction with physical devices that are deployed around the site. These devices will, apart from delivering information also absorb them, acting as a "Speakers Corner" for students, staff and visitors giving an opportunity to everyone to express their ideas, complaints and dreams regarding life on the university campus. The research supported by this award has helped to further develop the skills of the research team to explore commercialisation of IoT enabled objects in the built environment in conjunction with a industry partner in a wider context. We have been successful in securing follow-on funding for business planning and necessary additional technical development. We believe that the outcomes of the project are suitable to benefit individuals and organisations alike and they will be used in different sectors to increase public engagement and public participation in other secotors such a wellbeing.
First Year Of Impact 2016
Sector Education,Healthcare,Culture, Heritage, Museums and Collections
Impact Types Societal,Economic

 
Description Proof of Concept fund
Amount £30,692 (GBP)
Funding ID HEIF-PoC-ACH-02/15 
Organisation University of Greenwich 
Sector Academic/University
Country United Kingdom
Start 12/2015 
End 06/2016