Mediated Urbanism: Data, Ubiquitous Digital Sensors and Interactive Screen Media in 21st Century Urban Advertising Practice

Lead Research Organisation: University of Liverpool
Department Name: Architecture

Abstract

Mixed-reality, augmentation technologies and sensor networks enable advertisers to radically re-contextualise the function of physical space as a platform for digital storytelling and advertising innovation. As connected sensors, bio-informatic data capture and pervasive media technologies develop across the built environment, through the expansion of the internet into physical space and things (Internet of Things, Smart Cities), inanimate objects and physical locations will increasingly become an opportunity to extend storytelling and interactive advertising experiences across our urban landscapes.

Through the development of SMART urban infrastructures and the placement of sensors throughout the city, the urban environment increasingly becomes a physical platform, or software, for digitally enhanced augmented experiences. This enables organisations to connect people to stories, spaces and experiences in highly personalised and contextually aware ways. Responsive technology has the capability to give city architectures, and its 'mediatecture', a fluidity and adaptability to its environment to effectively become a dynamic, living and sensing entity within highly personalised flows of urban visuality, sensuality and playability. This has profound implications for the way we think through questions around personal data, our own embodiment within the city and its relationship to the new forms of mediated public and private space emerging in contemporary digital culture.

This PhD is part of a collaboration with McCann Manchester as key partner and supported by the Virtual Engineering Centre (VEC). Key research questions include theorising how contemporary advertisers and media artists are using sensor-based content approaches within location specific contexts and how this is leading to new forms of narrative expression and relations within urban environments. The project will create a theoretical basis for the practice-based elements of the research, which will aim to develop a prototype for forms of interactive, augmented and playable advertising that can be both mass-personalised, made location specific as well as responding to the needs of its stakeholders. Other research questions related to this will address important social and ethical concerns around the gathering and use of personalised data in physical spaces and the design implications of developing sensor networks connected to mediated architectural practice.

Publications

10 25 50
 
Title If these walls could talk - Mixed Reality Experience 
Description "If these walls could talk" is a collaboration with the Centre for Architecture and the Visual Arts (CAVA) at University of Liverpool, Culture Liverpool and Draw and Code. We are creating an interactive mixed reality Hololens prototype that will take place in the old 19th Century jail cells in the iconic Unesco Heritage building St. Georges Hall. Using cutting edge holographic capture technology to create interactive performances with characters from the past, this experience enables visi 
Type Of Art Artefact (including digital) 
Year Produced 2019 
Impact Led to other commercial opportunities 
URL http://www.cava-research.org/research-projects/st-georges
 
Title Secret Coast 
Description Peter Woodbridge's company, Immersive Storylab - with the support of partners including CAVA, LIPA and a number of other creative businesses in the region - has recently been selected to take part in an Arts Council and Digital Catapult scheme to develop a prototype for an augmented reality experience along the UK Coast called Secret Coast His entry was selected from over 250 applications as one of 20 CreativeXR projects (http://creativexr.co.uk/), to support new creative content formats enabled by immersive technology, such as virtual reality, augmented reality and mixed reality. The programme gives teams the opportunity to focus on R&D and develop riskier, content-driven projects that contribute to developing new skills, tools and business opportunities for the UK's creative content industries. Using augmented reality's unique capabilities at revealing invisible history and bringing place-based cultural heritage to life, Coast AR will enable users to re-encounter the UK's coastal landscape through a story driven experience that melds together fact and fiction with folklore and mythology. The initial prototype will be developed for a stretch of the River Mersey and use a combination of GPS technology and augmented reality, alongside gamification features such as the ability to collect virtual items, solve puzzles and dynamically interact with the storyworld. Pete and other researchers are exploring new narrative and placemaking possibilities afforded by immersive technologies, which is a growing area of research for a number of ongoing R&D projects at CAVA. 
Type Of Art Artefact (including digital) 
Year Produced 2016 
Impact Got to pitch the project to investors and has led to further R&D work in the area of augmented reality tourism applications 
URL http://www.cava-research.org/research-projects/coast-ar
 
Title Storyscopes 
Description StoryScopes are public viewing telescopes that I developed that tell stories using augmented reality and sound. The device enables people to see stories in landscapes and bring environments to life with history and future visualisations. From seeing history and imagination unfold before your eyes in mountains and the oceans, to visualising Urban development and enhancing landmarks, the storyscopes device is aiming to get people to learn more about their surroundings by combining the digital and physical world. Using head tracking and games based interactions, users can also interact with the digital and physical content they see using the device, from playing games across cityscapes to engaging in interactive narratives. There's also the ability to teleport into virtual world's and see beyond the immediate environment Installation options include single devices and multiple scopes to encourage exploration through stories, mysteries and interactivity. We also have screen based (holographic) viewing options to enable a number of ways for groups to share the experience. There is also the option for the device to take micro-payments for revenue generation. 
Type Of Art Artefact (including digital) 
Year Produced 2019 
Impact Led to a collaboration with the Abandon Normal Devices Festival 2020, working with McCann Manchester and Artists at realising an installation for Telescopes in May 2020 
URL https://www.curiousmagic.co.uk/storyscopes
 
Description Several technological and user experience breakthroughs in Immersive Experience development in Mixed Reality and Augmented Reality. Including new software development and applications of digital technologies to placemaking activities in urban environments and high streets.
Exploitation Route Use by companies and practitioners working in the medium of Augmented Reality and Mixed Reality
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections

 
Description Have consulted a number of companies and spoken at business events about the findings of the research, primarily in the North West region but also internationally (France, China and Ireland).
First Year Of Impact 2018
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections,Transport
Impact Types Cultural,Economic

 
Company Name IMMERSIVE STORYLAB LTD 
Description We are storytellers, artists and researchers working with immersive technology, augmented reality and emerging technology to create imaginative possibilities for the future of place. We use immersive technology, augmented reality, holographic projection and spatial computing to bring storytelling magic into public places, environments and landscapes. Our experimental and interdisciplinary R&D work fuses UX design, creative writing, theatre, games, interactive media, cinematic arts and mixed reality technologies together to think about the transformative possibilities for story in our everyday lives and environments. Our experience includes working with heritage & arts organisations, museums & galleries, the transport sector, creative agencies, brands, film and media companies. We collaborate with a number of artists, designers, organisations and companies who want to combine storytelling with a range of interactive and immersive arts technologies to make imaginative possibilities for the future. We believe in shared storytelling experiences, and their emotional impact, as the most important force for positive and transformative change in our world and we are particularly keen to work on projects that help make a difference to people's wellbeing, our environment and the places we use together every day. We also create learning, consultancy, workshops, events and training packages for organisations and business interested in the huge potential of immersive storytelling, technology and arts. This includes running the events programme for Immersive Liverpool and working on collaborative research with Universities. 
Year Established 2017 
Impact Has won major funding from Innovate UK
Website http://immersivestorylab.com/
 
Description Immersive Liverpool 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact Co-Ordination of the Immersive Liverpool strategic group in partnership with companies and public sector bodies in the UK
Year(s) Of Engagement Activity 2019
URL https://www.curiousmagic.co.uk/immersive-liverpool
 
Description Immersive Liverpool 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Immersive Liverpool is a networking group set up by Peter Woodbridge for everyone in the region interested in Immersive Technology including Augmented Reality, Mixed Reality, XR, Virtual Reality and emerging interfaces (Haptics, VUIs).

The aim is to bring together people working across the digital and creative sectors in Liverpool, including artists, developers, creative technologists, researchers, designers, content makers, businesses, freelancers and emerging talent to think about the future opportunities that immersive technologies can bring to various sectors in the region. We are also keen promote the pioneering immersive work coming out of the regions businesses, Universities and organisations.

In particular, the group should interest anyone looking at building work across the spectrum of mixed reality and augmented reality devices. Especially people working with Unity, Unreal, Hololens, ARKit, ARCore, Vuforia, Meta and other software, SDKs, MR headsets, peripherals and haptic devices related to Immersive Technology.

Our aim is to develop a community that shares ideas, skills, resources, funding information, demos and knowledge in the growing immersive sector and to help people meet like-minded folk to talk about developing new projects, experiments, experiences and applications.

Over the course of 2018 we ran over 8 events focused on a number of industry sectors, including health, creative industries, games and heritage sectors.
Year(s) Of Engagement Activity 2017
URL https://www.meetup.com/Immersive-Liverpool/
 
Description Immersive Storylab Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact 4 day workshop organised in collaboration with Brightblack Ltd. Looking at techniques and approaches to Immersive Storytelling/AR/VR etc etc
Year(s) Of Engagement Activity 2019
URL https://www.eventbrite.co.uk/e/storyhack-liverpool-tickets-58077634769
 
Description Immersive Storytelling Experience Research Symposium 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact On December 14th, we welcomed over 200 guests from industry and academia, with over 30 speakers and demos, to discuss the way that Immersive Technologies such as AR, VR, MR and XR are impacting on storytelling. Organised by myself.

The day included 8 panels exploring the diverse ways in which people are engaging with new technologies. Speakers included creative writers, theatremakers, game developers, filmmakers, artists, creative coders and academics discussing about how they are are approaching the emerging mediums of virtual reality, augmented reality, digital theatre, projection mapping and mixed reality for storytelling experiences.

Looking at research into languages of production, narrative mechanisms and approaches to engaging audiences in experiential media across the spectrum of immersive technology.

Speakers included thinkers and creative talent from BBC R&D, Digital Catapult's Creative XR Programme, Bristol VR Lab, York Creativity Labs, Institute of Creative Technologies, Liverpool Screen School and practice-based academics from a number of Universities from across the UK.

Companies included Marshmallow Laser Feast, Draw and Code, Limbik Theatre, Pilot Theatre, Boom Clap Play, Kinicho, Production Park, The Hatch, Mnemoscene, Immersive Storylab and Charisma.ai. There were also a number of demos including a 6 metre immersive dome from Igloo Vision and a Magic Leap, a new Sonic Reality engine by Kinicho and a hololens project by UWE's Robert Eagle.

Liverpool 's own work on using immersive technology for storytelling about climate change and their use of mixed reality and augmented reality to create novel storytelling experiences was also on show. The day also included work from the AHRC's Immersive Experiences programme including projects that enable people to experience a forgotten castle in Sheffield and a digital ghost hunt project in London.

There were also many other projects on display that give a glimpse into the future, including a project to bring augmented reality to trains across the UK, a holographic theatre in a box, a project that used mixed reality to reconnect Somali communities with heritage, dance and performance projects that blend digital and physical space, and talks about the way that programmers are working with the oral storytelling traditions. The event also explored how digital place is affecting physical communities and there were also projects that were using storytelling to tackle health issues and even to question question gender norms. People talked about MoCAP, Volumetric Video and new tools for creating more realistic environments in virtual worlds.

Pete Woodbridge, Event Producer and Senior Lecturer in Digital Media at Liverpool Screen School, said, "Storytelling is certainly going through a renaissance and it's amazing how immersive technology is bringing so many creative disciplines together. We had such a range of thinking and approaches being discussed on the day and its a really great time to be working in such a collaborative area of research and practice. There's real desire to move storytelling beyond the limitations screen and put people inside of experiences, or to paraphrase Robin McNicholas from Marshmallow Laser Feast 'its about taking things out of the rectangle and into the real world'.
Year(s) Of Engagement Activity 2018
URL http://immersivestorylab.com/wp-content/uploads/2018/12/Immersive-Programme-Final-Print.pdf
 
Description Woodbridge, P (2018) Augmented Reality Storytelling, At the Laval Virtual International Conference, France 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited Talk at the Laval Virtual Conference in France
Year(s) Of Engagement Activity 2018
URL https://www.laval-virtual.com/
 
Description Woodbridge, P (2019) Immersive Storytelling Panel Member at ARVR Innovate Dublin 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Woodbridge, P (2019) Immersive Storytelling Panel Member at ARVR Innovate Dublin Conference
Year(s) Of Engagement Activity 2019
URL https://arvrinnovate.com/pf/25855/
 
Description Woodbridge, P (2019) Immersive Storytelling in Tourism Talk at Advantage Austria Event in Dublin 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A talk at an event on Immersive Storytelling in Tourism, organised by Advantage Austria

Woodbridge, P (2019) Immersive Storytelling in Tourism Talk at Advantage Austria Event in Dublin
Year(s) Of Engagement Activity 2019
 
Description Woodbridge, P (and others) (2018) Immersive Storytelling Panel - Augmented Reality Storytelling at ARVR Manchester International Conference. ARVR-Conference-Programme-2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk at an international conference
Year(s) Of Engagement Activity 2018
URL https://www2.mmu.ac.uk/creativear/conferences/2018-augmented-and-virtual-reality-conference/
 
Description Woodbridge, P, McCartan, S Eilbeck, A (2020) 'Maritime augmented reality, as a tool for immersive experiences for ferry and cruise operators between the UK and Ireland, creating new commercial opportunities for the marketing, advertising and tourism sectors'. Marine Design International Conference, RINA, Cadiz, Spain 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Conference precedings

Woodbridge, P, McCartan, S Eilbeck, A (2020) 'Maritime augmented reality, as a tool for immersive experiences for ferry and cruise operators between the UK and Ireland, creating new commercial opportunities for the marketing, advertising and tourism sectors'. Marine Design International Conference, RINA, Cadiz, Spain
Year(s) Of Engagement Activity 2019