Living with Digital Ubiquity

Lead Research Organisation: University of Nottingham
Department Name: School of Computer Science

Abstract

Since its formation in 2000 the MRL has focused on creatively interleaving the physical and the digital for everyday life. We have simultaneously explored new interactive possibilities and developed understandings of how these might be put to use in the real world. Much has changed over last 14 years; some of these changes we have driven, while others we have responded to.

When we began our journey, notions of ubiquitous, tangible and mixed reality computing were very much in their infancy, as were potential applications and methods. Our first seven years was therefore concerned with laying the conceptual and methodological foundations for these fields as well as demonstrating potential applications. Much of this was delivered through the Equator Interdisciplinary Research Collaboration (IRC) that we led until 2007.

We then used our previous platform grant to bridge the gap between Equator and the establishment of the Horizon Digital Economy Hub and Centre for Doctoral Training, and also the Orchid programme grant. The subsequent second stage of our journey has focused on driving the widespread adoption of ubiquitous computing as platforms such as smartphones and cloud infrastructures spread into everyday life. Now, as both Horizon Hub funding and Orchid imminently draw to a close in 2015, we need to once again reinvent our research agenda and also ensure our sustainability by bridging any funding gaps as we refresh our grant portfolio.

The focus of this next phase will be the overarching challenge living with digital ubiquity? The interleaving of physical and digital that was a distant vision back in 2000, is now becoming a reality for many, with a plethora of devices, from smartphones, to Kinects, to Fitbits allowing us to experience the digital in a bewildering variety of ways. The digital world has become rich, available 24/7, increasingly aware of our physical activities, and populated with our personal data. Ubiquitous technologies that were once considered radical are now becoming an unremarkable feature of our lives. Meanwhile, remarkable new technologies continue to emerge. Technologies for rapidly fabricating physical-digital products or for sensing and actuating the human body raise possibilities for radical new experiences.

Our vision of living with digital ubiquity is therefore one of balancing the challenges of living with increasingly mundane ubiquitous technologies, while also enabling people to experience and create extraordinary new experiences. We see this as a productive tension. People must both live in the everyday and yet also seek the profound and creative. It also reflects the essential - and highly creative - tension in the MRL between addressing everyday technology challenges as revealed by ethnographers, while simultaneously engaging with artists and performers to create provocative new experiences.

We therefore propose to exploit platform funding to catalyse a step-change in our research by identifying the new research challenges that will arise from living with digital ubiquity, exploring potential approaches to these challenges, developing our key researchers so that they are ready to face them, and forming the partnerships that will help us address them through future research projects. Platform funding will enable us to deliver a programme of exploratory projects, visitor exchanges, and impact projects to explore our vision while also enabling us to retain and develop key staff over the coming critical period.

Planned Impact

The MRL has a long history of delivering cultural impact by working with artists to create, tour and study interactive performances and installations that combine the physical and digital in unusual ways. Works produced since 2008 have toured to over 40 venues in 18 countries, have been experienced by over 200,000 people, and have received multiple arts awards. Cultural impact then drives economic impact, where we collaborate with major players in more mainstream entertainment to create new kinds of games and television shows. Prototype games produced with Sony and Nokia have garnered further industry awards, while our technologies and know-how have also been employed to produce articles for TV shows, from Blue Peter to Bang Goes The Theory and the One Show. In turn, these drive wider societal impact as our artworks often foreground and provoke public debate around societal challenges (e.g., works produced with the artists Blast Theory have tackled surveillance and terrorism and with Active Ingredient have tackled climate change), while features on mainstream TV shows draw the attention of the wider public to the potential of emerging technologies (e.g., highlighting the capabilities of wearable biosensors).

Platform funding will help us sustain this kind of impact in the future. Moreover, we will utilize the flexibility of a platform grant to pilot novel impact mechanisms that establish pathways to the new communities associated with our two research challenges. One important new pathway to establish concerns impact on the many millions of amateur and everyday creative practitioners that are increasingly engaging with digital technologies, and whose activities make a vital - if sometimes unsung - contribution to the economic and social wellbeing of the UK. A second, is to establish new pathways to impact on policy makers and shapers, including national and local government as well as campaign organisations.

Our proposal therefore includes a range of impact mechanisms, both to sustain our traditional pathways and also to open up these new ones. These include bespoke impact projects such as KE Secondments (supplemented with PhD internships) targeted at industry and policy partners; community engagements that take place in maker- and hack-spaces and other local venues where amateur and DIY makers can be found; and the release of products as impact probes, including downloadable apps, but also physical artefacts distributed through pop-up shops and craft fairs. Wider public impact will be enhanced by the production of a series of videos for the increasingly popular Computerphile YouTube channel. These mechanisms will be explicitly badged as 'impact' mechanisms so as to focus our researcher's attention on the increasing importance of impact.

Publications

10 25 50
publication icon
Benford S (2016) On Lions, Impala, and Bigraphs in ACM Transactions on Computer-Human Interaction

publication icon
Jäger N (2017) Reciprocal Control in Adaptive Environments in Interacting with Computers

publication icon
Malizia A (2016) Screwbots in Ubiquity

publication icon
North S (2016) Do Androids dream of electric steeds? in interactions

publication icon
Rennick-Egglestone S (2016) Families and Mobile Devices in Museums in Journal on Computing and Cultural Heritage

 
Title Apocalypse of MoP 
Description As well as giving gamers the chance to enter an online world as a double agent and infiltrate a corrupt government, new online alternate-reality game Apocalypse of MoP also uncovers players' perceptions of provenance and how this affects their decision-making. The game involves players answering questions and completing tasks relating to the digital provenance of objects. These decisions will then be used to assess how people process information and how important provenance - the origin of information - is in decision-making. It was developed by researchers at The University of Nottingham's Mixed Reality Lab in collaboration with Nottingham-based production company Urban Angel and researchers at the University of Southampton. 
Type Of Art Artwork 
Year Produced 2014 
Impact This online game was played by over 900 players over the course of s year who completed 13,000 provenance tasks. The game was the focus of a full technical paper at the ACM's Computer Human Interaction (CHI 2015) conference. 
URL http://apocalypseofmop.com
 
Title Karen 
Description Created by the artists group Blast Theory, Karen is an interactive artwork that explores the topic of psychological profiling by engaging audiences with an interactive video narrative delivered by a mobile app. We provided technical support fro delivering Karen and also supported a visiting residency for Nick Tandavanitj from Blast Theory. 
Type Of Art Artwork 
Year Produced 2014 
Impact Presentations: Film Festival, Storyscapes at Tribeca Film Festival, New York, 2015 Exhibition, Stedelijk Museum Bureau Amsterdam, 2015 Exhibition, ZKM Center for Art and Media, Karlsruhe, 2015 Film Festival, Festival du nouveau cinema, Montreal, 2015 Geneva International Film Festival, Monokino, Geneva, 2015 Awards: Data Category, Best of British Digital, British Interactive Media Association, Winner, 2015 Innovation Award, Festival du nouveau cinéma, Winner, 2015 The People's Lovie Awards, Experimental & Innovation, The Lovie Awards, Bronze Winner, 2015 
URL http://www.blasttheory.co.uk/projects/karen/
 
Title Oscillations 
Description Virtual reality swing rise that played with sensory discrepancies between visual and kinaesthetic stimuli. 
Type Of Art Artwork 
Year Produced 2016 
Impact Exhibition at Sheffield DocFest 
 
Title Scanners 
Description Scanners is a new immersive art installation that literally reads your brain waves and allows you use the power of your thoughts to edit the film you are watching. Created in partnership with arts group Albino Mosquito, Scanners is an interactive film controlled by a brain-computer interfaces. While the film always runs for 17 minutes and follows the same plot, viewers' blinking triggers cuts between views of reality and dreams while their levels of attention and meditation control the mixing of four different video layers. 
Type Of Art Film/Video/Animation 
Year Produced 2015 
Impact Scanners premiered at The Foundation for Art and Creative Technology (FACT) in Liverpool in July 2015 which it was screened for a week. It has also been shown at the ACM Creativity and Cognition (C&C 2015) and Computer Human Interaction (CHI 2016) conferences. 
URL http://www.albinomosquito.com/2013/06/scanners-teaser/
 
Title Sentiment 
Description An interactive installation in which participants navigate an array of speakers to piece together interviews with people about traumatic events and where the galvanic skin respond of the those people captured during the interviews is replayed as a series of vibrations on the listener's body. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact Staging of the exhibition for the Foundation for Art and Creative technology (FACT) at Liverpool. 
 
Title The Body 
Description A theatrical performance incorporating sensors and output devices embedded into physical props (dolls) and seating so that the audience can interact with the performance. The work was created in collaboration with the theatre group Shunt, supported by a prestigious award from the Samuel Beckett Trust and premiered at London's Barbican Theatre in November 2015. 
Type Of Art Artwork 
Year Produced 2015 
Impact It received excellent reviews including four star reviews from both the Guardian and The Stage. "Arguably more installation than theatre, The Body is bold in its rejection of story and embrace of technology. Those hoping for plot or character will be disappointed, but as a set of images and sensations it's often breathtakingly beautiful." (The Guardian, ****) http://www.theguardian.com/stage/2015/nov/22/the-body-review-barbican "Work of this calibre is worth shouting about" (The Stage, ****) https://www.thestage.co.uk/reviews/2015/the-body-review-at-the-pit-barbican-centre-london-bold-and-off-kilter/ "Impressive work by the University of Nottingham's Mixed Reality Lab" (LONDONist, ***) https://londonist.com/2015/11/hello-dollies-the-body-barbican-pit 
URL https://www.thestage.co.uk/reviews/2015/the-body-review-at-the-pit-barbican-centre-london-bold-and-o...
 
Title Thresholds 
Description Restating the world's first photographic exhibition in virtual reality 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2017 
Impact Yes to arise 
 
Title Touch-o-matic 
Description A two player arcade game that measures how much players are touching one another as a basis for controlling the game. 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2015 
Impact Experienced by 2000 members of the public when exhibited at the National Video Game Archive in Nottingham. 
 
Title VR Playground 
Description Series of interactive rides that play with sensory discrepancies between visual and kinaesthetic stimuli in virtual reality 
Type Of Art Artwork 
Year Produced 2017 
Impact Planning a major touring programme beginning with the Norwich and Norfolk Festival in May 2017 
 
Description A variety of interactive techniques for integrating ubiquitous technologies into everyday life and demonstrations of future applications.
Exploitation Route Used by various commercial partners including BBC, B3 Media and Unilever.
Sectors Creative Economy,Culture, Heritage, Museums and Collections,Retail

 
Description The Mixed reality Lab's Artist Residency Programme has engaged artists in creating new interactive works as cultural products.
First Year Of Impact 2015
Sector Creative Economy
Impact Types Cultural

 
Description ADAM: Anthropomorphic Design for Advanced Manufacture
Amount £269,486 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 01/2016 
End 07/2017
 
Description Digitally Enhanced FMCG
Amount £426,000 (GBP)
Organisation Unilever 
Department Unilever Research and Development
Sector Private
Country United Kingdom
Start 11/2015 
End 10/2018
 
Description FAST Programme Grant
Amount £5,119,944 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 06/2014 
End 06/2019
 
Description GIFT - European Union Horizon 2020
Amount € 500,000 (EUR)
Organisation European Union 
Sector Public
Country European Union (EU)
Start 01/2017 
End 12/2020
 
Description Horizon Follow-On Fund
Amount £4,062,954 (GBP)
Funding ID EP/M02315X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 08/2015 
End 07/2020
 
Description B3 Media 
Organisation B3 Media
Country United Kingdom 
Sector Private 
PI Contribution We have jointly run three iterations of the annual the Talentlab Programme to recruit and train young black and minority and ethnic (BAME) artists in the design and creation of interactive artworks.
Collaborator Contribution B3 media have coordinated the programme, providing core training, recruiting artists and working to secure opportunities for presenting the work to sponsors in the creative industries, for example a showcase at BAFTA.
Impact To date we have trained 30 new artists by running workshops, hosting residencies and supporting the development of prototypes.
Start Year 2014
 
Description Chinese University of Hong Kong 
Organisation Chinese University of Hong Kong
Country Hong Kong 
Sector Academic/University 
PI Contribution Supporting Christian Greiffenhagen as a visiting researcher to our lab.
Collaborator Contribution Collaboration on a series of research studies.
Impact Papers under preparation
Start Year 2016
 
Description FACT 
Organisation Foundation for Art and Creative Technology (FACT)
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution We developed a exhibition called Performing Data for them.
Collaborator Contribution They hosted the exhibition, delivering an estimated audience on excess of 1000 visitors. They subsequently have engaged as external supervisors for a PhD student.
Impact In prepearation
Start Year 2015
 
Description National Videogame Archive 
Organisation National Videogame Archive
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Providing a series of interactive experiences that we hosting at the Archive as public exhibitions.
Collaborator Contribution Hosting of our experiences so as to open them up to thousands of members of the public.
Impact Papers arising from studies of public interaction.
Start Year 2016
 
Description Unilever 
Organisation Unilever
Department Unilever UK R&D Centre Port Sunlight
Country United Kingdom 
Sector Private 
PI Contribution Developing concerts, enabling technologies and prototypes to explore how augmented fact moving consumer goods connect to smart appliances to augment to consumer experience as key touchpoint during their interaction.
Collaborator Contribution Direct contributions to research from Unilever staff as well as cash funding for our research.
Impact Four patents filed. Papers in preparation.
Start Year 2015