Next generation paper

The technology of paper is so old and so ubiquitous that we have stopped thinking of it as a technology. Nevertheless, paper and ink has continued to evolve from the earliest experiments to record handwriting on parchment in China 2000 years ago, to the medieval printing press in Germany, and modern advances in the printing of true colour digital images. Until recently, these developments have treated paper as a passive reading surface for the presentation of visual information. However, the advent of printed electronics and augmented reality have brought a new interactive possibility to paper: that it could communicate with things around it. In particular, it can be made to trigger the presentation of overlaid or adjunct information in both visual and auditory form on nearby digital devices. This is a potentially revolutionary development of paper for the digital content economy, because it begins to tie together as never before, traditionally separate media forms of printed books and magazines with music, film and information on the world wide web. Paper can become both a carrier of information and an interface to further information beyond itself.

In this project, we aim to develop these properties into a generic mass media 'next generation' paper. We will harness developments in both image recognition and printed electronics to explore two technical trajectories for the support of interactive paper over the short and long term, within a common interaction framework. This framework will be tied into new and emerging MPEG standards for media linking, to allow media producers to author hybrid print-and-digital interactive documents in a common international format. The framework itself will be generated by a multidisciplinary team of scientists and engineers working in a promising domain of application, and embodied in an authoring toolkit for creating interactive e-books with printed components. Computer scientists and electronic engineers will collaborate with media and interaction designers, social scientists and business researchers to design, explore and test the paradigm of interactive printed books for travel and tourism.

The travel domain is both promising and challenging, because it still makes use of both printed and online information in professional content to guide tourist choices, and in the capture of tourist reviews and experiences in social and personal media. For this reason we will work with travel companies and the publishing industry to generate a series of professional travel books which have interactive properties for playing back additional content on a variety of devices. We will also work with tourists themselves to evaluate the usability and usefulness of these interactive travel books, and to create multimedia photobooks or travelogues of their own travel experiences, with content separated across paper, screen and speakers. These investigations will be informed by a prior ethnographic study of the use of paper and screen-based information across the tourist journey, to identify the problems and opportunities for combining them in new ways. They will also be underpinned by core technology development to realise a practical and robust way of triggering media playback from page turning and pointing actions over the surface of a printed book, sensed either by an overhead camera or by light sensitive patches wired to a wireless chip in the spine of the books. Finally, the commercial potential of the technology will be examined and defined in a dedicated business investigation in the second half of the project. This will involve consideration of the immediate and short term potential of image recognition for interactive paper, as well as the longer term potential of printed electronics where a camera is not required.

A next generation paper (beyond '1GP') will open up countless opportunities, enabling new forms of graphic design for hybrid physical-digital documents in which content is shared between paper, speaker and screen. It will support new forms of multimedia literacy and expression that extend currently separate forms of expression in the creative industries stemming from film, music, radio, TV, books and the web. It will also make hidden digital content more tangible and accessible, allowing simple indexes to be printed in a variety of forms for circulation, display and storage in the physical world. Consequently, we envisage a very significant impact, in the following domains.
KNOWLEDGE: The project will impact all the academic communities represented by the project team (see also Academic beneficiaries). For example, in augmented paper and interaction design the project may realise new benefits of augmentation, and explore tangible paper interfaces to computing. In the computing area the project will extend previous architectures for cross-media representation and linking into the paper domain, and extend international ISO MPEG standards to include physical objects and interactions. In printed and embedded electronics, the project will advance the development of practical paper and flexible electronics, and showcase a new application to augmented books. In cross-media publishing, the project will enable transmedia storytelling with paper. In tourism and digital geographies, the project will extend augmented reality to paper and contribute new theory on the cultural concepts of 'convergence culture' and 'interface'. In new business innovation, it will contribute to new theories and approaches to agile innovation in the media industry, where platform, content and experience interact to produce new media experiences.
PEOPLE: The project may enable new career roles in the creative industries, for hybrid print-and-digital content creators and entrepreneurial thinking. It will also increase the complexity of publishing systems, and could lead to new forms of paper production, requiring new training in media publishing and paper manufacturing.
ECONOMIC: The findings of the project will be of immediate benefit to the tourist industry, which was worth £121 billion to the UK economy in 2014 (UK Tourism statistics 2016). However, the longer-term impact will be on the global print and e-book publishing industry, through a research and development contribution to its current debate about the future of paper in a digital age. Hence the project will accelerate the use of cross-media content, initially in the travel domain, by delivering: (a) technological solutions to bridge the content gap between the material and virtual worlds, (b) solutions to bring together user-generated content and industry generated content, (c) easy software solutions to create cross-media content involving interactive paper. We anticipate a two stage rollout of project technology in the digital economy, with 2GP paper based on paper-and-hand image recognition, and 3GP paper based on flexible printed electronics with wireless communications. At both stages the solution will leverage the simplicity, ubiquity and accessibility of paper with a user model that seamlessly activates adjunct content from simple reading gestures, and a cost point which is affordable to all.
SOCIETAL: Societal: The project will amplify societal trend towards transmedia reading across devices, and enable both professionals and prosumers to create connections across physical and virtual media, easily and cheaply. This will lead to a twofold impact on ordinary people as both consumers and producers of interactive paper, with benefits in the form of new reading and writing behaviours. The capability to augment and "virtually" update printed documents reduces the requirement for documents to be reprinted, thus reducing global paper consumption.