Interaction with Projector Phones

We currently observe the trend towards mobile and wearable personal projectors, in particular, projector phones, which allow the user to project any information, anytime and anywhere on any surface. The introduction of pico projectors into mobile phones will completely change the way we interact with and use these devices. This technology allows the user to project any information, anytime and anywhere on any surface. Although we see initial products appearing in the market, very little research has been conducted towards the interaction design and developer support for those devices. The overall aim of this project is to address the challenges and opportunities provided through the introduction of pico projectors and to lay the foundations for interaction design for projector phones. This will be achieved in terms of 1) the creation of the fundamentals through a conceptual framework, design rules and a user interface toolkit, and 2) through a new understanding and new solutions for three principle problems or opportunities: attention shifts, direct or indirect interaction, and multi-user interaction. Our exploration of the research idea mentioned above and overall aims is guided by the following concrete objectives:1) To establish a conceptual framework for interaction with personal projectors, in particular projector phones, that analyses problems, shows the potential and attributes and shows the interplay of personal projection, mobile in- and output-devices, users, interaction concepts and applications. The expected results consist of a classification of usable in- and output configurations, the definition of overall interaction concepts, a description of collaborative interaction models, an analysis of technology requirements and methods for measuring the usability. 2) To develop design rules that will inform interaction designers and developers about the optimal interaction design of applications for projector phones. These rules will provide guidance regarding which interaction concept to use in which context, how to use the in- and output capabilities at hand and how to design the interaction flow. The design rules will be described in the form of heuristics, design patterns and style guides which can be easily accessed, understood and applied by others.3) To design new interaction techniques for single and multi-user applications that offer optimal usage of the projection for specific tasks and which could be used by designers and developers for the implementation of user interface widgets or applications. These interaction techniques will be described in a textual, visual and formalized way, and will show the interplay of in- and output in order to achieve a certain task in an optimal way. These new interaction techniques will focus on addressing the three main problems though, firstly, reducing the number of attention shifts between input- and output (e.g. mobile phone and projection), secondly, studying several direct and indirect interaction techniques (e.g. using the mobile phone's touch screen as a touchpad for controlling a cursor on the projection, using finger gestures to control projected content or turning the projection into a touch screen), and thirdly, through the provision of collaborative interaction techniques (e.g. partially or fully overlapping projections of two persons). 4) To adapt a user interface toolkit in order to provide optimal support for the development of applications for projector phones. This toolkit will provide support for different interaction concepts, will address the fact that the overall user interface is distributed and could span more than one screen (e.g. mobile phone screen and projection), and will provide functionalities for supporting multiple users who all have projector phones.

Direct beneficiaries of the research in the commercial sector are mobile phone manufacturers, mobile network providers, software companies developing mobile applications, and media companies. A company from each of the four mentioned business sectors will be involved in the research via regular meetings and workshops, namely Nokia (donated 7000 Euro), NTT DOCMO Euro-Labs (support letter), M-Ventions (support letter) and BBC R&D (support letter) in order to discuss and potentially exploit the results of the project in a collaborative way. The conceptual framework (WP1) would help mobile phone manufacturers and manufacturers of pico projectors to see the range of interaction concepts when using personal projectors and shows the potential for new applications. The developed preview functions for personal projectors (WP2) will help to increase the usability of projector phones leading to increased sale numbers of such devices. The development and evaluation of indirect and direct techniques for interaction with projections (WP3) also increases the acceptance of such devices and their widespread usage as concrete guidance regarding the advantages and disadvantages of each interaction concept is provided. The introduction of intuitive multi-user applications (WP4) will open up the possibility for completely new applications and services. An extended version of the user interface toolkit (WP5) and the design rules (WP6) can be used by the application developers and designers of the mobile phone manufacturers to adapt standard applications (media player, calendar, web browser, etc.). The mobile network providers, whose business interests often overlap with those of the mobile phone manufacturer, are certainly more interested in how they could adapt their existing services, web pages and applications so that they support projections and would benefit especially from the results of WP2, WP4 and WP6. Companies who specialize on the development of mobile services, web pages and applications would especially benefit from the results of WP2, WP5 and WP6. Media companies see these personal projectors as a completely new distribution channel through which everybody can show everywhere to anyone media content such as pictures, videos, maps, web pages and other applications. Often, special players (e.g. BBC iPlayer) or browsers (news) are used to display information that also has to be adapted so they can benefit from the available projections. When performing those adaptations, the results from WP1, WP5 and WP6 will be especially useful. The most likely possibility will be the protection of the knowledge gained through patents on the interaction techniques (WP2-4) and user interface widgets (WP5) which could be bought by, licensed to or collaboratively exploited with the previously mentioned partners. NTT DOCMO Euro-Labs has already expressed a concrete interest in the project results and plans to fund a complementary studentship. Furthermore, it is envisioned to write a proposal for an European project with some of the previously mentioned partners to continue to proposed research, bring it closer to market and communicate the research results widely. The project results will be communicated by a Technology Day that is organized in collaboration with the Knowledge Business Center being hosted in Infolab21 in Lancaster. Here, local companies will be invited and will see an overview of the state of the art, a presentation of the research results, a demonstration session, a panel and a discussion on the exploitation of the shown research. The wider public will benefit from user-friendly projector phones and from mobile network providers, media companies and software distributors offering services and applications that support projector phones in an effective way. Everybody using such technology will be able to project any information and intuitively interact with it everywhere for personal use or collaborative scenarios.