MORPHED: Modelling Interaction and Perceptions of Modality Coupling in Self-Actuated Shape-Changing Displays

Self-actuated shape-changing displays---visual output surfaces that support physical re-configuration---provide the user with an additional (physical) information channel, opening a wide range novel interaction capabilities and applications. They allow developers to enhance familiar applications such as terrain modeling and photography by rendering 3D scenes in a physical-3D manner and open opportunities for novel applications in collaboration, tangible entertainment, and accessibility.

While there is strong progress in the development of technological approaches to shape-change, researchers lack the fundamental understanding of low-level user interactions with these devices. Specifically, the performance impact of shape-change on atomic touch interactions (object selection and manipulation) is unknown. Similarly, the perceptual impact of coupling high-resolution, high-refresh rate visual output and low-resolution, low refresh-rate physical output on the same surface is also unexplored. Understanding these fundamental interactions and perceptions will lay the foundation for development of efficient and fluid interactions for this next generation of displays.

The development of self-actuated shape-changing displays has the potential to revolutionise the display technology market and create a diverse array of applications and research opportunities. Deformable display interaction is a hot topic in industry (for example, Nokia's Kinetic device, http://research.nokia.com/news/12110, has recently gained widespread media attention); our work will lay the foundations for the next generation of such displays, where deformation is provided through self-actuation.

There are two main non-academic beneficiaries from this work: industry and end-users. Shape-changing displays have the potential to become as ubiquitous as the touch-screen, a market that was worth $USD4.58billion in 2010 and expected to grow to $USD9.65billion in 2014 (http://goo.gl/ZZnmA). For shape-changing interfaces to gain the same success, display and device manufacturers will require a foundational understanding of user interactions, guidelines, recommendations, and the know-how to develop such hardware for commercial sale. Software developers will require design recommendations for interaction techniques and visual and physical coupling. These are all expected outputs of this work.

Shape-changing displays are an enabling technology that open a diverse range of opportunities for end-users by increasing information accessibility, expanding the bandwidth of communication, exploring opportunities for tangible gaming, and encouraging spontaneous collaboration. Shape-changing displays will allow partially-sighted users to physically feel environments and/or travel routes without the cognitive overload of having to translate crude haptic vibrations into a mental picture of the environment. Sighted users will also benefit from direct representation of terrain ("I'll have the kids in the pram, so lets not climb that hill") and groups of friends or colleagues will more easily be able share information and photos around a single device. Larger table-based devices will allow end users to dynamically interact with large-scale maps and GIS data, and will support creative uses such as vertical mounting for novel public displays and/or advertising spaces. Shape-changing displays will redefine our interaction with computing devices.