Exploration of Ultrasound based haptic interaction on a multi-touch surface

Multi-touch tables, such as Microsoft Surface, are now widely available. Users can walk-up and use these systems in hotel lobbies and other public settings with very little instruction and with no need to wear or hold intrusive sensors with their hands or body. The ability to 'walk-up and use' with unadorned hands and fingers removes the barrier between human and technology, encouraging spontaneous use.

One of the primary disadvantages of current interactive surfaces is that users can touch objects, but they are unable to feel them. There are a plethora of applications where it is beneficial for the user to have their touches augmented with 'feel-based' haptic feedback. For example, medical applications, virtual training, and modelling applications require precise control from the user-haptic feedback can aid the user in effectively performing these tasks. These applications demonstrate the benefit of augmenting haptic feedback with visual feedback in an interactive application. Often, the visual space has been disconnected from the force-feedback, requiring a prolonged training period for the user to become accustomed to moving a digital object and watching the interactions a small distance away on a monitor.

In this proposal we will investigate the use of ultrasound transducers to provide 'feelable' feedback through air. The skin on a human hand can feel the ultrasonic pressure wave produced by a carefully calibrated series of transducers, in much the same manner that is apparent from loud sub-woofers on a stereo system. Ultrasound waves are outside the human's range of hearing and so provide silent, through-air haptic feedback. We will use this technology to provide multi-point haptic feedback on the surface of a multi-touch horizontal surface.

The team consists of Dr. Sriram Subramanian, Dr. Mark Marshall and Dr. Jason Alexander from the computer science departments and Prof. Bruce Drinkwater from the Mechanical Engineering department of the University of Bristol and Prof. Stephen Brewster from the Computer Science department of the University of Glasgow. The team is internationally recognised for its research in Human-Computer Interaction (HCI), novel integration of hardware for HCI, and Ultrasonic sensors. Microsoft research (Cambridge) and XMOS will serve as project partners and offer valuable resources and support for the project.

Development of ultrasonic haptic feedback has the potential to not only revolutionise several current areas of research, but to also create opportunities for immersive new experiences with large commercial value. Haptics is currently a hot topic in industry (Next Generation Haptics: Market Analysis and Forecasts. ARC Chart. Feb 2011.www.arcchart.com) and our work will develop a new form of haptic technology and demonstrate its usefulness in a new range of areas. This will increase the potential opportunities for haptics beyond what is available now.

Our demonstrator systems (as listed in WP5) will allow Dynamic Holographic Assembler (DHA) operators to feel the forces required to move molecules on their multi-touch interfaces to these systems (currently force data cannot be used in any way as it is difficult to let users feel forces at multiple locations simultaneously). This will provide added value to current research in this area funded through EPSRC (EP/G037310/1 and EP/G012067/1). In turn, this will allow non-specialist researchers to also use a DHA with minimal training.

Once we have developed the technology and interaction techniques that support multi-point haptic feedback, there is a large number of potential domains for applying these techniques. Examples of these applications include mobile TV broadcasting and medical applications. Mobile TV broadcasts could be augmented with a haptic feedback channel to let users 'feel' the action. For example, multi-point ultrasonic haptic feedback could be applied to the back of the mobile device, with the user's hand sensing the forces exerted by the athletes. The gloves of amateur boxers already include sensors for registering the force of their punches-this data could be transmitted with the video stream to provide a more immersive user experience. A demonstrator for this application will be developed by a project technician to showcase this idea.
There are also several medical applications for this technology. Stem-cell researchers can also use multi-point haptic feedback as a method for interpreting, through mechanical property changes, structural changes in the cell even before chemical or other visible identification is possible.