A 3d psychological face database and tools

The primary aim of this work is to deliver the images and software tools to enable future research on face perception and recognition. Faces are inherently 3D, and our own data suggests, for example, that a 3D presentation improves subsequent recognition from video. However, the technology is still quite challenging; simply displaying a 3D image is surprisingly fraught and doing so in the controlled way needed for reliable data collection requires programming. We propose to make available images and software to make such studies more straightforward, allowing research to concentrate on the underlying science.

Enabling interesting face research requires much more than a single 3D image of a number of people. We propose to collect an exceptionally diverse set of images and videos of each person in our database, taken with different cameras and under varying conditions of viewpoint and lighting, including stereo pair images and video and both natural and posed expressions. While we cannot hope to cover every requirement, our intention is to provide the richest database currently available, including rating data on the faces, such as their distinctiveness.

Crucially, we shall also make available software tools for modifying and presenting these images. Much recent research has made use of our ability to 'morph' photographs: blending them together to make average images; changing their apparent masculinity or race; caricaturing them. Much of this has been made possible through PsychoMorph, written by Bernard Tiddeman, CI on this proposal. This allows batch processing of images, which greatly simplifies the job of producing a consistent set for an experiment. Dr Tiddeman has developed an equivalent program for handling 3D images (MorphAnalyser) and a major aim of this proposal is to take this to the point where it is reliable, easy to use and well documented. PsychoMorph has recently been rewritten to make it platform-independent, to allow easy writing and integration of "plug-in" extensions and easier batch processing. MorphAnalyser is already platform independent, this grant will also extend the plug-in/batch processing architecture developed for PsychoMorph to the more complex 3D domain. Documentation will be provided via an online wiki, to permit easy updating.

Presentation of 3D images is not possible with the standard software typically used for conducting psychological experiments. The project partners have independently developed complementary approaches to the problem: Aberystwyth have developed presentation software that will allow non-programmers to run a limited set of standard designs, while Stirling have Matlab routines that allow programmers to develop their own paradigms. As part of this project, we shall refine and document these packages and make them available for use with the image set. This will allow mainstream face researchers to use 3D images for the first time.

The image database will be hosted by the Psychological Image Collection at Stirling (PICS) website, while the software will be hosted at Aberystwyth, with links from each to the other. PICS already contains a variety of image sets, plus links to other databases and is well-used by the research community world-wide. Both will be free for academic use. The dataset will also be valuable for those attempting automated face recognition, providing a diverse set of training images and videos, and a designated test set to allow comparison with other systems.

Our intention is to continue to update both the database and the software as resources allow. For example, we hope to add a Chinese image set. One route to doing so will be to make parts of the database commercially available, with the income paying for additional photography and coding. Opportunities for improving the morphing software will be afforded by student projects, which have already contributed to some aspects of the current version.

This project is essentially all about end impact, the only proposed academic output being a paper describing the image set and software for Behavior Research Methods, which itself will be a route to increasing the visibility of the collection. The aim is to facilitate advanced research, primarily in psychology, but also in artificial face recognition, as described in the section on academic beneficiaries. The consequences of that work have the potential for much wider impact.

Understanding how humans perform face recognition may be key to building effective artificial systems. The best such can already out-perform humans given limited training on a new face, but human performance on really familiar faces is still far better. Understanding exactly how we do this will be of interest in its own right, since it is probably prototypical of the way we perform recognition of other things, but should inform those building computer systems. Truly effective automated face recognition could have enormous impact on society, with consequences that would be open for debate. While it would be much easier to track down those wanted for crime, for example, it would have potentially interesting consequences for our privacy. There are many potential applications; for example, if a cash machine did not recognise the person using a card it could trigger an alert, or ask additional security questions before continuing. Someone with Alzheimer's could be prompted about the identity of those they meet; indeed, such a device, if unobtrusive, could help most of us!

While the UK has a very strong face research community, both within Psychology and Computing Science, we expect this collection to be used world-wide, as both PsychoMorph and the PICS image collection already are.

Face research is inherently interesting to the general public; we all do face perception every time we meet someone. 3D imagery and stereo still has a 'wow' factor, providing a great opportunity to grab attention, for example at open days or science centre visits. The software will allow us to grab a 3D image with our camera, then present it in stereo on our 3D monitor, possibly transforming their apparent age or gender. This will provide a framework for explaining the serious research that we undertake. For those without their own camera, it will still be possible to show images changing in eye-catching ways.

Possible industrial beneficiaries from the improved software include those involved in perceptual evaluation of cosmetics and similar products. For example, Tiddeman is currently a joint principal investigator on a Unilever funded project relating to skin health and cosmetics, where improvements in such tools would prove extremely useful.

The Aberystwyth RA will benefit from learning about programming cutting-edge computer graphics; a highly marketable skill set.

The Stirling RA will learn the techniques of gathering and preparing complex and consistent image sets which would be an excellent base for pursuing further research in this area.