A 3d psychological face database and tools

Lead Research Organisation: University of Stirling
Department Name: Psychology

Abstract

The primary aim of this work is to deliver the images and software tools to enable future research on face perception and recognition. Faces are inherently 3D, and our own data suggests, for example, that a 3D presentation improves subsequent recognition from video. However, the technology is still quite challenging; simply displaying a 3D image is surprisingly fraught and doing so in the controlled way needed for reliable data collection requires programming. We propose to make available images and software to make such studies more straightforward, allowing research to concentrate on the underlying science.

Enabling interesting face research requires much more than a single 3D image of a number of people. We propose to collect an exceptionally diverse set of images and videos of each person in our database, taken with different cameras and under varying conditions of viewpoint and lighting, including stereo pair images and video and both natural and posed expressions. While we cannot hope to cover every requirement, our intention is to provide the richest database currently available, including rating data on the faces, such as their distinctiveness.

Crucially, we shall also make available software tools for modifying and presenting these images. Much recent research has made use of our ability to 'morph' photographs: blending them together to make average images; changing their apparent masculinity or race; caricaturing them. Much of this has been made possible through PsychoMorph, written by Bernard Tiddeman, CI on this proposal. This allows batch processing of images, which greatly simplifies the job of producing a consistent set for an experiment. Dr Tiddeman has developed an equivalent program for handling 3D images (MorphAnalyser) and a major aim of this proposal is to take this to the point where it is reliable, easy to use and well documented. PsychoMorph has recently been rewritten to make it platform-independent, to allow easy writing and integration of "plug-in" extensions and easier batch processing. MorphAnalyser is already platform independent, this grant will also extend the plug-in/batch processing architecture developed for PsychoMorph to the more complex 3D domain. Documentation will be provided via an online wiki, to permit easy updating.

Presentation of 3D images is not possible with the standard software typically used for conducting psychological experiments. The project partners have independently developed complementary approaches to the problem: Aberystwyth have developed presentation software that will allow non-programmers to run a limited set of standard designs, while Stirling have Matlab routines that allow programmers to develop their own paradigms. As part of this project, we shall refine and document these packages and make them available for use with the image set. This will allow mainstream face researchers to use 3D images for the first time.

The image database will be hosted by the Psychological Image Collection at Stirling (PICS) website, while the software will be hosted at Aberystwyth, with links from each to the other. PICS already contains a variety of image sets, plus links to other databases and is well-used by the research community world-wide. Both will be free for academic use. The dataset will also be valuable for those attempting automated face recognition, providing a diverse set of training images and videos, and a designated test set to allow comparison with other systems.

Our intention is to continue to update both the database and the software as resources allow. For example, we hope to add a Chinese image set. One route to doing so will be to make parts of the database commercially available, with the income paying for additional photography and coding. Opportunities for improving the morphing software will be afforded by student projects, which have already contributed to some aspects of the current version.

Planned Impact

This project is essentially all about end impact, the only proposed academic output being a paper describing the image set and software for Behavior Research Methods, which itself will be a route to increasing the visibility of the collection. The aim is to facilitate advanced research, primarily in psychology, but also in artificial face recognition, as described in the section on academic beneficiaries. The consequences of that work have the potential for much wider impact.

Understanding how humans perform face recognition may be key to building effective artificial systems. The best such can already out-perform humans given limited training on a new face, but human performance on really familiar faces is still far better. Understanding exactly how we do this will be of interest in its own right, since it is probably prototypical of the way we perform recognition of other things, but should inform those building computer systems. Truly effective automated face recognition could have enormous impact on society, with consequences that would be open for debate. While it would be much easier to track down those wanted for crime, for example, it would have potentially interesting consequences for our privacy. There are many potential applications; for example, if a cash machine did not recognise the person using a card it could trigger an alert, or ask additional security questions before continuing. Someone with Alzheimer's could be prompted about the identity of those they meet; indeed, such a device, if unobtrusive, could help most of us!

While the UK has a very strong face research community, both within Psychology and Computing Science, we expect this collection to be used world-wide, as both PsychoMorph and the PICS image collection already are.

Face research is inherently interesting to the general public; we all do face perception every time we meet someone. 3D imagery and stereo still has a 'wow' factor, providing a great opportunity to grab attention, for example at open days or science centre visits. The software will allow us to grab a 3D image with our camera, then present it in stereo on our 3D monitor, possibly transforming their apparent age or gender. This will provide a framework for explaining the serious research that we undertake. For those without their own camera, it will still be possible to show images changing in eye-catching ways.

Possible industrial beneficiaries from the improved software include those involved in perceptual evaluation of cosmetics and similar products. For example, Tiddeman is currently a joint principal investigator on a Unilever funded project relating to skin health and cosmetics, where improvements in such tools would prove extremely useful.

The Aberystwyth RA will benefit from learning about programming cutting-edge computer graphics; a highly marketable skill set.

The Stirling RA will learn the techniques of gathering and preparing complex and consistent image sets which would be an excellent base for pursuing further research in this area.

Publications

10 25 50
 
Description We have developed the planned face database; it remains to our knowledge the richest set of imagery of each person available. There have been 275 requests to use the collection as of March 2019.
Exploitation Route It is intended for academic research use only. I get occasional enquiries about commercial use, but the permissions given by the participants, and now GDPR, would not allow that.
Sectors Other

URL http://pics.stir.ac.uk/ESRC/
 
Description EPSRC Programme grant
Amount £6,104,265 (GBP)
Funding ID EP/N007743/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 01/2016 
End 12/2020
 
Title Updated database 
Description This is an update to the previously published face database, to add extra identities and some extra image types. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact I continue to receive 2 or 3 requests per week to use the database. It has now been used as the basis for a competition on Dense 3D Reconstruction of 2D Face Images, https://www.facer2vm.org/fg2018/ 
URL http://pics.stir.ac.uk/ESRC/index.htm
 
Title face database 
Description A collection of face images, 2D, stereo, 3D and video 
Type Of Material Database/Collection of data 
Year Produced 2013 
Provided To Others? Yes  
Impact Too early to know 
URL http://pics.stir.ac.uk/ESRC/
 
Description BA festival of science 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact We collected 3D images of 96 people attending the festival of science, to add to the database. We gave them copies of their images and a display program with which to view them

None
Year(s) Of Engagement Activity 2012