Learning from the expert: Can observing the oculomotor behaviour of expert face processors improve training of face matching?

Lead Research Organisation: University of Portsmouth
Department Name: Psychology


Face recognition and face matching are crucial to forensic investigations and security, in jobs such as checking passports at border control, searching CCTV footage for suspects of a crime, and in trying to identify people who have been missing either a short time or for many years. It is important to identify methods to improve face recognition and face matching.
Face recognition of unfamiliar people is surprisingly difficult in the best of circumstances. Factors that can make it more difficult include changes in age (as when the observer is trying to identify someone from an old photograph), changes in variable characteristics (as when trying to identify someone in disguise), and differences in lighting and viewpoint from when the face was learned.
One way to develop new methods is to find out why some people are better at face matching than others. As such, the first part of the proposed project tests a cross-section of the public until we identify people particularly good at face matching. We then observe how they compare faces, focusing on the eye movements they make, and ask them about the strategies they use.
The knowledge gained will then be tested to see if explicit training about strategies or implicit training through demonstrating expert's behaviour while they do the task will be effective training. Previous research in other perceptual tasks has suggested that showing the eye movements of those skilled at the task to novices as part of training will improve the rate of training. This is done by showing the trainee the image being evaluated superimposed by a dynamic representation of th eye movements a skilled person made when inspecting the image. The trainee is simply told to follow the gaze of the skilled person during this presentation. The effectiveness of this simple technique may be because people innately attend to other people's gaze to understand what in the environment those other people consider important. This following of another's gaze toward objects, called "joint attention", is a powerful learning tool in childhood. Those who do not develop joint attention, such as autistic children, also appear to be delayed in developing understanding of the goals and intentions of others. Adults, too, are involuntarily drawn to follow gaze and implicitly learn to value higher objects that others gaze at over objects the others do not gaze at.

As this "oculomotor demonstration" training method is new, it has not been fully tested. For instance, the effectiveness of training has only been tested in the short term. Therefore, this research will test whether any benefits last for a week or a month. Further, testing it on face matching provides a strong test of whether the training generalises, as it is the first task to which this training is applied for which both the stimulus and the task is familiar to the trainee. Although the average person is not particularly good at face matching, everybody has experience of it.

The outcome of this study will be reported in an international journal read by both researchers and practitioners, will be shared with people responsible for training face comparison in forensic contexts, and will inform our further research that will more directly test why this method is effective for some perceptual tasks.

Planned Impact

Face matching errors can have devastating effects in criminal justice. For instance, a surveillance officer's mis-comparison of a photo of a terrorist suspect to the CCTV image of Jean Charles de Menenez is thought to be one of the reasons for the tragic shooting of him by police in 2005. Further, it is important to block known terrorists from entering the country, and one of the steps to doing that is to ensure that border security personnel can recognise them from their photos. Likewise, at a local level, the effectiveness of drinking banning orders is critically dependent on distribution of photos of the banned person and accurate identification of the person from those photos by security personnel at bars and off-licences.

This work will also impact criminal prosecutions. Identification evidence often represents a substantial amount of the evidence supporting a prosecution (Brewer, et al., 2005), and influences decision making throughout the forensic setting. Consequently, reliable identification evidence is essential for effective criminal investigations and the successful prosecution of criminals. Thus, our proposed research can improve the efficiency of criminal investigations and will therefore be of interest to forensic investigators attempting to maximise the reliability of identification evidence.

If the training method proves successful, this work will have an impact on society by providing the forensic community a new training method that would improve face matching. Improving forensic investigations and the effectiveness of border control will benefit society by reducing crime and terrorism, both of which have cultural and economic impacts. The forensic community includes security firms, police, UKBA, and the Home Office. The public should be safer, which will benefit quality of life.

This research would also benefit other training providers in that it tests a way of transferring the skills of an expert to those of a trainee without one on one contact. Historically, apprenticeships have been used to transfer practical skill from an expert to a trainee. As business becomes more distributed, even globalised, it is more difficult for experts to be geographically present with trainees, and so alternatives to apprenticeships are needed. If showing the eye movements of an expert provides some of the benefit of having the expert present, this could be an invaluable addition to training methods used by businesses.

If a beneficiary decided to implement the training, they would only need to identify the face-matching tasks that they use, identify one or more people particularly good at the task, use an eye-tracker to record eye movements of the expert doing the task, and develop a short training course where the trainee watches the video and practices the task. It is easy to envisage that these steps could be done in less than 6 month, which means that the timescale of impact is relatively short.

We will be advertising the project to the wider community. As the topic is engaging, this should also benefit society by giving the general public an opportunity to become involved in research.

The research assistant will gain invaluable research experience and, in particular, learn how to use an eye-tracker. That particular tool would be of benefit for a research career either in academia or in other domains (e.g., marketing).


10 25 50
Description FINDING 1: The core scientific impact was that, contrary to previous work in related fields, modelling good inspection behaviour (i.e., showing the eye movements of skilled matchers) is not necessarily a better training method than those previously used. Inspection modelling is no better or worse than other training methods. This is of interest because (a) we believe this is the first test of the method that tried to improve performance on a task that was already practiced by observers, (b) this is one of the first tests of the method that trained with one set of stimuli and tested with another set of stimuli (and hence our evaluation of the training method was more rigorous than most past evaluations), and (c) there is to date no particularly effective method to train face matching whereas there are many jobs in which it would be helpful to have an effective training method.

FINDING 2: We found some evidence that for a set of face-pairs that was relatively easy to judge, inspection modelling improved performance better than the other training methods, and that the improvements generalised to other relatively easy matching trials and endured for at least three weeks. This finding needs to be replicated because of weaknesses in the evidence.

FINDING 3: We have developed a test that is unusual and useful in that faces differ considerably in terms of superficial characteristics and documented age.

FINDING 4: We learned that accuracy in matching faces does not vary consistently with the difference in age between the faces. Our stimulus set was small enough that this will require replication, but there was no hint of an effect of age difference on matching accuracy.

FINDING 5: We replicated some previous findings about differences between experts and novices in the way they do their work (experts thought longer before starting to inspect) and in their confidence about their behaviour (experts were more aware of whether they were accurate).
Exploitation Route The principle way to take our results forward is to continue pursuing ways to train face-matching. The method we were keen to test - modelling of inspection - needs to be tested more rigorously by lengthening the amount of training and by giving feedback to people about their accuracy. The most promising result (successful training of relatively easy comparisons) needs replication. If it proves reliable, we need to understand why the training method worked for easier comparisons but not for more difficult comparisons.

If modelling of inspection is unsuccessful for face-matching, after this extension of research, it is worth pursuing why the method works for other tasks but not for this one.

Outside of academia, there are many sectors that need to check people's identities by comparing two photographic identifications or comparing one photographic identification with a live person. These sectors are identified in the next question. Once a successful training method is identified, it should be conveyed to all of these sectors.
Sectors Communities and Social Services/Policy,Education,Financial Services, and Management Consultancy,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Retail,Security and Diplomacy

Description The project consisted of the development of a face-matching skill test, efforts to find differences between skilled and unskilled face-matchers, and an evaluation of three methods for training face matching. One of the training methods was specifically targeted, as it was a new method that involved having trainees observe the way a skilled matcher inspected faces. The trainees were shown where the matcher fixated on the images of the face-pairs in the training stimuli. It was hoped that this observation would be a form of implicit training. By the end of the funding period, the project found no strong evidence that the new method was better than the other methods. There was, however, some promising weak evidence that in some limited cases it would be better. For full understanding of these cases, our research is continuing even today. There has been no societal or economic impact of the project, because the training method was not particularly effective.
First Year Of Impact 2013
Description Invitation to small Home Office workshop about identity checks in airports
Geographic Reach National 
Policy Influence Type Participation in advisory committee
Description Commissioned Review about individual differences in the ability to spot rare, non-salient or hidden targets
Amount £50,497 (GBP)
Organisation Centre for Research and Evidence on Security Threats 
Sector Academic/University
Country United Kingdom
Start 05/2016 
End 10/2016
Title Face Matching Skill Test 
Description We have a set of pairs of faces that made up our face-matching skill tests and training materials, along with the performance of our population of participants on the tests. The faces themselves are not our property, so we do not share them directly with interested parties. However, we are happy to share the list of which faces were used and where to source them. This will be added to UK Data Service once our results are published. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact None 
Description Multi-centre study 
Organisation University of Adelaide
Country Australia 
Sector Academic/University 
PI Contribution A member of our team is working with partners at University of Adelaide to further investigate face matching. The entire group is collecting and analysing data.
Collaborator Contribution The entire group is collecting and analysing data.
Impact 3 x conference presentations: 1. Stephens, R., Semmler, C., & Sauer, J.D. (2013, April). Confidence-accuracy calibration for positive and negative face matching decisions. 40th Australasian Experimental Psychology Conference, Adelaide, Australia; 2. Stephens, R., Semmler, C., & Sauer, J.D. (2013, February). Confidence-accuracy calibration for positive and negative face matching decisions. Australian Mathematical Psychology Conference, Sydney, Australia. 3. Semmler, C., Stephens, R., & Sauer, J.D. (2015, June). The confidence-accuracy relationship in identity verification settings: Base rates, task orientation and the positive-negative asymmetry. 11th Biennial Conference of the Society for Applied Research in Memory and Cognition, Victoria, Canada). Also, about to submit a manuscript.
Start Year 2013