Face Recognition using Photometric Stereo

Lead Research Organisation: University of the West of England
Department Name: Faculty of Environment and Technology


We propose to construct a system for 3D face recognition. We propose to use photometric stereo for face reconstruction in order to by pass the problems of conventional stereo (that needs to solve the matching problem first), structured light (that does not supply colour information) and photometric stereo with spectrally distinct light sources (that relies on the assumption of uniformly coloured imaged objects). Photometric stereo (PS) can reproduce structural details and colour on a per pixel basis in a way that no other 3D system can. The proposed scheme will be appropriate for use in a controlled environment for authentication purposes, but also in a general environment e.g. the entrance of a public event. We shall use two routes: surface reconstruction from the data and direct extraction of facial characteristics from the PS set. In the first approach, once surface normal and albedo is recovered, images of the face may be synthetically rendered under arbitrary new pose and illumination conditions to allow novel viewing conditions. We also aim to use a new multi-scale facial feature matching approach in the recognition process, where facial features range from overall face and head shape to fine skin dermal topography, reflectance and texture. The latter may be thought of as a form of detailed surface bump map forming a unique skin-print or signature and represents a new approach. Hence both the 3D shape and 2D intensity data will be used in recognition or authentication tasks. We propose to use scalable methods for matching, so we can cope with large databases. 3D matching will be done with the newly proposed invaders algorithm which is FFT cross-correlation based, and more detailed matching will be done by using features and classifier combination. The novelty of our approach lies in the use of PS to extract 3D information, the use of detailed facial characteristics like moles, scratches, and skin texture, and in the design of the system so that it can operate while the person is moving, with minimum intrusion and maximum efficiency. We have two industrial collaborators who will contribute in system design, data gathering and exploitation and support from the Home Office. We shall evaluate our system following three possible scenaria: a face searched in the crowd (real time face recognition), a person has to be identified (off-line face recognition) and a person has to be checked against a claimed identity (face authentication). We shall install the first prototype system in the offices of one of our industrial partners in month 12, so that data can be collected. We envisage a door like structure with lights flashing in succession as a person walks through, while a camera is capturing images. We propose to investigate the optimal number of lights in terms of efficiency and accuracy of the reconstruction, and the option of using non-visible light to avoid problems with people sensitive to flashes. We shall also investigate the relationship between detail that has to be captured and the geometry of the construction.
Description A demonstrator 3D face recognition system was developed. The system uses a photometric stereo technique.
Exploitation Route We are currently discussing commercial application with Aralia Systems Ltd
Sectors Digital/Communication/Information Technologies (including Software),Security and Diplomacy,Transport

URL http://www.youtube.com/watch?v=L5NPGbJnte4&feature=youtu.be
Description Future Railway
Amount £680,917 (GBP)
Organisation Rail Safety and Standards Board 
Sector Public
Country United Kingdom
Start 05/2016 
End 04/2017
Description Future Ticket Detection (Gateless Gate-lines)
Amount £145,000 (GBP)
Organisation Rail Safety and Standards Board 
Sector Public
Country United Kingdom
Start 04/2016 
End 03/2017
Description Intelligent Advertising Proof of Concept
Amount £56,000 (GBP)
Funding ID 710243 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 11/2012 
End 05/2014
Description Knowledge Transfer Partnership
Amount £168,205 (GBP)
Funding ID KTP010680 
Organisation Innova UK 
Sector Private
Country United Kingdom
Start 06/2017 
End 05/2019
Description Knowledge Transfer Partnership
Amount £215,081 (GBP)
Funding ID ktp008278 
Organisation Knowledge Transfer Partnerships 
Sector Charity/Non Profit
Country United Kingdom
Start 01/2010 
End 01/2013
Description Seeing More
Amount £150,000 (GBP)
Funding ID 131749 
Organisation TSB Bank plc 
Sector Private
Country United Kingdom
Start 10/2014 
End 09/2015
Description Shape and outline amplification using 2.5D and 3D imaging.
Amount £30,000 (GBP)
Funding ID CDE29353 
Organisation Ministry of Defence (MOD) 
Sector Public
Country United Kingdom
Start 01/2013 
End 04/2013
Description Sustainable Agriculture Innovation Call May 2016
Amount £199,999 (GBP)
Funding ID NE/P007945/1 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 10/2016 
End 09/2018
Title Contactless 3D Hand Recognition from Photometric Stereo 
Description A general purpose portable, high speed, high resolution photometric stereo (PS) device. The device is intended as a general purpose experimental tool. The initial application will be to investigate contactless extraction of biometrics from hands such as palm prints, fingerprints, and vascular structures. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact Nothing yet. 
Title Very high resolution moving 3D face data capture system 
Description A second system, evolved from the Photoface system, was developed as a research tool in high spatial resolution moving 3D face imaging and analysis. The availability of this technology formed the bases of a pilot study collaboration with Prof Marcus Munafò, Professor of Biological Psychology in the School of Experimental Psychology at Bristol University into the dynamics of changing facial expressions in depressed subjects. The system was loaned to Prof Munafò's team as part of a collaborative pilot study. The system also supported an internally funded PhD bursary (Laurence Broadbent) in automated expression recognition using moving facial dynamics. 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2012 
Provided To Others? Yes  
Impact Successful PhD project 
URL http://www.newscientist.com/blogs/nstv/2011/03/4d-system-recreates-your-face.html
Title Photoface Database 
Description This unique 3D face database is amongst the largest currently available, containing 3187 sessions of 453 subjects, captured in two recording periods of approximately six months each. Our Photoface device was located in an unsupervised corridor allowing real-world and unconstrained capture. Each session comprises four differently lit colour photographs of the subject, from which surface normal and albedo estimations can be calculated (photometric stereo MATLAB code implementation included). This allows for many testing scenarios and data fusion modalities. Eleven facial landmarks have been manually located on each session for alignment purposes. Additionally, the Photoface Query Tool is supplied (implemented in MATLAB), which allows for subsets of the database to be extracted according to selected metadata e.g. gender, facial hair, pose, expression. The Photoface database is available to download for research purposes 
Type Of Material Database/Collection of data 
Year Produced 2011 
Provided To Others? Yes  
Impact The Photoface database has been accessed worldwide by 12 institutes (as of Sept 2013) We've had 44 downloads from 19 countries for face database. (Feb 2016) 
URL http://www1.uwe.ac.uk/et/mvl/projects/facerecognition.aspx
Description General Dynamics UK Ltd (Wales) 
Organisation General Dynamics
Country United Kingdom 
Sector Private 
Start Year 2007
Description Home Office Science 
Organisation Home Office
Department Home Office Scientific Development Branch
Country United Kingdom 
Sector Public 
Start Year 2007
Description Identity Solutions Ltd 
Organisation Identity Solutions Ltd
Country New Zealand 
Sector Private 
Start Year 2007
Description Several broadcast items on Photoface and related follow-on work 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact A program item was produced with Discovery Channel for their flagship program Daily Planet and for a BBC World Service program called ''4tech''. A related item also appeared in New Scientist (http://bit.ly/hhteeF), 2011 and on BBC's 'Bang goes the Theory' and a Channel 4 documentary entitled 'End of the World Night' in 2015. Also interviewed for BBC Radio Bristol, BBC Radio 2's Symon Mayo Show, Heart Radio, Three Counties Radio, BBC World Service about Photoface project. Items also appeared on BBC Points West, ITV West Tonight News, and BBC's The Bubble, March 2010. The discovery Item led to contact with Aralia Systems Ltd and several successful follow-on projects in closely related areas.
Year(s) Of Engagement Activity 2011,2012,2013,2014,2015,2016
URL http://www.youtube.com/watch?v=L5NPGbJnte4&feature=youtu.be