Face Recognition using Photometric Stereo
Lead Research Organisation:
University of the West of England
Department Name: Faculty of Environment and Technology
Abstract
We propose to construct a system for 3D face recognition. We propose to use photometric stereo for face reconstruction in order to by pass the problems of conventional stereo (that needs to solve the matching problem first), structured light (that does not supply colour information) and photometric stereo with spectrally distinct light sources (that relies on the assumption of uniformly coloured imaged objects). Photometric stereo (PS) can reproduce structural details and colour on a per pixel basis in a way that no other 3D system can. The proposed scheme will be appropriate for use in a controlled environment for authentication purposes, but also in a general environment e.g. the entrance of a public event. We shall use two routes: surface reconstruction from the data and direct extraction of facial characteristics from the PS set. In the first approach, once surface normal and albedo is recovered, images of the face may be synthetically rendered under arbitrary new pose and illumination conditions to allow novel viewing conditions. We also aim to use a new multi-scale facial feature matching approach in the recognition process, where facial features range from overall face and head shape to fine skin dermal topography, reflectance and texture. The latter may be thought of as a form of detailed surface bump map forming a unique skin-print or signature and represents a new approach. Hence both the 3D shape and 2D intensity data will be used in recognition or authentication tasks. We propose to use scalable methods for matching, so we can cope with large databases. 3D matching will be done with the newly proposed invaders algorithm which is FFT cross-correlation based, and more detailed matching will be done by using features and classifier combination. The novelty of our approach lies in the use of PS to extract 3D information, the use of detailed facial characteristics like moles, scratches, and skin texture, and in the design of the system so that it can operate while the person is moving, with minimum intrusion and maximum efficiency. We have two industrial collaborators who will contribute in system design, data gathering and exploitation and support from the Home Office. We shall evaluate our system following three possible scenaria: a face searched in the crowd (real time face recognition), a person has to be identified (off-line face recognition) and a person has to be checked against a claimed identity (face authentication). We shall install the first prototype system in the offices of one of our industrial partners in month 12, so that data can be collected. We envisage a door like structure with lights flashing in succession as a person walks through, while a camera is capturing images. We propose to investigate the optimal number of lights in terms of efficiency and accuracy of the reconstruction, and the option of using non-visible light to avoid problems with people sensitive to flashes. We shall also investigate the relationship between detail that has to be captured and the geometry of the construction.
People |
ORCID iD |
Melvyn Smith (Principal Investigator) | |
Lyndon Smith (Co-Investigator) |
Publications
Zhang W
(2016)
Eye center localization and gaze gesture recognition for human-computer interaction.
in Journal of the Optical Society of America. A, Optics, image science, and vision
Zhang W
(2016)
Eye centre localisation: an unsupervised modular approach
in Sensor Review
Zhang W
(2016)
Gender recognition from facial images: two or three dimensions?
in Journal of the Optical Society of America. A, Optics, image science, and vision
Zhang W
(2016)
Gender and gaze gesture recognition for human-computer interaction
in Computer Vision and Image Understanding
Zafeiriou S
(2010)
Baseline face recognition using photometric stereo data
in Procedia Computer Science
Zafeiriou S
(2010)
Nonlinear non-negative component analysis algorithms.
in IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Zafeiriou S
(2011)
2.5D Elastic graph matching
in Computer Vision and Image Understanding
Zafeiriou S
(2011)
The Photoface database
Zafeiriou S
(2013)
Face Recognition and Verification Using Photometric Stereo: The Photoface Database and a Comprehensive Evaluation
in IEEE Transactions on Information Forensics and Security
Zafeiriou S
(2010)
Nonnegative tensor factorization as an alternative Csiszar-Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms
in Data Mining and Knowledge Discovery
Vijayan V
(2011)
Twins 3D face recognition challenge
Toderici G
(2010)
Bidirectional relighting for 3D-aided 2D face recognition
Sun J
(2013)
Sampling Light Field for Photometric Stereo
in International Journal of Computer Theory and Engineering
Sohaib A
(2015)
3D reconstruction of concave surfaces using polarisation imaging
in Journal of Modern Optics
Sohaib A
(2017)
BRDF of human skin in the visible spectrum
in Sensor Review
Smith LN
(2017)
Realistic and interactive high-resolution 4D environments for real-time surgeon and patient interaction.
in The international journal of medical robotics + computer assisted surgery : MRCAS
Saracchini R
(2013)
Robust 3D face capture using example-based photometric stereo
in Computers in Industry
Saracchini R
(2012)
A robust multi-scale integration method to obtain the depth from gradient maps
in Computer Vision and Image Understanding
Saracchini R
(2012)
Advances in Image and Video Technology
Poullis A
(2015)
PTU-024 Photometric stereo reconstruction for surface analysis of mucosal tissue
in Gut
N/a Saracchini
(2010)
Multi-Scale Depth from Slope with Weights
Moorhouse A
(2009)
The nose on your face may not be so plain: using the nose as a biometric
Kautkar S
(2012)
Face recognition in 2D and 2.5D using ridgelets and photometric stereo
in Pattern Recognition
Hansen Mark F.
(2012)
3D face recognition using photometric stereo
Hansen M F
(2016)
Multispectral Contactless 3D Handprint Acquisition for Identification
Hansen M
(2013)
Computer Analysis of Images and Patterns
Hansen M
(2010)
3D face reconstructions from photometric stereo using near infrared and visible light
in Computer Vision and Image Understanding
Emrith K
(2013)
Real-time recovery of moving 3D faces for emerging applications
in Computers in Industry
Emambakhsh M
(2013)
Using nasal curves matching for expression robust 3D nose recognition
Da Silva L
(2023)
Antibacterial potential of chalcones and its derivatives against Staphylococcus aureus.
in 3 Biotech
Atkinson G A
(2011)
Using Photometric Stereo for Face Recognition
in International Journal of Bio-Science and Bio-Technology
Atkinson G
(2010)
A efficient and practical 3D face scanner using near infrared and visible photometric stereo
in Procedia Computer Science
Atkinson G
(2009)
Pattern Recognition and Image Analysis
Argyriou V
(2013)
A sparse representation method for determining the optimal illumination directions in Photometric Stereo
in Signal Processing
Ahmad J
(2014)
An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region
in Pattern Recognition Letters
Description | A demonstrator 3D face recognition system was developed. The system uses a photometric stereo technique. |
Exploitation Route | We are currently discussing commercial application with Aralia Systems Ltd |
Sectors | Digital/Communication/Information Technologies (including Software),Security and Diplomacy,Transport |
URL | http://www.youtube.com/watch?v=L5NPGbJnte4&feature=youtu.be |
Description | Future Railway |
Amount | £680,917 (GBP) |
Organisation | Rail Safety and Standards Board |
Sector | Public |
Country | United Kingdom |
Start | 05/2016 |
End | 04/2017 |
Description | Future Ticket Detection (Gateless Gate-lines) |
Amount | £145,000 (GBP) |
Organisation | Rail Safety and Standards Board |
Sector | Public |
Country | United Kingdom |
Start | 04/2016 |
End | 03/2017 |
Description | Intelligent Advertising Proof of Concept |
Amount | £56,000 (GBP) |
Funding ID | 710243 |
Organisation | Innovate UK |
Sector | Public |
Country | United Kingdom |
Start | 11/2012 |
End | 05/2014 |
Description | Knowledge Transfer Partnership |
Amount | £215,081 (GBP) |
Funding ID | ktp008278 |
Organisation | Knowledge Transfer Partnerships |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 01/2010 |
End | 01/2013 |
Description | Knowledge Transfer Partnership |
Amount | £168,205 (GBP) |
Funding ID | KTP010680 |
Organisation | Innova UK |
Sector | Private |
Country | United Kingdom |
Start | 06/2017 |
End | 05/2019 |
Description | Seeing More |
Amount | £150,000 (GBP) |
Funding ID | 131749 |
Organisation | TSB Bank plc |
Sector | Private |
Country | United Kingdom |
Start | 10/2014 |
End | 09/2015 |
Description | Shape and outline amplification using 2.5D and 3D imaging. |
Amount | £30,000 (GBP) |
Funding ID | CDE29353 |
Organisation | Ministry of Defence (MOD) |
Sector | Public |
Country | United Kingdom |
Start | 01/2013 |
End | 04/2013 |
Description | Sustainable Agriculture Innovation Call May 2016 |
Amount | £199,999 (GBP) |
Funding ID | NE/P007945/1 |
Organisation | Natural Environment Research Council |
Sector | Public |
Country | United Kingdom |
Start | 10/2016 |
End | 09/2018 |
Title | Contactless 3D Hand Recognition from Photometric Stereo |
Description | A general purpose portable, high speed, high resolution photometric stereo (PS) device. The device is intended as a general purpose experimental tool. The initial application will be to investigate contactless extraction of biometrics from hands such as palm prints, fingerprints, and vascular structures. |
Type Of Material | Improvements to research infrastructure |
Provided To Others? | No |
Impact | Nothing yet. |
Title | Very high resolution moving 3D face data capture system |
Description | A second system, evolved from the Photoface system, was developed as a research tool in high spatial resolution moving 3D face imaging and analysis. The availability of this technology formed the bases of a pilot study collaboration with Prof Marcus Munafò, Professor of Biological Psychology in the School of Experimental Psychology at Bristol University into the dynamics of changing facial expressions in depressed subjects. The system was loaned to Prof Munafò's team as part of a collaborative pilot study. The system also supported an internally funded PhD bursary (Laurence Broadbent) in automated expression recognition using moving facial dynamics. |
Type Of Material | Model of mechanisms or symptoms - human |
Year Produced | 2012 |
Provided To Others? | Yes |
Impact | Successful PhD project |
URL | http://www.newscientist.com/blogs/nstv/2011/03/4d-system-recreates-your-face.html |
Title | Photoface Database |
Description | This unique 3D face database is amongst the largest currently available, containing 3187 sessions of 453 subjects, captured in two recording periods of approximately six months each. Our Photoface device was located in an unsupervised corridor allowing real-world and unconstrained capture. Each session comprises four differently lit colour photographs of the subject, from which surface normal and albedo estimations can be calculated (photometric stereo MATLAB code implementation included). This allows for many testing scenarios and data fusion modalities. Eleven facial landmarks have been manually located on each session for alignment purposes. Additionally, the Photoface Query Tool is supplied (implemented in MATLAB), which allows for subsets of the database to be extracted according to selected metadata e.g. gender, facial hair, pose, expression. The Photoface database is available to download for research purposes |
Type Of Material | Database/Collection of data |
Year Produced | 2011 |
Provided To Others? | Yes |
Impact | The Photoface database has been accessed worldwide by 12 institutes (as of Sept 2013) We've had 44 downloads from 19 countries for face database. (Feb 2016) |
URL | http://www1.uwe.ac.uk/et/mvl/projects/facerecognition.aspx |
Description | General Dynamics UK Ltd (Wales) |
Organisation | General Dynamics |
Country | United Kingdom |
Sector | Private |
Start Year | 2007 |
Description | Home Office Science |
Organisation | Home Office |
Department | Home Office Scientific Development Branch |
Country | United Kingdom |
Sector | Public |
Start Year | 2007 |
Description | Identity Solutions Ltd |
Organisation | Identity Solutions Ltd |
Country | New Zealand |
Sector | Private |
Start Year | 2007 |
Description | Several broadcast items on Photoface and related follow-on work |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | A program item was produced with Discovery Channel for their flagship program Daily Planet and for a BBC World Service program called ''4tech''. A related item also appeared in New Scientist (http://bit.ly/hhteeF), 2011 and on BBC's 'Bang goes the Theory' and a Channel 4 documentary entitled 'End of the World Night' in 2015. Also interviewed for BBC Radio Bristol, BBC Radio 2's Symon Mayo Show, Heart Radio, Three Counties Radio, BBC World Service about Photoface project. Items also appeared on BBC Points West, ITV West Tonight News, and BBC's The Bubble, March 2010. The discovery Item led to contact with Aralia Systems Ltd and several successful follow-on projects in closely related areas. |
Year(s) Of Engagement Activity | 2011,2012,2013,2014,2015,2016 |
URL | http://www.youtube.com/watch?v=L5NPGbJnte4&feature=youtu.be |