Skin Reflectance and Face Shape Estimation Using Photometric Stereo

This proposal aims to advance the state-of-the-art in 3D face recognition by means of a novel, non-intrusive and highly efficient skin reflectance capture technology. The techniques developed will, in-turn, enable rapid facial geometry analysis and enhanced recognition rates.Face recognition is currently a rapidly growing area of research within industry and academia. Indeed, 2D face recognition is now at a stage where a few industrial applications are possible. However, these methods, which just use a single 2D image of a face to perform the recognition, are excessively limited by the fact that the face becomes unrecognisable when variations such as pose, illumination, make-up or expression are present. However, the 3D shape of the face does not change at all with many of these variations, and changes only minimally with expression. Consequently, an increasing amount of face recognition research is focussing on ways to use the 3D shape of the face for identification.Here, we are proposing to use a Photometric Stereo (PS) method for 3D shape estimation. The main advantages of the proposed method compared to other 3D face shape capture devices will be (1) cheaper to construct hardware, (2) fast acquisition and processing, (3) largely unaffected by ambient illumination, (4) person-specific reflectance considered, (5) more accurate than standard PS, (6) possibility of using the reflectance properties to aid recognition, and (7) minimal calibration required.A large number of methods for using the 3D facial geometry have been proposed in the scientific literature and very promising results have been attained. However, the question of how to capture a subject's 3D face shape prior to recognition is an open one. Existing approaches use technology that is too expensive and too slow for most applications. This proposal is motivated by the need to address this question.The main contributions of the proposed work will be in two areas: photometric stereo (PS) and reflectance analysis. Photometric stereo is a method of estimating the 3D geometry of an object by imaging it under three or more illumination directions. For this project, we will be using five light sources, and aim to simultaneously acquire both shape and reflectance information. We will be using a high speed light-camera synchronisation device developed here at UWE for this task. This will allow deducing a mapping between the orientations of the recovered surface and the measured pixel intensities which will form a quantitative measure of the skin reflectance properties. An iterative method will then be used to update the surface estimate and the reflectance properties until convergence. Thus, we will arrive at a lookup-table set of reflectance measurements and an optimal shape estimate, which will allow for improved face recognition. This is a novel approach to PS and should allow us to diminish some of the strong assumptions on surface orientation that most current methods impose. The main challenge here will be in forming the relationships between the image-based skin reflectance measurements and the skin orientation for the whole face in order to acquire the optimal 3D shape estimate.The final stage of the project will involve applying face recognition methods developed previously both at the MVL and at other institutions for a comparative analysis. This will demonstrate improvements in recognition rates compared to 3D methods using standard PS and other techniques.

1. Direct impact - Users in the security field will greatly benefit from this research. The PI hopes to demonstrate the work to security experts at an exhibition such as IPOT (Image Processing and Optical Technology). - General Dynamics (GD) will benefit from the new face recognition technology which could be used to improve the security of their military vision systems. GD will also assist in exposing the new technology to a wider marketplace outside the defence sector via their business contacts. It is expected that the results would be suitable for a commercial application, as detailed in the impact plan. GD has previously successfully collaborated with MVL and the PI on a previous grant. - The University of Central Lancashire (UCLAN) will benefit as they will gain the ability to augment their EvoFit face synthesis system with enhanced reflectance modelling. The PI will engage closely with UCLAN and their partners and where appropriate seek means to prolong these relationships into future collaborations. - MVL and UWE will benefit as results will be translated into future research, exploitation and opportunities in teaching. The PI has close and ongoing interaction with the Research, Business and Innovation (RBI) at UWE, which are well structured to promote the take up of research in the application domain. 2. Indirect Impact - Medical Practitioners may benefit from enhanced modelling of skin disorder for improved skin cancer diagnosis, following on from other projects in the MVL. The existing collaboration of MVL with Frenchay Hospital and North Bristol NHS Trust favour the take up of any opportunities in this area. - South West region - development of the region through an enhanced portfolio of research and provision of high level skills training is essential. MVL will be working to maximise the opportunities for user collaborations through existing links with South West RDA. - Policy makers - In the longer term, the outcomes of this research might impact on policies related with techniques for security screening and how these may be acceptable to the public. In particular, the enhanced levels of security, combined with ease of use, are likely to be attractive to policy makers. User partners will have a stake as these. - Government - The nature of this research makes it suitable for use as a case study to highlight the importance of research to the wider public, (e.g. EPSRC impact website). The PI will ensure that any relevant publicity is appropriately highlighted to EPSRC. - Societal impact - The potential to generate new technologies will benefit the public to improve security and prevent crime as well as help identify the perpetrators of crime. The partners in this grant will enable any potential applications to be taken forward for this purpose. Society will also benefit through the knowledge and skills which could be input into the education system as well as generate employment. - Economic impact - Any new products in the market or licensing arrangements that might originate from this research will bring economic benefits to the UK. Also, the research is likely to attract international students who want to develop such skills at UWE.