MEASURING AND REPRODUCING THE 3D APPEARANCE OF HUMAN FACIAL SKIN UNDER VARYING ILLUMINATION CONDITIONS: A 3D IMAGING SYSTEM FOR HUMAN FACIAL SKIN

Understanding human skin appearance is a subject of great interest in science, medicine and technology. In medicine, skin appearance is a vital factor in surgical/prosthetic reconstruction, medical make-up/tattooing and disease diagnosis. The production of facial prostheses to replace missing facial structures requires the skills of highly trained anaplastologists to correctly match the shape and colour of the prosthesis to that of the host skin. With the 3D printing of human skin now available the process involved in matching natural and manufactured skin samples has become essential; a robust, accurate and efficient imaging system is required that acquires the relevant skin information and predicts a good match and translates this information through this new and innovative manufacturing process.

A major problem with manufactured skin is that the match to the individual's natural skin must hold not only be accurate under a particular ambient illumination but the match needs to be preserved when the individual is moving between different environments, e.g. when the individual moves from office or LED lighting into daylight. To achieve this illumination invariance, the physical properties of the skin need to be taken into account. A further requirement for successful skin reproduction is the development of appearance models. These can be considered as individual "recipes' or 'blueprints" for each skin type and these not only represent inter-personal differences - different ethnic groups and age ranges, but also intra-personal differences - for each individual. Features of the human skin (wrinkles, pores, freckles, spots etc) make human skin as individual as a finger prints and thus, for facial prosthetics applications, skin appearance models also need to be fine-tuned for each individual area.

The purpose of this work is to develop a complete spectral-based 3D imaging system which will allow us to additively manufacture soft tissue prosthetics or deliver predictable tattooing techniques that will exactly match the skin colour of a particular individual (Application 1) or have the capability to rapidly manufacture/3D print soft tissue replacements representative of a particular ethnic/age/gender group with a high degree of accuracy (Application 2). In application 1, the input to this 3D imaging system will consist of a 3D colour skin image (of a particular individual) obtained with a 3D camera in conjunction other specific skin characteristics. The skin sample will then be printed using a printer profile that maximises the match between the natural and printed skin across different ambient illuminations. In application 2, the skin manufacturing process will not be fine-tuned for a particular individual, but input to the 3D imaging system will consist of basic information about the age, gender and ethnicity. Representative skin samples (colour; texture; translucency; geometry) for this group will then be loaded from a pre-computed library instead of using the measurements from an individual.

1. Healthcare provision for general population: In medicine, skin appearance is a vital factor in surgical/prosthetic reconstruction, medical make-up/tattooing, disease diagnosis (e.g. skin cancer) and evaluation of phototherapy.
2. Individualised health care: With 3D printing of skin on the horizon (see quotation and supporting statement from Fripp Design & Research), a robust and quick method to predict the appearance of skin prostheses is essential. With the knowledge gained in this project the skin manufacturing processes (traditional or new) can be tailored to individual patients to allow for colour shifts that occur during the production of facial prostheses. Quantitative colour models will allow clinicians to understand and improve predictions of colour changes that occur in donor tissues following reconstructive procedures. It may also allow for tissue selection based of predictive colour/aesthetic outcomes rather than solely on operator preference or "ease of harvest". These could be tailored to patient specific requirements and outcomes.
3. Cosmetic Industry: The aesthetic relevance of skin appearance has led to an enormous investment in skin research, mostly for make-up development and cosmetic surgery. Accurate models of skin appearance, particularly under a change in ambient illumination, is vital for the production of skin make-up and will therefore support the UK economy.
4. Entertainment industry: The advent of new lighting technologies (LEDs) and image capture/projection systems generates new challenges for rendering skin on displays and this project will help developers in the entertainment industry to improve the rendering of human faces (see the attached supporting statement from SONY)
5. National Security/Government: Computer vision systems for people tracking and face recognition used in national security applications rely on realistic skin appearance models. With the advent and continued evolution of high definition security/CCTV camera systems on the horizon, colour recognition as well as geometric recognition models will be vitally important.
6. CIE requirements: The CIE (Commission Internationale de l'Eclairage) is an international organisation that is active in all professional matters related to light, lighting and image technology. While the CIE cannot be a project partner, Professor Mike Pointer (CIE, Division I; see supporting letter) will provide his expertise on the acquisition process of the skin spectra and RGB images, which will then feed into the CIE committee on skin data bases (Reporter: KX, the co-investigator). Via the CIE the data bases will be made available for bench marking purposes to the computer vision and lighting/engineering community.
7. Lighting, Engineering and Virtual Reality applications: Accurate rendering or simulation of human faces under different viewing conditions relies on satisfactory spectral reconstruction algorithms for facial skin and on a knowledge of the perceptual tolerance to skin tone and texture distortions. The outcome of this project will therefore benefit these emerging industries (see the attached supporting letter from OPTIS).
8. Prosthetics industry: The Skin colour database and spectral reconstruction techniques of skin colour from digital camera images is essential for skin colour formulation used in the prosthetics industry. The outcome of this project will therefore benefit these industries (see supporting letter from SpectroMatch)
9. Public Engagement Activities. After the successful completion of the proposed research we intend to put on a 3-months-long installation at the World Museum in Liverpool. Visitors will be able to have their 3D face picture taken and then manipulated. They will be able to manipulate their 3D facial image by making their skin look younger or older, change their skin texture, and view their face using the skin colour and texture of a different ethnicity.