Face Matching for Automatic Identity Retrieval, Recognition, Verification and Management

Lead Research Organisation: University of Surrey
Department Name: Vision Speech and Signal Proc CVSSP

Abstract

In the past, when the majority of people were born, lived and died in the same locality where everybody knew each other, there was no need for biometrics. However, nowadays, with the society moving rapidly towards Digital Economy, and the people mobility within the country and across borders reaching unprecedented levels, efficient, robust and effective ways of recognising and verifying individuals automatically, based on biometrics, is emerging as an essential requirement and element of the fabric of the information infrastructure. Identity verification is required to facilitate commerce, and remote working, to enable access to remote services and physical sites in smart cities, as well as contributing to a safer society by fighting crime and terrorism through automatic surveillance. In this context face biometrics is a preferred biometric modality, as it can be captured unobtrusively, even without subjects' being aware of being monitored and potentially recognised. It is also the modality used by humans and thus, when needed, it supports a seamless transition and cooperation between machine and human face recognition.

Although face biometrics is beginning to be deployed in several sectors, it is currently limited to applications where a strict control can be imposed on the process of face image capture (frontal face recognition in controlled lighting). However, automatic face recognition in uncontrolled scenarios is an unsolved problem because of the variability of face appearance in images captured in different poses, with diverse expressions, under changing illumination. Furthermore, the image variability is aggravated by degradation phenomena such as noise, blur and occlusion.

The project will develop unconstrained face recognition technology, which is robust to a range of degradation factors, for applications in the Digital Economy and in a world facing global security issues, as well as demographic changes. The approach adopted will endeavour to devise novel machine learning solutions, which combine the technique of deep learning with sophisticated prior information conveyed by 3D face models. The scientific challenge will be to develop a face image representation, which is invariant to various imaging factors. This will necessitate gaining better understanding of the effect of natural face appearance variations and face image degradation phenomena on face image representation. The work will be carried out by a multidisciplinary team constituted by three academic partners, University of Surrey, Imperial College London and University of Stirling, which has extensive experience in biometrics and face modelling, and jointly possesses the necessary expertise, including psychology of human face perception. The research direction will be regularly reappraised and if necessary revised, with steering provided by a team of external experts representing the biometrics industry, government agencies, and potential users of the unconstrained face recognition technology. The progress of the project will be measured by extensive evaluations of the solutions developed using challenging benchmarking tests devised by the biometrics community and compared with evolving commercial offerings.

Planned Impact

The proposed EPSRC programme FACER2VM will generate highly visible scientific, economic and societal impact.
Advancement in the understanding of the process of human and machine face matching, and in particular how to model natural face appearance variations and how to handle sensor signal degradation phenomena will contribute to scientific knowledge with wider implications on machine perception. The scientific impact will be enhanced by vigorous dissemination activities through top ranking journal and conference publications.

The scientific achievements will impact on education and training through the partner institutions' research led postgraduate degree programmes and their continuous professional education offerings. It will also contribute to training highly skilled researchers and engineers for the national economy.

Effective solutions to unconstrained face matching will enable the development of new applications and services of biometrics technology in Digital economy, which will foster economic impact. The expected results of the programme grant will help to enhance UK industry competitiveness in a sector forecast by TechNavio to grow in the facial recognition market alone at a CAGR of 26.6% during the period 2013-2018. The commercial impact will be promoted by the industrial partners with keen interest in exploitation of advanced face recognition technology to be developed by the project. An important role in the impact generation will be played by the European Association for Biometrics as a channel to its 65 industrial members with interests in biometrics.

The societal impact is anticipated to be multifaceted. Unconstrained face biometrics capability will significantly contribute to the government's security agenda in the framework of smart cities and national security. It can effectively facilitate access to public services. The impact will be maximized by the involvement of the Home Office CAST, and of Working Group on Biometrics in the FACER2VM user group, an organization representing diverse users of biometric technology in government departments.

Face recognition technology also has diverse applications aiming at enhancing the quality of life of individuals, such as facilitating the personalization of devices and person specific monitoring of individuals in smart homes for all, but especially for the benefit of the aging society.

The outreach element of FACER2VM will contribute to attracting new generations to careers in knowledge economy in general, and science and engineering in particular.

Publications

10 25 50
 
Description As part of the programme of work aiming to develop facial biometric technology for unconstrained scenarios based on machine learning and prior knowledge,
1) a dataset of 3D face images of Chinese ethnicity has been collected. Using this dataset,
2) a new 3D morphable face model has been constructed. The new model is not a global face model, but conceived as a mixture of cohort models (MG3DMM) where each cohort represents different ethnicity. The key advantage of MG3DMM is in facilitating more accurate 3D model fitting to arbitrary 2D face images. This has a beneficial impact on face recognition performance.
3) We have developed a very accurate facial landmarking method which takes advantage of a new loss function for deep convolutional neural network learning in the task of landmarking. Since landmarks underpins all face matching and verification, the very high accuracy landmarking positively affects the accuracy in the other fields.
4) An award-winning algorithm was developed for tracking in unconstrained video. Arbitrary objects can be marked and tracked though occlusion, camera movement (e.g. shaking), light changes and other distractors.
5) Algorithms for the reconstruction of 3D from 2D images / video (both model-based and model-free) have been developed, presented at major conferences and published in top-ranked journals.
6) A soft-biometric system was developed that can match images to textual descriptions and vice versa. In fact, matching between any combination of image data and text, depending on what is available from the users viewpoint.
7) Deeper understanding of human face recognition and face matching ability has been developed and has exposed key differences in human visual qualities compared to current machine learning approaches.
8) Internationally leading face recognition algorithm "arcface" and cross resolution face matching technology.
Exploitation Route 3D face model licensing.
Collaborations and technology transfer activity arising from the User Group.
Spin outs, such as 4dface, FaceSoft and Senses Futuris.
Sectors Aerospace

Defence and Marine

Creative Economy

Digital/Communication/Information Technologies (including Software)

Electronics

Government

Democracy and Justice

Retail

URL https://facer2vm.org
 
Description 3D face model developed has been licensed to NAVER Corporation and several other companies. The facer recognition technology developed by the project is commercialised by spinouts FaceSoft and Sensus Futuris.
First Year Of Impact 2017
Sector Digital/Communication/Information Technologies (including Software)
Impact Types Economic

 
Description Biometrics POST Note
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://www.parliament.uk/post
 
Description Face biometric technology: ethics versus usability. Prof Josef Kittler participated in the Westminster eForum Policy Conference on Biometrics and Digital Identity in the UK:Security and privacy, ethics and trust, public and private sector applications, and next steps for regulation}, held on Tuesday 5 May 2020, where Prof Kittler presented an invited talk prepared jointly with Mr David McIntosh, Chairman FACER2VM Steering Board. The talk pleaded for a balanced view on ethics and usability of biometrics.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description Future face recognition functionality to be introduced in the Shengen Information System of the European Union
Geographic Reach Europe 
Policy Influence Type Membership of a guideline committee
Impact Josef Kittler was a member of an external review board that provided feedback on the plans for the introduction of face recognition in the Shengen Information System of the European Union. The aim is to enhance efficiency and security of the Shengen border control. The plans were developed by European Commission DG Joint Research Centre, Directorate E - Space, Security and Migration, Unit E.3 - Cyber & Digital Citizens' Security
 
Description Interactive Perception-Action-Learning for Modelling Objects
Amount £397,394 (GBP)
Funding ID EP/S032398/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 08/2019 
End 03/2023
 
Description MURI
Amount £8,000,000 (GBP)
Funding ID EP/R018456/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2018 
End 12/2022
 
Description Multimodal Video Search by Examples (MVSE)
Amount £863,564 (GBP)
Funding ID EP/V002856/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2021 
End 09/2024
 
Description Platform Grant
Amount £1,539,000 (GBP)
Funding ID EP/P022529/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 06/2017 
End 06/2022
 
Title 3D Menpo Benchmark 
Description Using our recent breakthroughs for fitting large-scale 3D Morphable Models on in-the-wild images and videos, we made a significant step in advancing the state of the art in the field and developed the first benchmark for 3D facial landmark localization in videos recorded in the wild. The landmark annotations in this case have been produced in a more sophisticated way. In short, a non-rigid structure from motion algorithm was used on the original 2D landmarks of the 300 VW competition to get a first estimate of the 3D coordinates. Afterwards, a modified version of 3D Morphable Model fitting was used that fits all the frames of the video simultaneously. Finally, all landmarks have been visually inspected and manually corrected if needed. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact The 3D Menpo benchmark has been used to organise the 1st 3D Face Tracking in-the-wild Competition in conjunction with ICCV 2017. 
URL https://ibug.doc.ic.ac.uk/resources/1st-3d-face-tracking-wild-competition/
 
Title Additional file 1: of A grey area: how does image hue affect unfamiliar face matching? 
Description Data colour match experiment (Exp.) 1 (SAV 6 kb) 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
URL https://springernature.figshare.com/articles/Additional_file_1_of_A_grey_area_how_does_image_hue_aff...
 
Title Additional file 1: of A grey area: how does image hue affect unfamiliar face matching? 
Description Data colour match experiment (Exp.) 1 (SAV 6 kb) 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
URL https://springernature.figshare.com/articles/Additional_file_1_of_A_grey_area_how_does_image_hue_aff...
 
Title Additional file 2: of A grey area: how does image hue affect unfamiliar face matching? 
Description Data colour match Exp. 2 (SAV 13 kb) 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
URL https://springernature.figshare.com/articles/Additional_file_2_of_A_grey_area_how_does_image_hue_aff...
 
Title Additional file 2: of A grey area: how does image hue affect unfamiliar face matching? 
Description Data colour match Exp. 2 (SAV 13 kb) 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
URL https://springernature.figshare.com/articles/Additional_file_2_of_A_grey_area_how_does_image_hue_aff...
 
Title AgeDB: the first manually collected, in-the-wild age database 
Description Over the last few years, increased interest has arisen with respect to age-related tasks in the Computer Vision community. As a result, several "in-the-wild" databases annotated with respect to the age attribute became available in the literature. Nevertheless, one major drawback of these databases is that they are semi-automatically collected and annotated and thus they contain noisy labels. Therefore, the algorithms that are evaluated in such databases are prone to noisy estimates. In order to overcome such drawbacks, we present in this paper the first, to the best of knowledge, manually collected "in-the-wild" age database, dubbed AgeDB, containing images annotated with accurate to the year, noise-free labels. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact Provide a new benchmark for age invariant face recognition. 
URL https://ibug.doc.ic.ac.uk/resources/agedb/
 
Title BreakingNews: Article Annotation by Image and Text Processing 
Description BreakingNews, a novel dataset with approximately 100K news articles including images, text and captions, and enriched with heterogeneous meta-data (such as GPS coordinates and user comments). We show this dataset to be appropriate to explore intersection of computer vision and natural language processing have achieved unprecedented breakthroughs in tasks like automatic captioning or image retrieval. 
Type Of Material Database/Collection of data 
Year Produced 2016 
Provided To Others? Yes  
Impact This is the largest existing database for analysing visual and natural language content in form of news articles rather than short captions. The database has already been used and referenced by other research labs. 
URL http://www.iri.upc.edu/people/aramisa/BreakingNews/
 
Title Hpatches 
Description Database for training and evaluation of methods for image matching 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact The largest benchmark in this area. It has been and will continue to be used in a number of publications by others. 
URL https://github.com/hpatches/hpatches-benchmark
 
Title Large Scale Facial Model 
Description LSFM is the largest-scale 3D Morphable Model (3DMM) of facial shapes ever constructed, based on a dataset of around 10,000 distinct facial identities from a huge range of gender, age and ethnicity combinations. This model has been built using an especially-designed, fully automated system that accurately establishes dense correspondences among 3D facial scans and is robust to the large shape variability exhibited in human faces. LSFM includes not only a global 3DMM model but also models tailored for specific age, gender or ethnicity groups. This was made possible thanks to the extremely rich demographic information that the used dataset has. LSFM is built from two orders of magnitude more identity variation than current state-of-the-art models. Extensive experimental evaluations (Booth et al., CVPR'16) have shown that this additional training data leads to significant improvements in the characteristics of the statistical modelling of the 3D shape of human faces, and demonstrate that LSFM outperforms existing state-of-the-art models by a wide margin. 
Type Of Material Computer model/algorithm 
Year Produced 2017 
Provided To Others? Yes  
Impact The model is currently used by many hospitals worldwide for face reconstructive surgery planning. 
URL https://xip.uclb.com/i/healthcare_tools/LSFM.html
 
Title Menpo BenchMark 
Description Currently comprehensive benchmarks exist for facial landmark localization and tracking (see 300W [1] and 300VW [5] challenges). Nevertheless, these benchmarks are mainly about (near) frontal faces. In CVPR 2017, we make a significant step further and present a new comprehensive multi-pose benchmark, as well as organize a workshop-challenge for landmark detection in images displaying arbitrary poses. To this end we have annotated a large set of profile faces with 39 fiducial points. Furthermore, we have annotated many new images of (near) frontal faces using the standard 68 point markup. The challenge will represent the very first thorough quantitative evaluation on multipose face landmark detection. Furthermore, the competition will explore how far we are from attaining satisfactory facial landmark localisation in arbitrary poses. The results of the Challenge will be presented at the Faces " in-the-wild" (Wild-Face) Workshop to be held in conjunction with CVPR 2017. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact The menpo benchmark was used to organise the 2nd Facial Landmark Localisation Competition in conjunction with CVPR 2017. 
URL https://ibug.doc.ic.ac.uk/resources/2nd-facial-landmark-tracking-competition-menpo-ben/
 
Title Surrey 3D Morphable Face Shape Model 
Description The Surrey 3D Morphable Face Shape Model is a multi-resolution 3D Morphable Model that we make available to the public for non-commercial purposes. The model contains different mesh resolution levels and landmark point annotations as well as metadata for texture remapping. Accompanying the model is a lightweight open-source C++ library designed with simplicity and ease of integration as its foremost goals. In addition to basic functionality, it contains pose estimation and face frontalisation algorithms. 
Type Of Material Computer model/algorithm 
Year Produced 2016 
Provided To Others? Yes  
Impact * Using the different model resolution levels and fast fitting functionality, we have been able to use a 3D Morphable Model in time-critical applications like tracking. * The software library makes it easy for the community to adopt the 3D Morphable Face Model in their research, and it offers a public place for collaboration. * The model has been downloaded by more than 60 international parties. 
URL http://www.cvssp.org/faceweb/3dmm/facemodels/
 
Description 3D reconstruction 
Organisation University of York
Department Department of Computer Science
Country United Kingdom 
Sector Academic/University 
PI Contribution Developing deep learning methods for 3D facial reconstruction.
Collaborator Contribution Developing deep learning methods for 3D facial reconstruction.
Impact Publication: 3D Morphable Models as Spatial Transformer Networks
Start Year 2017
 
Description Anomaly detection 
Organisation Bilkent University
Country Turkey 
Sector Academic/University 
PI Contribution Joint research on one class classification methods for applications to anomaly detection
Collaborator Contribution The kernel null-space technique and its regression-based formulation (called one-class kernel spectral regression, a.k.a. OC-KSR) is known to be an effective and computationally attractive one-class classification framework. Despite its outstanding performance, the applicability of kernel null-space method is limited due to its susceptibility to possible training data corruptions and inability to rank training observations according to their conformity with the model. This work addresses these shortcomings by studying the effect of regularising the solution of the null-space kernel Fisher methodology in the context of its regression-based formulation (OC-KSR). In this respect, first, the effect of a Tikhonov regularisation in the Hilbert space is analysed where the one-class learning problem in presence of contaminations in the training set is posed as a sensitivity analysis problem. Next, driven by the success of the sparse representation methodology, the effect of a sparsity regularisation on the solution is studied. For both alternative regularisation schemes, iterative algorithms are proposed which recursively update label confidences and rank training observations based on their fit with the model. Through extensive experiments conducted on different data sets, the proposed methodology is found to enhance robustness against contaminations in the training set as compared with the baseline kernel null-space technique as well as other existing approaches in a one-class classification paradigm while providing the functionality to rank training samples effectively.
Impact Paper submitted for publication
Start Year 2018
 
Description Crime investigation 
Organisation Metropolitan Police Service
Country United Kingdom 
Sector Public 
PI Contribution Investigating the use of facial and soft biometrics in CCTV based crime investigation.
Collaborator Contribution User specification and guidance regarding this particular type of application of biometric technologies.
Impact Not applicable
Start Year 2016
 
Description Dissemination 
Organisation European Association for Biometrics
Country Netherlands 
Sector Charity/Non Profit 
PI Contribution Providing information about the FACER2VM project and its research output.
Collaborator Contribution Disseminating information about the FACER2VM project and ts research outputs to the EAB membership. Providing feedback to the project.
Impact Not applicable
Start Year 2016
 
Description Facial biometrics technology 
Organisation Digital Barriers Ltd
Country United Kingdom 
Sector Private 
PI Contribution The team has developed facial biometrics technology for unconstrained face recognition scenarios.
Collaborator Contribution Sharing proprietary database, conducting in-house evaluation of biometric technologies developed by the project, providing guidance regarding facial recognition technology requirements.
Impact Not applicable
Start Year 2016
 
Description International Joint Laboratory for Pattern Recognition and Computational Intelligence 
Organisation Jianghan University
Country China 
Sector Academic/University 
PI Contribution Supervising visiting PhD students/researchers from Jiangnan University, visiting Jiangnan University to collaborate with the local research team.
Collaborator Contribution Working on joint problems in machine learning.
Impact See the Publications section
Start Year 2016
 
Description Long-term identification 
Organisation Jiangnan University
Country China 
Sector Academic/University 
PI Contribution Actively contributed to the creation of the International Joint Laboratory for Pattern Recognition and Computational Intelligence. Acting as one of its co-Directors. The laboratory has facilitated collaborative research between Jiangnan University and Surrey University and is instrumental in making both teams more productive.
Collaborator Contribution facilitating concerted effort, sharing data and resources, and joint work profiting from complementary expertise, making our research more effective.
Impact Not applicable
Start Year 2016
 
Description Matching image patterns 
Organisation Stage Technologies Ltd
Country United Kingdom 
Sector Private 
PI Contribution Actively contributed on creation of a new benchmark and methods for matching image patterns that can also be used in soft biometrics
Collaborator Contribution Providing expertise and training of PhD students and researchers, collecting and processing new data, providing access to computing resources
Impact New benchmark dataset used for international challenge on Landmark localisation within a workshop at the international Conference on Computer Vision and Pattern Recognition. Has been adopted by the scientific community as one of the standards.
Start Year 2016
 
Description Soft biometrics 
Organisation 3rd Forensic Ltd
Country United Kingdom 
Sector Private 
PI Contribution Investigating the merit of soft biometrics in searching police CCTV and other video databases for suspect re-identification and identification purposes.
Collaborator Contribution 3rdForensic provided proprietary database to facilitate the collaborative investigation, as well as user specification and guidance regarding the application of soft biometrics.
Impact Not applicable
Start Year 2016
 
Description Surveillance 
Organisation Home Office
Department Home Office Scientific Development Branch
Country United Kingdom 
Sector Public 
PI Contribution Investigating the use of unconstrained facial biometrics for Home Office applications including watchlist surveillance.
Collaborator Contribution Home Office CAST provided proprietary database of videos representative of the types of surveillance scenarios of interest to Home Office.
Impact Not yet
Start Year 2016
 
Title Surrey Textured 3D Morphable Face Model 
Description The Surrey Textured 3D Morphable Face Model is a statistical model of the shape and texture of the human face, that we make available under a commercial licence. The model contains different mesh resolution levels of shape and texture, landmark point annotations, and metadata for texture remapping. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2016
Licensed Yes
Impact The model allows texture-based fitting for purposes such as face frontalisation, filling in missing data, improved 2D image fitting and extraction of 3D from 2D. The multiple resolution levels enable fast execution in time-critical applications like tracking.
 
Title THREE DIMENSIONAL MODELLING 
Description Two related methods of fitting a three dimensional model, and a method of performing facial recognition, are disclosed. One method comprises estimating and refining geometric information using image landmarks on an object in a two dimensional image. The other method comprises estimating and refining photometric information of the object in the two dimensional image. Furthermore, a method of performing image recognition is provided. 
IP Reference US2018046854 
Protection Patent granted
Year Protection Granted 2018
Licensed No
Impact No progress to date
 
Title 3D Face Model 
Description £D face shape and face skin texture model, defined in terms of shape and texture eigenfaces. 
Type Of Technology Software 
Year Produced 2017 
Impact 6 commercial licences have been issued and many academic licences 
 
Title TFeat 
Description It has recently been demonstrated that local feature descriptors based on convolutional neural networks (CNN) can significantly improve the matching performance. Previous work on learning such descriptors has focused on exploiting pairs of positive and negative patches to learn discriminative CNN representations. In this code, w utilize triplets of training samples, together with in-triplet mining of hard negatives. This implementation achieves state of the art results, without the computational overhead typically associated with mining of negatives and with lower complexity of the network architecture. We compare our approach to recently introduced convolutional local feature descriptors, and demonstrate the advantages of the proposed methods in terms of performance and speed. We also examine different loss functions associated with triplets. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact The method gives state of the art results in standard matching benchmarks. It has been followed by 32 developers in Github and forked by 26 which extended this approach. 
URL https://github.com/vbalnt/tfeat
 
Title eos 
Description eos is a lightweight 3D Morphable Face Model fitting library that provides basic functionality to use face models, as well as camera and shape fitting functionality. It's written in modern C++11/14. Some of the functionality it provides: * MorphableModel and PcaModel classes to represent 3DMMs * Fast head pose, shape and facial expression fitting * Camera pose estimation * Shape-to-landmarks fitting * Isomap texture extraction 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact State of the art 3D from 2D extraction, face texture fusion from in-the-wild videos, 3D face super-resolution from monocular videos. The code has been followed by 280 developers on GitHub and forked by 120. 
URL https://github.com/patrikhuber/eos
 
Company Name 4Dface Ltd 
Description 4dface specializes in facial tracking and 3D personal avatar creation for professional use in various industries such as healthcare, robotics, automotive safety, AR/VR, games, entertainment, and marketing research. They utilize a high-resolution 3D face scan to build their 4D Face Model which enables the creation and reconstruction of 3D face avatars from images or video footage. 
Year Established 2017 
Impact Licencing face technology developed at the University of Surrey to 2+ commercial entities.
Website http://www.4dface.io
 
Company Name Facesoft 
Description Facesoft develops facial recognition software designed for a range of industries, such as virtual reality and security. 
Year Established 2017 
Impact NA
Website http://www.facesoft.io
 
Company Name Sensus Futuris Ltd 
Description Sensus Futuris is a specialized Artificial Intelligence company with a focus on advanced image and video analytics for solutions in security, law enforcement, retail, and other applications. They provide state-of-the-art facial recognition systems and age estimation systems, among other products. 
Year Established 2018 
Impact Product development stage
Website http://sensusfuturis.com/
 
Description Associate Editor of (Elsevier) Image and Vision Computing Journal 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I joined the editorial board of a premium international journal in the field of computer vision.
Year(s) Of Engagement Activity 2016,2017
 
Description Associate Editor of IPSJ Transactions on Computer Vision and Applications (CVA), 2016-2018 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I joined the editorial board of an international journal in the field of computer vision.
Year(s) Of Engagement Activity 2016,2017
 
Description BBC news 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Prof Josef Kittler was interviewed for an article "You are on camera" on Facial Biometrics published by BBC News Business Section.
Year(s) Of Engagement Activity 2017
URL http://www.bbc.co.uk/news/business-38879530
 
Description BMVA Executive Committee member 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact I joined the BMVA Executive Committee for the period of Jan 2016-Dec 2018. The BMVA organises/supports various academic events and activities in the field of computer vision, and the BMVA executive committee meet to discuss regularly throughout each year.
Year(s) Of Engagement Activity 2016,2017
 
Description BPS Cognitive 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk at BPS Cognitive Psychology Section Annual meeting: What correlations can we expect?
Year(s) Of Engagement Activity 2018
 
Description BPS Cognitive Section 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk: "The effects of knowing" reporting the effects of foreknowledge of identity on ability to match faces.
Year(s) Of Engagement Activity 2016
 
Description BPS Cognitive Section 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk at BPS Cognitive Section annual conference 2018: Expertise for faces
Year(s) Of Engagement Activity 2018
 
Description Biometrics technology demonstration 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact This was an open day organised by the Centre for Vision, Speech and Signal Processing on the occasion of the 30th anniversary of its founding. The research and technologies developed at the Centre were demonstrated to circa 250 attendees, mainly form industry.
Year(s) Of Engagement Activity 2019
URL https://www.surrey.ac.uk/centre-vision-speech-signal-processing/about/30th-anniversary
 
Description CAIEP-UK Seminar of Foreign Experts 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact This seminar was hosted in London by the State Administration of Foreign Affairs, China to solicit feedback on various aspects of scientific endeavour in China. The feedback may impact on future cooperation in the field of science and technology between China and the UK.
Year(s) Of Engagement Activity 2018
 
Description Demo, Imperial College Science Festival May 2016 (500+ visitors) 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact We did a hands-on demo for (deformable and articulated object) pose estimation at Imperial College Science Festival, May 2016. Our demo attracted/received 500+ visitors.
Year(s) Of Engagement Activity 2016
 
Description Departmental seminars 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact Departmental seminars at Durham and Edinburgh, Comparing human and computer 'perception' of faces. Open University: What is super about super-recognisers?, Queen Margaret University: 'Selfie' images in face recognition
Year(s) Of Engagement Activity 2019,2020
 
Description EPS London meeting 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Poster describing the Stirling Face Recognition Scale at the London meeting of the Experimental Psychology Society. It generated a lot of interest.
Year(s) Of Engagement Activity 2017
URL http://www.eps.ac.uk/images/epsfiles/2017/epsjan17_programme.pdf
 
Description Explorathon 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Public engagement activity, piloting some experiment ideas and explaining our work to those who took part.
Year(s) Of Engagement Activity 2018
URL http://www.explorathon.co.uk/events/explorathon-extravaganza/
 
Description Explorathon 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Explorathon at the Riverside Museum in Glasgow. Visitors were invited to take part in various pilot studies, illustrating our work. We were able to explain what we are researching.
Year(s) Of Engagement Activity 2019
URL https://www.explorathon.co.uk/
 
Description Explorathon Glasgow 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Explorathon Glasgow was an evening public engagement event organised by the Riverside Museum in Glasgow
Year(s) Of Engagement Activity 2017
URL http://www.explorathon.co.uk/glasgow
 
Description Festival of Wonder at the University of Surrey 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact The University of Surrey celebrated 50 years in Guildford with a Festival of Wonder. Among other things it showcased research activity at the university where the CVSSP FACER2VM team had live interactive demonstrations of face analysis, detection and recognition. It was very well attended. Data collected at the event was shared with the participants, providing a follow-up opportunity to reach out.
Year(s) Of Engagement Activity 2017
 
Description General chair of BMVC17 in London, UK 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact We organised a top-tier premium computer vision conference in Imperial College. A record number of paper submissions, 650, and attendees, 500, were received, in the history of BMVC.
Year(s) Of Engagement Activity 2017
 
Description General co-chair of British Machine Vision Conference (BMVC), London, Sep 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I co-organise this premium international conference on computer vision in London. About +350 participants and +600 high quality paper submissions are expected. The most renowned academic figures in the field are confirmed as keynotes and tutorial speakers for the event. The event this year is expected to be a unique monument in various aspects.
Year(s) Of Engagement Activity 2017
 
Description Glasgow CCTV 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact We organised 4 workshops for operators of the Glasgow CCTV operations centre, to explain some of our understanding of face recognition and to understand better what issues face them in day to day operations
Year(s) Of Engagement Activity 2018
 
Description Guest Editor of (Elsevier) Pattern Recognition Letters Special Issue on Personalised and Context-sensitive Interfaces in the Wild, 2016. 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I have served as a guest editor of a special issue in Pattern Recognition Letter journal.
Year(s) Of Engagement Activity 2016,2017
 
Description Guest editor of Int. Journal of Computer Vision (IJCV), special issue on Deep Learning for Face Analysis 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact We received +60 paper submissions to this special issue. It is very timely and on the important topic, effectively disseminating research outcomes.
Year(s) Of Engagement Activity 2017,2018
 
Description IEEE FG 2018 Workshop on Dense 3D Reconstruction of 2D Face Images in the Wild - Special Session: Competition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Organised a challenge on 3D face reconstruction from a single photo at the IEEE Face and Gesture conference. The challenge also set the benchmark for future evaluations.
Year(s) Of Engagement Activity 2018
URL https://facer2vm.org/fg2018/
 
Description Interview with People's Daily 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Input from an interview with Professor Josef Kittler appeared in the China's People's Daily article written to introduce the recipients of the Chinese Government Friendship Award 2016 to the readership.
Year(s) Of Engagement Activity 2017
URL http://paper.people.com.cn/rmrb/html/2017-01/18/nw.D110000renmrb_20170118_1-22.htm.
 
Description Invited keynote talk on Quantifying Semantic Information, presented at the IAPR S+SSPR 2018 Workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Josef Kittler presented an invite keynote talk on Quantifying Semantic Information, at the International Association for Pattern Recognition Workshop on Statistical, Structural and Syntactic Pattern Recognition, held at Fragrance Hill, Beijing, China on August 17-19, 2018.
Year(s) Of Engagement Activity 2018
URL http://ssspr2018.buaa.edu.cn
 
Description Invited lecture/lab, BMVA computer vision summer school, Swansea, UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact About +60 PG students attended this school, and I gave 1.5 hour lecture and 1.5 hour hands-on session, on random decision forest with deep learning. The participants expressed lots of interest on the topics and told they would use the learnt for their PG studies. The school reported very good feedback received from the students.
Year(s) Of Engagement Activity 2016
 
Description Invited talk at Deep learning summit, London, UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact About +300 engineers/entrepreneurs and postgraduate students attended for the summit, I gave an invited talk with a live demo on the topic of deep learning and random forest, for the applications on hand pose estimation and face recognition. The summit organisers reported very good feedback from audience on the event/topics.
Year(s) Of Engagement Activity 2016
 
Description Invited talk at HiVisComp2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Invited talk at fifth computer graphics and computer vision symposium 2018, Slovakia
Year(s) Of Engagement Activity 2018
URL http://www.hiviscomp.cz/2018/index.html
 
Description Invited talk at Korean Conf. on Computer Vision 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact About +200 PG students and engineers from industry attended this event, where I gave an invited talk on the topic of deep learning and tree-structure algorithms, for hand (articulated object) pose and face (deformable object) recognition, and got lots of questions and discussions during the event.
Year(s) Of Engagement Activity 2016
 
Description Keynote at Korea-Japan joint workshop on Frontiers of Computer Vision, Takayama, Japan 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact About +150 PG students and professors attended this conference, where I gave a keynote. This is a unique event for computer vision researchers especially to promote collaborations and knowledge-sharing between Korea and Japan. My talk on the topic of deep learning and tree structure algorithm sparked lots of questions and interests, the conference organisers reported increased attendance, and excellent feedback on my talk.
Year(s) Of Engagement Activity 2016
 
Description Keynote at Samsung AI Forum, Samsung Software R&D Center, Seoul, Korea 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact I gave an invited talk to AI forum organised by the Samsung group. More than 300 people attended the event, the talk was broadcasted to the whole Samsung group.
Year(s) Of Engagement Activity 2017
 
Description Multimodal and cross modal person re-identification using vision and language 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Josef Kittler presented an invited keynote speech on future of video analytics for security applications at the Global Congress on Biometric Identification held at Shenzhen, October 19, 2018
Year(s) Of Engagement Activity 2018
URL https://www.zkteco.com/en/news_detail/481.html
 
Description Mysteries of Deep Learning 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact The aim of the workshop was for the participants to engage in discussion that would help to gain insight into and intuitive understanding of the workings of deep learning.
Year(s) Of Engagement Activity 2019
 
Description NHK programme on the human brain: face matching in humans compared to computers 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact The Japanese TV Broadcaster NHK came to film an experiment in face matching, comparing the performance of our face recognition system with that of a super recogniser, an individual with exceptional ability to recognise faces, whose company provides services to the UK police in analysing CCTV videos. The aim of the filming was to produce material for a programme on the human brain which NHK are putting together for broadcast in February next year, with an English version to follow three months later.
Year(s) Of Engagement Activity 2017
URL https://facer2vm.org/nhk-filming-at-cvssp/
 
Description ORBIT Seminar: Responsible Innovation for Industry - An Introduction to PAS 440 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact An RBIT seminar on responsible engineering.
Year(s) Of Engagement Activity 2020
URL http://www.orbit-rri.org
 
Description OU Policing Centre 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Presentation to a meeting at the Open University Policing Centre, attended by representatives of police forces, about 'Superrecognition'
Year(s) Of Engagement Activity 2019
URL https://www.open.ac.uk/centres/policing/
 
Description Royal Society discussion meeting on Facial Recognition and Biometrics- Technology and Ethics 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A discussion meeting on Facial Recognition and Biometrics - Technology and Ethics, held at the Royal Society to address issues of trust, privacy and ethics. The meeting was held on 29 January 2020.
Year(s) Of Engagement Activity 2020
URL https://www.adalovelaceinstitute.org/regulation-of-biometrics-debated/
 
Description SARMAC 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Presentation at SARMAC conference 2019: 'Selfie' images in face recognition and face matching by humans and computer algorithms.
Year(s) Of Engagement Activity 2019
URL https://osf.io/meetings/SARMAC2019/
 
Description Scottish Face Research Meeting 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact One day workshop for face researchers across Scotland, July 2019. Our group presented The effect of resemblance to known faces on unfamiliar face memory and matching; If the face fits: Does the perception of face pareidolia relate to holistic face processing?; What makes a good lookalike?; Beyond binary face recognition tasks: The effects of familiarity on accuracy of face memory; Detecting concealed face recognition by eye movements: a comparison of sequential and simultaneous display formats; Identifying Novel Markers of Concealed Face Recognition
Year(s) Of Engagement Activity 2019
 
Description Speaker at the Westminster eForum Policy Conference: Biometrics and digital identity in the UK - security and privacy, ethics and trust, public and private sector applications, and next steps for regulation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Debate on ethical issues of biometrics and digital identity in the UK. Discussing topics including security and privacy, ethics and trust,
public and private sector applications, with the aim of identifying next steps for regulation.
Year(s) Of Engagement Activity 2020
URL https://www.westminsterforumprojects.co.uk/conference/Biometrics-and-digital-identity-in-the-UK-20
 
Description User Group meeting 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact The FACER2VM User Group is a broad international group of companies, constituted by industry (biometrics technology manufacturers and integrators) and face recognition technology users. The purpose is to create a forum where the project results are disseminated, and pathways to impact discussed and identified. Registration to the user group is open.

This event was the third user group meeting. It was advertised internationally via, e.g. the European Association for Biometrics, and drew participation from existing member companies as well as new companies joining the group. Key innovations in face recognition, verification and management were presented and use cases from business discussed to identify new pathways to impact.
Year(s) Of Engagement Activity 2018
 
Description Visit and seminar at Greenwich University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Professor Kittler presented a seminar on Person identification and re-identification. A contact has been established with a company interested in technology transfer.
Year(s) Of Engagement Activity 2019
 
Description WISE 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Professor Kittler presented an invited talk on aspects of Artificial Intelligence at the Westlake International Symposium 2019. The title of his talk was: The changing face of the curse of dimensionality reduction in machine learning.
Year(s) Of Engagement Activity 2019
 
Description Winter School on Biometrics 2020 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Professor Josef Kittler participated as a lecturer at the IAPR Winter School on Biometrics held at Shenzhen in January 2020. The Winter School was attended by 65 PhD students from around the world.
Year(s) Of Engagement Activity 2020
URL https://www.comp.hkbu.edu.hk/wsb2020/