Transfer Learning for Person Re-identification

Person re-identification is an important task in distributed multi-camera surveillance. This is currently performed manually at great economic cost, and with high error rates due to operator attentive gaps. In this project we aim to achieve fast accurate and robust automated person re-identification that can be deployed to any given camera network scenario, without any expensive calibration steps.

Automated person re-identification is the task of associating people based on images captured in video across diverse spatially distributed camera views at different times. This is challenging because the articulation of the human body and variety of viewing conditions such as lighting, angle and distance means that observed appearance typically differs more for the same person in different views than it does for different people. At the same time, it is an important task to solve because re-identification underpins many key capabilities in visual surveillance such as multi-camera tracking. This in turn is a key capability for end-user organizations which need video analytics to achieve a variety of ends including retail optimization, operational efficiency, public safety, security, infrastructure protection and terrorism prevention. Moreover, it is important to automate re-identification because the manual process in large camera networks is both prohibitively costly and inaccurate due to attentive gaps.

Current state of the art re-identification systems use machine learning techniques to produce models for re- identifying across a particular pair of cameras based on manual annotation of person identity in those cameras. However, this is not scalable in practice, because every unique pair of cameras would need calibration with training data. In this project, we will develop new machine learning models that can automatically adapt re-identification models created for an initial set of source cameras to address the re-identification problem in each new pair of cameras without requiring new annotation. This will dramatically improve the practical impact of re-identification technology by making it significantly more accurate as well as cheaper and easier to deploy.

In the short to medium term, beneficiaries of advances in re-identification (ReId) specifically include:

* UK ReId research community. Despite initial groundbreaking impact in the early days of the field, the UK is in danger of loosing leadership to Asian competitors which are investing heavily (CUHK alone 7 people working on ReID).

* Video-content analytics companies who could commercialize the ReId IP outcomes of this project, includ- ing both SMEs like VSL (see included letter of support) as well as big industry such as BAE and IBM (who have submitted a flurry of patents in this area recently). ReId technology is used in a variety of downstream analytics tasks such as long-term tracking and person search, thus impacting the breadth of the analytics industry.

* End-user organizations tasked with video analytics for retail optimization, operational efficiency, public safety, security, infrastructure protection and terrorism prevention. Previous end user partners with whom we have partnered via VSL for trailing this type of research outputs include: Tesco, BAA (Heathrow Airport), BAE, UK MOD and London Metropolitan Police. As two specific examples:

- The London Met would like to use ReId technology for, among other tasks, automatic long-term monitoring of well known professional thieves. This is in order to catch them in the act so as to be able to secure arrests, convictions and deportations. Without this technology, the manpower does not exist to consistently keep tabs on these criminals, allowing thefts to go on unfettered. The thieves are fully aware of the Met's limitations and brazenly industrialize and professionalize their activities as a result.

- BAA have trialled person ReId for both security and operational efficiency reasons: In security they would like to perform long-term tracking of the whereabouts of someone leaving an illegally parked vehicle (a potential car-bomb danger). This is so they can intercept this person as soon as possible and resolve the situation quickly so as to minimize both risk of "true positives" and disruption to the airport by the numerous "false positives" (most violators are not of malicious intent, but all need to be dealt with just in case). On efficiency, monitoring how long it takes airport users to get through various checks and queues is key to optimizing operational processes. Without ReId technology, this information is actually quite hard to get. Thus ReId technology is being trialled to provide this information to help improve procedure efficiency.

- In each case, ReId technology has not yet fulfilled its promise well enough for operational deployment. This is because of limited practicality (annotation requirements) and the inter-related issue of significantly weaker performance out of the lab (change of domains). These are exactly issues which this proposal will address (annotation requirements, WP1), performance improvements (all WPs, but especially WP3).

* Citizens will enjoy more efficient service, increased safety and reduced crime resulting from more effective ReId.

* Other QMUL research groups. Imminent projects include: a Spanish collaboration on gait-based ReId will exploit our results (gait recognition needs to transfer across covariates of ground surface type); and an FP7 project on UAV-based surveillance will test our ReId algorithms from an overhead perspective: in which zero view annotation models are an intrinsic requirement since the view changes dynamically.

In the longer term, unsupervised transfer learning outcomes have potential to impact to a wide variety of applications including for example, transferring music recognition models across various acoustic conditions (work being done at QMUL's C4DM), transferring medical diagnosis and risk assessment Bayesian networks (work being done at QMUL's RIM).

These impacts will be achieved via trials through VSL and various collaborations and seminars - please see PTI for details.