LOCATE: LOcation adaptive Constrained Activity recognition using Transfer learning

It is estimated that there are six million surveillance cameras in the UK, with only 17% of them publicly operated. Increasingly, people are installing CCTV cameras in their homes for security or remote monitoring of elderly, infants or pets. Despite this increase, the use of the overwhelming majority of these cameras is limited to evidence gathering or live viewing. These sensors are currently incapable of providing smart monitoring - identifying an infant in danger or a dehydrated elderly. Similarly, CCTV in public places is mostly used for evidence gathering.

Following years of research, methods capable of automatically recognising activities of interest, such as a person departing a service station without making a payment for refueling the car, or one tampering with a fuel dispenser, are now available, achieving acceptable levels of success and low false alarms. Though automatic after installation, the installation process not only requires putting the hardware in place but also involves an expert studying the footage and designing a model suitable for the monitored location. At each new location, e.g. each new service station, a new model is needed, requiring the effort and time of an expert. This is expensive, difficult to scale and at times implausible such as for home monitoring for example. This requirement to build location-specific models is currently limiting the adoption of automatic recognition of activities, despite the potential benefits.

This project, LOCATE, proposes an algorithmic solution that is capable of using a pre-built model in a different location and adapting it by simply observing the new scene for a few days. The solution is inspired by the human ability to intelligently apply previously-acquired knowledge to solve new challenges. The researchers will work with senior scientists from two leading UK video analytics industrial partners; QinetiQ and Thales. Using these partners' expertise, the project will provide practical and valuable insight that can further boost the strong UK industry of video analytics. The United Kingdom is currently a global player in the video analytics market, and the leading country in the Europe, Middle East and Africa (EMEA) region.

The method will be applicable to various domains, including for home monitoring and CCTV in public places. To evaluate the proposed approach for home monitoring, LOCATE will work alongside the EPSRC-funded project SPHERE, which aims to develop and deploy a sensor-based platform for residential healthcare in and around Bristol. The findings of LOCATE will be integrated within the SPHERE platform, towards automatic monitoring of activities of daily living in a new home, such as preparing a meal, eating or taking medication.

The targeted plug-and-play approach will enable a non-expert user to setup a camera and automatically detect whether an elderly in the home had had their meal and medication, for example. A shop owner can similarly detect pickpocketing attempts in their store. The community can thus make better use of the already in place network of visual sensors.

A) Economic Impact:
The LOCATE framework attempts to enable plug-and-play automatic activity recognition using visual sensors. This is central to the already strong and growing UK industry in video analytics. Current approaches require hand-crafted location-specific activity representation models, increasing costs and at times limiting the applicability of automatic monitoring. A location-adaptive solution would (i) decrease installation costs and enable wider adoption of automatic activity recognition, (ii) extend solutions to domains where location-specific models are difficult to obtain such as homes and highly-sensitive security environments, and (iii) encourage other established companies and start-ups to enter the video analytics market, as a result of the decrease in cost and increase in applicability.

Recently, a number of UK and international SMEs have focused on developing mobile applications that are capable of achieving basic image processing such as motion detection towards alarms for intrusions in residential environments. The LOCATE framework could encourage these SMEs to expand their approaches to more advanced computer vision methods that are capable of detecting activities such as an infant in danger or a pet unable to access food or water. This could result in further boosting the customer base of these SMEs. During the follow-up phase of the project, links will be established to these SMEs and start-ups.

LOCATE primarily aims to make the most of the already installed and functioning network of wired and wireless cameras in the UK. Empowering this infrastructure to detect and prevent, rather than to be used for evidence gathering or only scarcely for live viewing, is a better utilisation of available resources.

B) Societal Impact:
Following from the economic impact, wider adoption of automatic monitoring and its extension to novel domains is one step closer to a healthier and safer society.
When applied to healthcare monitoring, Activities of Daily Living (ADL) have been established as a measure of one's functional status and quality of life. Automatic monitoring of ADLs would allow better assessment of one's health as well as intervention when needed.
When used for surveillance, automatic detection of activities of interest will enable intervention towards saving belongings as well as lives.
Automatic understanding of a person's activities can also encourage developing approaches to human computer interaction as well as robot computer interaction that are smarter with agile responses.

C) Academic Impact:
The project contributes to two research areas: visual activity recognition and relational-knowledge transfer learning, establishing a novel area of research in relational-knowledge transductive transfer learning for visual activity recognition. A challenge will be released to encourage other researchers to pose solutions to this problem.
The LOCATE project aims to establish the PI as a leading researcher in this novel area, continuing what is already a successful career in video analysis and activity recognition. The project will establish working collaborations between the PI and the current project partners as well as new extended collaborations.
At least one postdoctoral researcher and PhD candidate will become proficient in transfer learning approaches - a skill in high demand in research laboratories world wide and of profound effects for applications in Computer Vision and beyond.