Pig ID: developing a deep learning machine vision system to track pigs using individual biometrics
Lead Research Organisation:
Scotland's Rural College
Department Name: Research
Abstract
Livestock farmers are turning to agri-technology to respond to the challenges of climate change, sustainability, anti-microbial resistance and food security while efficiently producing animals with good health and welfare. Machine vision technology, where software uses 'deep learning' neural networks to automatically process video images, could provide unique insights: 24/7 live data on growth/production and behavioural change as a measure of individual animal health and welfare. Problems such as disease, lameness or harmful behaviours such as aggression and tail-biting could be automatically detected. Continuous tracking in a farm environment is challenging as pigs change shape and can lie next to or on top of each other, and the best available systems can only track pigs for minutes unless they are marked somehow (which commercial growing pigs are not). The Pig ID project represents a step-change in capability by continuously monitoring individual unmarked pigs over weeks and months as they grow.
We previously developed a system which can identify (ID) individual pigs from their faces. Face ID is accurate but positioning cameras to capture pig faces is hard and does not capture whole-pen behaviour. Here we plan to develop a deep learning system using the latest neural networks to learn to recognise individual pigs, updating as they grow and continuously track their movements.
First, building on our face recognition system, we will develop a remote overhead biometric ID system based on head and body features. Only when confidence in the ID of a pig is lost will frame-to-frame tracking be used as a backup until biometric ID can re-acquire the pig. In this way, continuous accurate pig identity and location will be established.
Pigs grow rapidly, so their size and appearance change a lot over time. So next, we will determine how robust our trained ID and tracking system is to this change by testing it on images of the same pigs from different weeks. We will establish how robust it is to changing pig appearance and implement automatic retraining of biometric ID at regular intervals, enabling continued tracking over weeks.
To date, machine vision pig tracking has only been demonstrated with one or a few groups of pigs that it has been trained on. Another crucial innovation of this project is that we will take our trained system and work to develop automated enrolment using 'open set recognition', clustering the images of a new group of pigs by similarity to learn about those new individuals.
To achieve a large volume of training and validation data needed for this project, we will semi-automate by combining machine vision to find pigs in each image, with humans to confirm and label them with ID. Alongside in-person identification of pigs as a failsafe, we will develop a novel way of validating ID using visible colour and ultraviolet (UV) cameras side by side, while pigs have distinctive sun-cream markings, invisible except to the UV camera. These parallel UV images will enable human manual ground-truthing of pig ID in every frame, to check against biometric ID results from colour images where the sun-cream is invisible. Once the biometric ID system is complete, it will also be used with the UV images. It should easily learn to recognise these marked pigs providing further validation data.
This project builds on our proven Face ID technology and we are continuing our previous successful collaboration, bringing together expertise in animal behaviour and welfare, and in agri-technology, particularly in applying cutting-edge machine learning techniques in machine vision/learning to real-word problems. The project has considerable commercial support, with co-funding from animal health company Zoetis under an Industrial Partnership Award. Agri Tech company Innovent Technology Ltd, pig farming and pork processing company Karro, and breeding company PIC are ready to be involved in the next stage of commercialisation.
We previously developed a system which can identify (ID) individual pigs from their faces. Face ID is accurate but positioning cameras to capture pig faces is hard and does not capture whole-pen behaviour. Here we plan to develop a deep learning system using the latest neural networks to learn to recognise individual pigs, updating as they grow and continuously track their movements.
First, building on our face recognition system, we will develop a remote overhead biometric ID system based on head and body features. Only when confidence in the ID of a pig is lost will frame-to-frame tracking be used as a backup until biometric ID can re-acquire the pig. In this way, continuous accurate pig identity and location will be established.
Pigs grow rapidly, so their size and appearance change a lot over time. So next, we will determine how robust our trained ID and tracking system is to this change by testing it on images of the same pigs from different weeks. We will establish how robust it is to changing pig appearance and implement automatic retraining of biometric ID at regular intervals, enabling continued tracking over weeks.
To date, machine vision pig tracking has only been demonstrated with one or a few groups of pigs that it has been trained on. Another crucial innovation of this project is that we will take our trained system and work to develop automated enrolment using 'open set recognition', clustering the images of a new group of pigs by similarity to learn about those new individuals.
To achieve a large volume of training and validation data needed for this project, we will semi-automate by combining machine vision to find pigs in each image, with humans to confirm and label them with ID. Alongside in-person identification of pigs as a failsafe, we will develop a novel way of validating ID using visible colour and ultraviolet (UV) cameras side by side, while pigs have distinctive sun-cream markings, invisible except to the UV camera. These parallel UV images will enable human manual ground-truthing of pig ID in every frame, to check against biometric ID results from colour images where the sun-cream is invisible. Once the biometric ID system is complete, it will also be used with the UV images. It should easily learn to recognise these marked pigs providing further validation data.
This project builds on our proven Face ID technology and we are continuing our previous successful collaboration, bringing together expertise in animal behaviour and welfare, and in agri-technology, particularly in applying cutting-edge machine learning techniques in machine vision/learning to real-word problems. The project has considerable commercial support, with co-funding from animal health company Zoetis under an Industrial Partnership Award. Agri Tech company Innovent Technology Ltd, pig farming and pork processing company Karro, and breeding company PIC are ready to be involved in the next stage of commercialisation.
Technical Summary
We aim to develop a new technological tool using deep learning machine vision to biometrically identify and track individual pigs. Facilitating behaviour monitoring for health and welfare, it enables a step-change in 'precision livestock farming' and has strong industry support.
Training CNNs and validating their performance requires time-consuming manual labelling of ground-truth images. Obj 1 produces this manual data increasing data volume by: a) Using a semi-supervised approach with a modified Mask-RCNN for instance segmentation, and b) Developing a new approach for identity validation: simultaneous recording with a visible light video camera paired with a modified UV-detecting camera, pigs are individually marked with invisible sun-cream which appears black under UV. Our labelled, paired videos will be a valuable public resource for other researchers.
Commercial growing pigs are not ID marked. Building on our existing machine vision technology which identifies ID from pig faces, a convolutional neural network (CNN) will be trained to recognise biometrics of individual pigs from above in a group (Obj 2; we have shown feasibility in previous work).
Next (Obj 3), we use a refined Mask-RCNN to extract individual pigs from their surroundings and implement our trained CNN enabling spatial tracking. Our innovative 'tracking by recognition' eliminates the problem of lost ID and track-swaps due to close-proximity or occlusion, by re-establishing biometric ID. We retrain the CNN to achieve long-term tracking over weeks as pigs grow and change appearance.
We will develop 'open set recognition' (Obj 4), avoiding extensive manual re-training for each new group. Previous work showed that a latent feature space can be optimised to cluster images that appear similar together (because they are from the same pig), separating them from other pigs.
Obj 5 integrates 3 and 4 refining a state-of-the-art system to enrol and track individual pigs over weeks as they grow.
Training CNNs and validating their performance requires time-consuming manual labelling of ground-truth images. Obj 1 produces this manual data increasing data volume by: a) Using a semi-supervised approach with a modified Mask-RCNN for instance segmentation, and b) Developing a new approach for identity validation: simultaneous recording with a visible light video camera paired with a modified UV-detecting camera, pigs are individually marked with invisible sun-cream which appears black under UV. Our labelled, paired videos will be a valuable public resource for other researchers.
Commercial growing pigs are not ID marked. Building on our existing machine vision technology which identifies ID from pig faces, a convolutional neural network (CNN) will be trained to recognise biometrics of individual pigs from above in a group (Obj 2; we have shown feasibility in previous work).
Next (Obj 3), we use a refined Mask-RCNN to extract individual pigs from their surroundings and implement our trained CNN enabling spatial tracking. Our innovative 'tracking by recognition' eliminates the problem of lost ID and track-swaps due to close-proximity or occlusion, by re-establishing biometric ID. We retrain the CNN to achieve long-term tracking over weeks as pigs grow and change appearance.
We will develop 'open set recognition' (Obj 4), avoiding extensive manual re-training for each new group. Previous work showed that a latent feature space can be optimised to cluster images that appear similar together (because they are from the same pig), separating them from other pigs.
Obj 5 integrates 3 and 4 refining a state-of-the-art system to enrol and track individual pigs over weeks as they grow.
