Investigating automatic detection of emotion in biometrically identified pig faces using machine learning

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.

We propose a highly novel approach of using advances in machine vision and machine learning to automatically detect and monitor key affective states in individually identified pigs and measure performance traits using only the face. The basis for this project is our successful proof-of-concept work showing pig' faces are reliable biometrics.

Our project is phased: Phase 1 involves a series of controlled experimental studies to ground-truth the technology by generating facial images of animals in negative and positive affective states. Images will optimise our existing animal identification algorithms and establish and validate specific facial characteristics relating to affective states. Phase 2 applies this knowledge in a commercial context and investigates facial correlates of animal performance (i.e. weight). Negative affect will be investigated by using established models of pain (naturally occurring lameness) and stress (social defeat). Positive affect will be induced by temporary amelioration of chronic hunger (i.e. removal of a negative state). We have shown that deep learning achieves 97% accuracy in identifying pigs using conventional 2D images. The dataset size and composition will significantly increase to establish scalability limits and quantify the benefits of adding 2.5D and 3D features for more robust biometric recognition. Changes in facial expression can be subtle so 2D, 2.5D, 3D and infrared data will all be explored for automated expression recognition. To detect deviations in individual facial expressions and recognise emotional valence different machine learning techniques will be used (x2 established feature-based approaches, x1 novel, state-of-the-art Convolutional Neural Network). Finally a refined prototype system will be tested on a large commercial farm and additional data taken to investigate facial correlates of weight. This phase determines the feasibility of our innovative, animal-centric welfare assessment tool for the end-user.

Welfare of managed animals: Machine vision and machine learning offer potential to realise low-cost, non-intrusive and practical means to both biometrically identify individual animals and assess and record their condition daily using only the face. This would facilitate on-going learning about individuals and allows for early detection of altered health/welfare, personalised thresholds for intervention, and tailored treatment. Whilst the absence of negative affective state is and should be a priority such technology provides further opportunities to identify positive affective state thus moving closer to the ultimate goal of measuring whether animals experience "a good life".

Pig Producers, veterinarians, breeding companies: Our consortium has strong industry links, with academic partners demonstrating track records in engaging with industry to develop precision livestock technology for commercial uptake. Our industry veterinary partners say tools that can identify animals requiring attention "earlier than the most skilled stockperson" would be highly valued especially in larger operations with increased mechanisation where the ratio of pigs to stockpersons is increasing. Breeding companies are interested in technology that could help incorporate new traits into breeding programmes and select the most robust animals for clients. An ability to detect animals that might be particularly pain and/or stress susceptible at a pre-selection phase would be highly valued.

Welfare accreditation schemes, retailers, supply chain: Welfare assessment measures undertaken on farm for assurance schemes (RSPCA Assured, Red Tractor, Retailer-specific - e.g. "Taste the Difference") rely heavily on systems-based approaches by inspectors observing the farm intermittently at a group level. This project offers a highly animal-centric health and welfare assessment tool that would operate continuously to assess the affective state of individual animals on-farm.

Environmental: Benefits accrue from a more efficient, sustainable pig industry. Higher health and welfare status for breeding animals will lead to longevity in herds, will reduce environmental impact and economic losses for the industry and help improve feed conversion efficiency per kg of meat produced, boosting food security. The individualised data recording in our automated system can also be used in a wider precision farming context by association with other measurable parameters, eg individual food and water intake, treatment history, growth/weight gain, to better optimise farm production efficiency.

Society: Consumers are increasingly aware of and concerned for farm animal welfare. They have historically stimulated legislation to protect animal welfare and recent heated discussions between policy makers and citizens over animal sentience confirms the public's significant and enduring demand that welfare of farmed animals is assured by government. The potential for improved health, welfare and quality standards in the pig sector that this technology presents will lead to an enhanced public perception of industry as well as, via improved animal health and reduced pig farming costs, providing consumers with greater access to lower-cost, higher quality pork products.

Economic: The UK farm industry has a global reputation for livestock production under high welfare standards and transparency of such a reputation will enhance the integrity of the UK pig industry and provide a USP for pig meat produced in this way, giving the UK an added advantage and economic competitiveness. Such an endeavour is particularly timely as the UK agricultural sector aims to position itself in trade negotiations for a post-Brexit era. There will be more immediate economic impacts via exploitation by project commercial partners and pig farmers who will be provided with a precision tool for automatically and objectively monitoring individual pig health and wellbeing, thus improving herd efficiency.