PredictinB statg Tus of dairy cows from mid infra-red spectral data using machine learning

Lead Research Organisation: SRUC
Department Name: Research


Bovine tuberculosis (bTB) is a chronic, infectious and zoonotic (i.e., it can be transmitted to humans) disease endemic in the UK and other countries, and presents a significant challenge to the UK cattle sector particularly in the south west of England and south Wales. The Department for Environment, Food and Rural Affairs (DEFRA) lists bTB as one of the four most important livestock diseases globally. The continued spread of bTB among cattle in England and Wales has been a socioeconomic disaster for over 40 years, causing catastrophic and devastating damage to farming businesses both large and small. In 2017 the number of animals in the UK slaughtered due to bTB was in excess of 43,500. The disease has proven difficult to completely eradicate using techniques that are socially acceptable and at a cost acceptable to the UK taxpayer. Current costs are estimated at over £175 million per year with an average cost of £34,000 per bTB outbreak per farm. The continued polarised debate on the role of wildlife as a farmed cattle disease reservoir is making progress slow. This project seeks to develop a non-invasive tool created from routine milk recording of dairy cattle to predict bTB status from milk analysis (by spectrophotometry) by exploiting state of the art Deep Learning techniques. Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms (an algorithm is a process followed to solve calculations). Deep learning works by imitating the way that the human brain works and involves feeding a computer system a large volume of data, which it can use to make decisions about other data. This method of analysis has been successfully deployed by our group to predict pregnancy status in dairy cows with high accuracy and hence expectations are high that bTB leaves a signal in milk that can be detected with Deep Learning applied to MIR spectral data.

The involvement of a commercial partr (NMR, National Milk Records) that is extensively active in the bTB area ensures that results can be rapidly applied to maximise impact in the short term. Furthermore, NMR has a long history of supporting dairy farmers in herd management (including disease) and so the results of this project will be exploited in a familiar context for dairy farmers ensuring its widespread uptake.

Technical Summary

Using a Deep Learning approach we will develop a computer pipeline for routine prediction of bTB status from milk MIR spectra, a by-product of routine milk recording.

Individual cow records from multiple sources will be collated into a central database at SRUC. Records will include animal and herd identification, lactation information, pedigree, bTB skin test status, date of birth/death, movements and MIR spectra. Currently, after routine predictions for milk fat and protein have been carried out, the spectral data are stored for prediction of other important traits such as fatty acids and body energy.

Deep learning (a sub class of Machine Learning) will be utilised to analyse historic national bTB test results and milk MIR spectral data. Data will be modelled using deep convolutional neural networks following a supervised approach and validated in the first instance using SRUC's officially TB-free Langhill herd. Further validation will come from applying the model to a variety of different datasets set up to test predictions both between and within herd. Thereafter, the prediction pipeline will used to investigate whether the milk MIR can be used to determine the point at which a cow became infected with bTB. If successful this would offer the potential to significantly reduce the length of bTB breakdowns by allowing the removal/isolation of infected cows from the herd sooner than is the case currently.

The computing system will be constructed so as to allow for immediate deployment by NMR, the UK's largest milk recording organisation, in a commercial setting with on-going support from SRUC. A number of options will be considered including the use of a specialised GPU powered server vs. Cloud based server for model training or localised offline model training followed by real-time prediction. The objective is to construct a set of processes that allow real time predictions of bTB status at a cost that will be economically realistic and feasible for NMR.

Planned Impact

The impact of the proposed work is expected to be multi-faceted as detailed below:

Milk recording companies and other organisations involved in livestock breeding (e.g. breed societies, levy boards, breeding companies): A successful outcome of the project is expected to increase the return on investment in milk recording by using the same milk sample to predict a range of additional traits; explicitly bTB status. The industrial partner (NMR), in particular, will further benefit from being able to immediately and directly commercialise the outcomes in the form of additional services to farmers and to create new and novel services to assist dairy farmers in managing animal disease. Recording also allows monitoring bTB prevalence and so the efficacy of the testing and culling scheme.

Government: bTB is expensive for both government and taxpayer. It consumes finance that could be better utilised in other areas to generate more income (and tax). The potentially reduced incidence of major disease outbreaks will create a more vibrant and efficient dairy system by allowing reduced restrictions and less risky trading of cattle. The removal of infectious animals earlier is expected (but not yet known) to lower the general level of infection and potentially, the cow to wildlife transmission thereby altering the dynamics of bTB spread.

Farmers: At present over 3,700 herds have a breakdown status and in the last year (2017) alone over 43,500 animals were slaughtered as a result of testing positive for bTB. This has profound implications for farmers not only in a business sense but also psychologically. No farmer wants to have rampant bTB on their farm and so any initiative that helps in the early identification and rapid removal of potentially infectious animals from the dairy herd will find widespread approval. Most importantly, the optimal utilisation of genetic resources in short term selection will enhance the long term sustainability of the supply chain. Currently, high values animals are being lost to the selection pipeline through this disease.

Consumers: The successful implementation of the proposed testing framework will enhance the efficiency of the dairy industry and the profitability of the sector. The benefits of improved efficiency and robustness of agricultural systems has benefits right across the entire supply chain as seen recently when supply chains were disrupted due to extreme weather. Consumers will benefit through tax revenue being utilised for alternative initiatives.

The UK science base: The methods developed and project data produced will contribute to the increased research capacity within the UK (and beyond). The scope of the project is aligned with the BBSRC Strategic Research Priority 1 (Agriculture and Food Security) and successful completion will contribute to the competitiveness and excellence of the UK science base as well as its positioning at the frontiers of delivering novel tools to address the challenges of global agricultural production. Of particular note is the use of Deep Learning with large scale agricultural animal data to gain new insights to improve food production.

Training: The proposed research will feature in training courses that the applicants are regularly invited to present e.g. farmer training days, "Vetnomics" and undergrad teaching. The PDRA working on the project will have the opportunity to be trained in a cutting edge area of research on the use of Deep Learning to predict new disease phenotypes from milk mid infra-red spectral data, while interacting with other scientists in a world-leading research environment as well as with a leading commercial partner intent on application and making a difference.

Policy: The reduction of bTB in the dairy herd is a vital objective of DEFRA and the outcomes of this project have the potential to contribute to the 25 year TB Eradication Strategy without compromising long-term competitiveness of the dairy herd.


10 25 50
Description We have found that bTB skin test status can be predicted using cows milk mid infra-red spectral data. Using already collected data that is not included in model development we have internally validated the predictions to be of very high accuracy and warrant further validation using new data so far uncollected from herds in breakdown.
Exploitation Route further research could be undertaken to investigate the possibility of detecting a wide range of features in the cows milk. This might include diseases such as Johne, BVD but also other important characteristics like antibiotic presence. This is one of the first reported uses of Deep Learning in animal agriculture and could lead to an increase in the use of machine learning to agricultural data.
Sectors Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice,Retail

Description The commercial partner has been advertising the project to their customers and it has been very well received (and anecdotally dominated all of their farmer meetings). It has also led to a reappraisal of the use of spectral data by NMR and they are now considering future traits of interest such as methane emissions and feed intake.
First Year Of Impact 2020
Sector Agriculture, Food and Drink
Impact Types Economic

Description Interview - Vet Times 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Interview regarding the project published online and in print - increased interest in deep learning
Year(s) Of Engagement Activity 2019
Description Meeting with Cheif veterinary officer for Welsh assembly 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact We were due to go to Cardiff to discuss the project and its findings but Flybe went bust so we had to conduct a teleconference with representatives of Welsh Assembly and Defra. We talked through the presentation that shows we can predict TB skin test status with high accuracy using milk mid infer-red spectral data. we discussed how to take it further and conduct a field trial in a live setting for herds that were under breakdown
Year(s) Of Engagement Activity 2020
Description Press Release 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Project was launched with a press release that resulted in print and online coverage nationally
Year(s) Of Engagement Activity 2019