AUTOMAC: AUTOmated Mouse behAviour reCognition

Lead Research Organisation: Queen's University of Belfast
Department Name: Electronics Electrical Eng and Comp Sci

Abstract

Context of the research

Neurodegenerative diseases are characterised by motor deficiencies. For many of them, there are no successful neuroprotective or neuroregenerative therapies clinically available. In order to address this problem, the development of valid animal models for motor disorders has become active growing and vibrant field in preclinical research. Behaviour analysis of laboratory animals has been recognised as a useful tool to assess therapeutic efficacy. The entire process consists of animal tracking and motion categorisation. Despite tremendous efforts made within the research community, there is no system which can perform reliable recognition of complex animal behaviours and interactions. In this project, a fully automated and trainable computer vision system is proposed to monitor and analyse complex mouse behaviours and interactions using video data recorded by calibrated cameras.

Aims and objectives

Scientific:
(1) To develop a system of combining multi-camera tracking and a Hidden Markov Model.
(2) To improve the multi-camera tracking performance combining covariance descriptors and probabilistic data association.
(3) To accelerate K-means clustering using an approximate nearest neighbour algorithm.
Practical:
(1) To improve the performance and scalability of the existing behaviour analysis systems.
(2) To widen the scope of the applicability of the developed tracker and the behaviour recognition system.
(3) To associate healthcare applications with the image and vision computing community.

Potential applications and benefits
(1) This project will help researchers in the healthcare/medical community to significantly reduce annotation time/errors and hence improve medical research quality. The research outcomes of the proposed multidisciplinary project can reach both ICT and healthcare communities by our attendance at conferences in different domains. In the meantime, the research communities will benefit from our publications in journals and publicly accessible tools/databases for sharing skills and experiences.
(2) The proposed research may be commercialised in the form of software tools. The UK hosts many companies that offer services related to the treatment of neurodegenerative diseases, e.g. GSK, and Orion pharmaceutical companies. These companies and their clients stand to profit from improvements in disease modelling and diagnosis/treatment techniques that depend on animal modelling using a system like that proposed in this project. Significant scientific improvements in this field will have a transformative effect on these businesses.
(3) The technologies developed in this project can be directly transferred and applied in physical security, human computer interface and virtual reality. The tools developed in the proposed research can be used to monitor moving objects (e.g. humans and animals) in different set-ups (e.g. authentic and virtual environments). The software package produced in the proposed research can easily find its customers in manufacture and design, basic science, communication engineering, media and entertainment. As a result, there is great potential for wealth creation and boosted economic prosperity from the developed software package for a wider range of applications.

Planned Impact

Neurodegenerative diseases are characterised by motor deficiencies. For many of them, there are no successful neuroprotective or neuroregenerative therapies clinically available. In order to address this problem, the development of valid animal models for motor disorders has become an active field in preclinical research. Behaviour analysis of laboratory animals has been recognised as a useful tool to assess therapeutic efficacy. The entire process consists of animal tracking and motion categorisation. Despite tremendous efforts made within the research community, there is no system which can perform reliable recognition of complex animal behaviours and interactions. In this project, a fully automated and trainable computer vision system is proposed to monitor and analyse complex mouse behaviours and interactions using video data recorded by calibrated cameras.

Who will benefit from this research?

The direct beneficiaries of this project are organisations in Information, Communications Technologies and healthcare sectors. Research groups in these areas will benefit from our published research results and tools created from this work which will be made available as open source software in a Matlab format. In addition, pharmaceutical companies will benefit from the showcasing of our research results by contributing to the progress of clinical translational research. Furthermore the proposed research holds the potential to benefit industrial applications such as physical security (e.g. IBM and Tyco) and media production (e.g. Northern Ireland Screen and Wolfhound Media for animation).

How will they benefit from this research?

(1) Currently behaviour analysis of laboratory animals for neurodegenerative disease related research is still undertaken manually. This process is extremely tedious and labour-intensive with inter-annotator biases. This project, if successful, will help researchers in the healthcare/medical community to significantly reduce annotation time/errors and hence improve medical research quality. The research outcomes of the proposed multidisciplinary project can reach both ICT and healthcare communities by our attendance at conferences in different domains, e.g. British Machine Vision Conference in 2015 and World Parkinson Congress in 2016. In the meantime, the research communities will benefit from our publications in journals and publicly accessible tools/databases for sharing skills and experiences.
(2) The proposed research may be commercialised in the form of software tools. The UK hosts many companies that offer services related to the treatment of neurodegenerative diseases, e.g. GSK, and Orion pharmaceutical companies. These companies and their clients stand to profit from improvements in disease modelling and diagnosis/treatment techniques that depend on animal modelling using a system like that proposed in this project. Significant scientific improvements in this field will have a transformative effect on these businesses. As diagnosis and treatment of neurodegenerative diseases is a global challenge, UK products and services offered in this area benefit from the proposed research by commercialising and exploiting research results of this project and contributing to the creation international commercial opportunities.
(3) The technologies developed in this project can be directly transferred and applied in physical security, human computer interface and virtual reality. The tools developed in the proposed research can be used to monitor moving objects (e.g. humans and animals) in different set-ups (e.g. authentic and virtual environments). The software package produced in the proposed research can easily find its customers in manufacture and design, basic science, communication engineering, media and entertainment. As a result, wealth creation and economic prosperity will follow from the developed software package for a wider range of applications.

Publications

10 25 50
 
Description Manual measurement of mouse behaviour is highly labour intensive and prone to error. This investigation aims to efficiently and accurately recognise individual mouse behaviours in continuous videos. In general, the major novel technologies developed in this funded project include: (1) Variants of standard deep neural networks and autoencoder for object detection and recognition. (2) A novel Markov Chain method for motion trajectory clustering. (3) New bag of words based methods for gesture recognition. (4) New transfer learning techniques for object recognition. (5) New evidential reasoning tools for abnormal event detection on public transport vehicles. (6) New deep learning techniques for mouse motion analysis. (7) New low-ranking feature representation methods for image segmentation.
Exploitation Route The proposed research will directly benefit computer vision and movement disorder researchers, who make continuous efforts to reducing the inherent high labour cost and inter-investigator variability associated with the manual annotation of mouse behaviours. The traditional systems strongly rely on the use of specialised sensors and are thus limited to the analysis of relatively simple and pre-programmed behaviours. This project, technically still ongoing, will bring the research of mouse behaviour analysis a step forward in the assessment of therapeutic efficacy of neurodegenerative diseases. In addition, the proposed research may be commercialised in the form of software tools and monitoring devices. These commercialised outcomes will be adopted by medical device manufacturers and therefore the immediate beneficiaries of this project consist of pharmaceutical companies, medical associations and healthcare service agencies which perform translational research based on preclinical animal models.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare,Pharmaceuticals and Medical Biotechnology

URL https://www2.le.ac.uk/departments/informatics/people/huiyu-zhou
 
Description Research in mouse behaviour analysis starts making impacts on several areas: (1) The camera kit built in this project is used to capture animals' videos in different medical exercises and projects. (2) Animal motion detection technologies developed in this project are transferable and have helped us to win over £500K research grants across several disciplines. (3) Through public seminars and presentations, the public community increases their awareness to the damage and negative impacts of Parkinson's Disease. (4) The technologies developed in this project start attracting the attention of the public and hence have the potential to influence policy making in some areas.
First Year Of Impact 2017
Sector Digital/Communication/Information Technologies (including Software),Healthcare
Impact Types Cultural,Societal,Economic,Policy & public services

 
Description Post-graduate course development at Queen's University Belfast, Web and Mobile App Development (CSC7054)
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description An early warning foetal motion detection system
Amount £2,000 (GBP)
Organisation Puffin Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 05/2017 
End 04/2018
 
Description Centre for Secure Information Technologies
Amount £5,510,546 (GBP)
Organisation Invest Northern Ireland 
Sector Public
Country United Kingdom
Start 02/2017 
End 01/2022
 
Description Pathological cancer image analysis
Amount £298,887 (GBP)
Organisation Invest Northern Ireland 
Sector Public
Country United Kingdom
Start 06/2016 
End 05/2018
 
Description Plasmofluidic analytical device: a rapid point-of-sampling diagnostic and management support platform for infectious and antimicrobial resistant pathogens
Amount £187,470 (GBP)
Organisation Invest Northern Ireland 
Sector Public
Country United Kingdom
Start 11/2016 
End 10/2018
 
Description Smart distribution grid: a market driven approach for the next generation of advanced operation models and services
Amount € 3,996,125 (EUR)
Funding ID 771066 
Organisation European Commission H2020 
Sector Public
Country Belgium
Start 10/2017 
End 03/2021
 
Description Smartphone Analyzers for On-site Testing of Food Quality and Safety
Amount £569,130 (GBP)
Funding ID 720325 
Organisation European Commission H2020 
Sector Public
Country Belgium
Start 01/2017 
End 12/2020
 
Description Using game theory to assess the effects of social norms and social networks on adolescenet smoking in schools: a proof of concept study
Amount £727,000 (GBP)
Funding ID MR/R011176/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 01/2021
 
Description Video traffic analysis for abnormal event detection
Amount £105,000 (GBP)
Funding ID NA160342 
Organisation The Royal Society 
Sector Academic/University
Country United Kingdom
Start 03/2017 
End 02/2020
 
Title Mouse behaviour analysis 
Description This tool allows one to automate monitoring of mouse movements and interaction in a home cage. The key contribution of the proposed method is based on Hidden Markov Models with deep learning to handle the uncertainty and ambiguity in the estimation of actions in dynamic environments. 
Type Of Material Physiological assessment or outcome measure 
Year Produced 2017 
Provided To Others? Yes  
Impact The developed tool has the potential to change the human assessment criteria currently recruited in the medical science. 
 
Description China-UK 
Organisation Beihang University
Country China 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation Coventry University
Country United Kingdom 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation Harbin Institute of Technology
Country China 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation Queen's University Belfast
Country United Kingdom 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation University of Manchester
Country United Kingdom 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation University of Sydney
Country Australia 
Sector Academic/University 
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Description China-UK 
Organisation Xidian University
PI Contribution Algorithmic design and evaluation.
Collaborator Contribution Algorithmic design and evaluation.
Impact Zhang, X., Zhang, J., Li, C., Cheng, C., Jiao, L. and Zhou, H. "Hybrid unmixing based on adaptive region segmentation for hyperspectral imagery". IEEE Trans. On Geoscience and Remote Sensing, Accepted. Zhang, L., Zhang, S., Jiang, F., Qi, Y., Zhang, J., Guo, Y. and Zhou, H. "BOMW: Bag of manifold words for one-shot learning gesture recognition from Kinect". IEEE Trans. On Circuits and Systems for Video Technology. Early online publication. Zhang, X., Gao, Z., Jiao, L. and Zhou, H. "Multi-feature hyperspectral image classification with local and non-local spatial information via Markov Random Field in semantic space". IEEE Trans. On Geoscience and Remote Sensing, 56(3), 1409-1424, 2018. Zhang, X., Li, C., Zhang, J., Chen, Q., Feng, J., Jiao, L.C. and Zhou, H. "Hyperspectral unmixing via low-rank representation with space consistency constraint and spectral library pruning". Remote Sensing, 10(2), 339, 2018. Gao, F., Ma, F., Wang, J., Sun, J., Yang, E. and Zhou, H. "Visual saliency modeling for river detection in high-resolution SAR imagery", IEEE Access, Vol. 6, 1000-1014, 2018. Dong, X., Dong, J., Zhou, H., Sun, J. and Tao, D. "Automatic Chinese postal address block location using component-wise proximity descriptors and random forests". IEEE Trans. On Industrial Electronics, 65(5), 4401-4412, 2018. Zhang, S., Qi, Y., Jiang, F., Lan, X., Yuen, P.C. and Zhou, H. "Point-to-set distance metric learning on deep presentations for visual tracking". IEEE Trans. On Intelligent Transportation Systems, 19(1), 187-198, 2018. Li, D., Zhang, S., Zhou, H., Sun, X., Li, S. and Li, X. "Modelling information diffusion over social networks for temporal dynamic prediction". IEEE Trans. On Knowledge and Data Engineering, 29(9), 1985-1997, 2017. Zhang, S., Lan, X., Yao, H., Zhou, H., Tao, D. and Li, X. "A biologically inspired appearance model for robust visual tracking". IEEE Trans. On Neural Networks and Learning Systems, 28(10), 2357-2370, 2017. Santofimia, M.J., Martinez del Rincon, J., Hong, X., Zhou, H., Miller, P., Villa, D. and Lopez, J.C. "Hierarchical task network planning with common-sense reasoning for multiple-people behavior analysis". Expert Systems with Applications, 69, 118-134, 2017. Hong, X., Huang, Y., Ma, W., Varadarajan, S., Miller, P., Liu, W., Romero, M.J., del Rincon, J.M. and Zhou, H. "Evidential event inference in transport video surveillance". Computer Vision and Image Understanding, 144, C, 276-297, 2016. Ma, J., Liu, W., Miller, P. and Zhou, H. "An evidential fusion approach for gender profiling". Information Sciences, 333(3), 10-20, 2016.
Start Year 2012
 
Title Mouse behaviour recognition 
Description Automated recognition of mouse behaviours is crucial in studying psychiatric and neurologic diseases. To achieve this objective, it is very important to analyse temporal dynamics of mouse behaviours. Here, we release the code of our paper for mouse behavior recognition in a video. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact This is the first time that a software package was developed using Hidden Markov Models with deep learning for behaviour recognition. It is innovative in terms of academical contribution and will make a strong impact on animal behaviour analsysis. 
URL https://github.com/BIPL-UoL/CMBR
 
Description British Council Teaching Demo, Nov 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Undergraduate students
Results and Impact Dr. Huiyu Zhou was invited by British Council to give teaching demo to several Chinese universities, including Beijing Institute of Technology, Dalian University of Technology (Panjin), and Harbin University of Engineering/Harbin Institute of Technology. More than 200 under-and post-graduate students and professors from different disciplines attended the lectures that sparked questions and discussions afterwards. These universities reported increased interests in computer vision and animal behaviour analysis.
Year(s) Of Engagement Activity 2016
 
Description CECNet Keynote Speech 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact More than 80 people attended this keynote speech on healthcare technology and the audience highly commended the presentation and discussion. The organisers and the conference delegates invited the speaker to give an updated talk on the same conference in 2018.
Year(s) Of Engagement Activity 2018
URL http://www.cecnetconf.org/2017/KeynoteSpeakers.html
 
Description Chinese Doctoral Forum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact More than 50 people (academic staff, under- and post-graduates and professionals) attended the forum that provides an academic communication platform for scholars in the UK to share research results, promoting interdisciplinary professional communication, broadening academic horizons, stimulating innovative thinking and improving research ability. Dr. Huiyu Zhou gave a keynote speech of 30 minutes on the topic "Healthcare challenges and machine learning".
Year(s) Of Engagement Activity 2017
 
Description Institute visit, 24 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 50 under- and post-graduate students from Xi'an Institute of Optics and Precision Mechanics of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 24 July 2017, Xi'an, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
URL http://www.opt.ac.cn/xwzx/tzgg/201707/P020170720407110951835.pdf
 
Description Invited speech at International Conference on Biomedical Engineering and Biotechnology (ICBEB), Nanjing , China, 17-20 Oct. 2018. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Around 100 students, academic staff, medical doctors and industrial visitors attended the talk which sparked many questions and discussion afterwards. The audience reported increased interest in related subject areas.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Big Data Centre, Universiti Teknologi Malaysia (UTM), 3-7 December, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact 30+ researchers attended the invited talk, which sparked questions and discussion afterwards, and the university reported increased interest in related subject areas.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Nanjing University of Science and Technology, China, 18 October, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 30+ postgraduates and academic staff attended the talk, which sparked questions and discussion afterwards, and the audience reported increased interest in related subject areas.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Queen Mary, University of London, UK, 27 September, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 50+ researchers attended the research talk and had taken part in the question and answer session which is overwhelming. The audience reported increased interest in the medical image processing areas.
Year(s) Of Engagement Activity 2018
URL http://cis.eecs.qmul.ac.uk/seminars.html
 
Description Invited talk at Shanghai Jiaotong University, China, 15 October, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 10+ researchers attended the research talk, which sparked questions and discussion afterwards, and the audience reported increased interest in related subject areas.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at University of Exeter, December 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact Approximately 30 professors and research students attended the research talk given by Dr. Huiyu Zhou at University of Exeter. This talk, entitled "Mouse Behaviour Recognition", sparked questions and discussion afterwards and the University reported increased interest in image processing and behaviour analysis areas.
Year(s) Of Engagement Activity 2016
 
Description Invited talk at University of Kent, UK, 25 April, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 20 postgraduates and academic staff attended this talk and contributed to the discussion during and after the talk. The audience reported increased interest in related subject areas, e.g. machine learning in medical signal processing.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at University of Portsmouth, UK, 19-20 September, 2018. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 20+ postgraduate and academic staff attended the research talk, which sparked many questions and discussion afterwards and the audience reported increased interest in related artificial intelligence and machine learning areas.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at University of Ulster, UK, 24 Jan, 2019. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 40+ researchers attended the research talk, which sparked questions and discussion afterwards, and the school reported increased interest in related subject areas.
Year(s) Of Engagement Activity 2019
 
Description Medical conference attendance 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 250 professional practitioners took part in the knowledge sharing and comprehensive discussion on research and management of Parkinson's Disease. The conference attendees actively engaged the questions and answer session and the feedback from the people was very positive.
Year(s) Of Engagement Activity 2017
URL https://www.parkinsons.org.uk/professionals/events/scotland/parkinsons-disease-200-back-future-confe...
 
Description Research conference attendance, 15-16 July 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr H Zhou attended Medical Image Computing Seminar 2017, held in Shanghai of China, 15-16 July 2017. He communicated his recent research progress on animal mouse behaviour analysis for Parkinson's disease with the conference attendees, which sparked overwhelming questions and discussion. More than 10 different research groups from China and other countries raised major interest to this research study.
Year(s) Of Engagement Activity 2017
URL http://mic.sjtu.edu.cn/mics-2017/home
 
Description School visit (Bristol) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Approx. 10 researchers and students attended the seminar and discussed the progress in research.
Year(s) Of Engagement Activity 2018
 
Description University visit, 17 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 30 under- and post-graduate students from Shanghai University of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 17 July 2017, Shanghai, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 19 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 50 under- and post-graduate students from Ocean University of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 19 July 2017, Qingdao, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 22 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 35 under- and post-graduate students from China University of Petroleum attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 22 July 2017, Qingdao, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 24 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 50 under- and post-graduate students from Xi'an Jiaotong University of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 24 July 2017, Xi'an, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 25 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 50 under- and post-graduate students from Northwest Polytechnic University of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 25 July 2017, Xi'an, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 6 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 40 under- and post-graduate students from Jiangsu University of Science and Technology (JUST) of China attended Dr H Zhou's research talk, entitled "Healthcare challenges and machine learning", 6 July 2017, Zhengjiang, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visit, 7 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 20 under- and post-graduate students from Southeast University of China attended Dr H Zhou's research talk, entitled "Healthcare challenges and machine learning", 7 July 2017, Nanjing, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017
 
Description University visits, 14 July 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 60 under- and post-graduate students from Nanjing University of Aeronautics and Astronautics (NUAA) of China attended Dr H Zhou's research talk, entitled "Machine Learning for Healthcare Applications", 14 July 2017, Nanjing, China. This talk raised the awareness of the students in animal behaviour analysis and Parkinson's disease, and the students reported increased interest in these subject areas.
Year(s) Of Engagement Activity 2017