Robust Systems for Automated Analysis of Structures in 2D Medical Images

Lead Research Organisation: University of Manchester
Department Name: School of Health Sciences

Abstract

This project aims to develop robust and accurate systems for locating the outlines of bones and other structures in widely used medical images such as radiographs. A key motivation is to provide a set of tools for the clinical research community to help analyse bone shape and thus better understand, monitor and treat musculoskeletal disease. Such diseases affect about 16% of all adults and more than 30% in the over 65s. The annual total cost of arthritis alone is estimated to be more than £30 billion for the UK. There is increasing clinical interest in studying bone shape, but progress is hampered by the time required to annotate the outlines of structures of interest on the large databases now available.

We have recently developed new machine-learning based methods for fitting statistical shape models to images which are achieving state-of-the-art performance. In this project we will build on this approach. Our goal is to provide the clinical research community with a system to allow analysis of large medical image databases as efficiently as possible. To achieve this we will address fundamental research challenges including (i) How to ensure model matching is robust and (ii) How to construct models from un-labelled images efficiently, with minimal human interaction. The project will produce both a practical and useful system for clinicians, and new algorithms and insights into tackling fundamental problems in computer vision and medical image analysis.

Planned Impact

This project will benefit the following groups:

Clinicians and Radiologists:
Clinicians will benefit from a better understanding of the range of variation of the human body and the changes caused by disease. The improved information provided by the models will lead to earlier and more accurate diagnosis and more effective choice of treatment and treatment monitoring.
We are working with a range of clinicians in the UK and abroad, who will be able to use the technology developed in the project.

Patients:
In the longer term this will lead to improvements to patients - their diseases will be better understood, diagnosed earlier and more accurately, and it will be easier to select the most appropriate treatment.
A better understanding of who is at risk of disease will lead to more targeted therapies and intervention, helping delay and prevent the worst effects.

Medical Image Analysis Companies:
There are many companies in the UK who develop products to help manage and analyse medical images who will benefit from the algorithms and technology developed in this project. We currently have close links with three local companies working in the field who could license the technology and help take it into wider use.

Pharmaceutical Companies:
Accurately detecting and monitoring the effect of new drugs on disease is of critical importance to pharmaceutical companies
during drug development. The methods developed will be directly applicable to this field, making it easier to study how bones and joints change over time during clinical trials.

Epidemiologists and Health Service Planners:
The project will provide tools to allow the analysis of large groups of images. These will produce valuable data for epidemiologists to understand variations in populations and diseases across populations.
This in turn will allow those in the health service to plan ahead and evaluate the cost and benefits of screening programs and interventions.

Publications

10 25 50

 
Description We have developed software which can accurately and reliably find the outlines of bones in medical images such as X-rays. Given sets of images and associated points (placed by an expert), we can build models which can locate equivalent points automatically on new images.
Exploitation Route The software has already been licensed to over 180 different groups (mostly medical) in 35 countries for research. They are using it to make measurements of bone shape in order to look for links between bone shape and signs of disease or to monitor changes in bone shape in response to treatment.
Evaluation licences have been signed with five companies.
A commercial licence has been signed with one company, who is using the system to help locate landmark points in software for orthodontists. The resulting system has been sold worldwide.
We anticipate that the software will be used more widely, and will eventually be adopted by several medical image analysis companies.
Sectors Healthcare

URL http://www.bone-finder.com
 
Description In the biomedical research community, BoneFinder® is recognised as the leading technology to automatically outline skeletal structures in radiographs. BoneFinder® is freely available (via UMIP) for research purposes. As of October 2020, over 180 research licences had been distributed to over 120 research groups in 40 countries. In dentistry, BoneFinder® has been commercialised to aid and facilitate the diagnosis/assessment of dentoskeletal malformations. As of December 2020, over 930 licences have been issued to orthodontists from 80 countries.
Sector Education,Healthcare
Impact Types Economic

 
Description Amgen
Amount £11,747 (GBP)
Organisation Amgen Inc 
Sector Private
Country United States
Start 09/2015 
End 01/2016
 
Description An automated system for monitoring developmental dysplasia of the hip in children. Wellcome Trust iTPA Projects for Translation Award
Amount £36,000 (GBP)
Organisation University of Manchester 
Sector Academic/University
Country United Kingdom
Start 03/2021 
End 10/2021
 
Description Collaborative Grant
Amount £1,600,000 (GBP)
Funding ID 209233/Z/17/Z 
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2018 
End 08/2022
 
Description ORCHID STUDY - Outcome Research in Children's HIp Disease
Amount £277,643 (GBP)
Funding ID 21356 
Organisation Versus Arthritis 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2017 
End 02/2020
 
Description Wellcome Trust iTPA Access to Expertise: Establishing a data sharing platform...
Amount £25,000 (GBP)
Organisation University of Manchester 
Sector Academic/University
Country United Kingdom
Start 10/2020 
End 05/2021
 
Description AUGMENT study (Universities of Bristol, Southampton, Aberdeen, Cardiff and Queensland) 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution We are developing methods and software to automatically analyse the hips and knees from DXA images.
Collaborator Contribution Partners provide data and clinical expertise/skills, and study the role of bone and joint size and shape in explaining common musculoskeletal diseases.
Impact This collaboration has led to a Wellcome Trust Collaborative Award. Three peer-reviewed journal papers (Rheumatology, Osteoarthritis and Cartilage, Bone) and ten peer-reviewed abstract presentations (British Society of Rheumatology Annual Conference: 2022; Osteoarthritis Research Society International World Congress on Osteoarthritis: OARSI-2021, OARSI-2022; American College of Rheumatology Annual Meeting: ACR-Convergence-2021; American Society of Bone and Mineral Research Annual Meeting: ASBMR-2021; Bone Research Society Annual Meeting: BRS-2020) have been published so far.
Start Year 2017
 
Description Automated cephalometric evaluation using BoneFinder 
Organisation National Taiwan University
Country Taiwan, Province of China 
Sector Academic/University 
PI Contribution Generation of automatic evaluation software and experimental analyses for accurately locating landmark points on cepalograms.
Collaborator Contribution Provision of medical images and manually positioned cephalometric landmarks, and clinical expertise on cephalometric evaluation
Impact Journal paper published. Wang et al., "A benchmark for comparison of dental radiography analysis algorithms", Medical Image Analysis 2016, Vol.31, pp.63-76
Start Year 2015
 
Description Substudy-collaboration with Alder Hey Children's NHS Foundation Trust (AHFT) 
Organisation Alder Hey Children's NHS Foundation Trust
Country United Kingdom 
Sector Public 
PI Contribution We are developing methods and software to automatically assess the hips of children from radiographic images and related clinical data. We are publishing results at conferences and in journals.
Collaborator Contribution AHFT provides radiographic images and related clinical data as well as clinical expertise. AHFT sets up the technical infrastructure for facilitated data sharing between the University and the Trust. AHFT contributes to all publications.
Impact This is a multi-disciplinary collaboration (computer science, radiology, pediatric orthopaedic surgery) that has so far led to three grants as well as three conference publications. Our presentation at the Bone Research Society Annual Meeting 2020 was awarded the Best Clinical Poster Prize. As part of this collaboration we are currently establishing a data sharing platform between the University and the Trust which will be of benefit to any future research studies.
Start Year 2016
 
Description Substudy-collaboration with SRM Dental College & Hospital Chennai 
Organisation SRM Dental College
Country India 
Sector Academic/University 
PI Contribution We have developed a software system to evaluate the variation in morphology of the TMJ condyles in orthopantomogram (OPG) images.
Collaborator Contribution SRM provided the OPG images and the manual annotations of the outline of the TMJ condyles. The images and annotations were used to develop the system. SRM was leading on writing the journal paper.
Impact This is a multi-disciplinary collaboration (computer science, dentistry) that has resulted in one journal paper.
Start Year 2015
 
Description Substudy-collaboration with Stockport NHS Foundation Trust (SFT) 
Organisation Stockport NHS Foundation Trust
Department Stepping Hill Hospital
Country United Kingdom 
Sector Hospitals 
PI Contribution We are developing methods and software to automatically assess pre-/post-operative knee radiographs and related clinical data (including patient reported outcomes) of patients that underwent knee replacement surgery.
Collaborator Contribution SFT is providing the data as well as clinical expertise.
Impact This is a multi-disciplinary collaboration (computer science, orthopaedic surgery). We have so far collected over 18000 radiographic images of patients that underwent knee replacement surgery.
Start Year 2018
 
Description UMCU - Bonefinder 
Organisation University Medical Center Utrecht (UMC)
Country Netherlands 
Sector Academic/University 
PI Contribution We are building models to automatically locate the outlines of the bones in the hip and the knee in radiographs, and provide the models and software tools. We contribute to all publications.
Collaborator Contribution UMCU provides radiographs of the knee and the hip as well as manual bone outlines. UMCU uses the outlines for association studies and publishes the results at conferences and in journals.
Impact Three abstracts and two journal papers have been published so far.
Start Year 2014
 
Title Bonefinder system 
Description An automatic system for locating the outlines of various bones in medical images such as radiographs. It uses statistical shape models and machine learning techniques to create models for each type of bone, which can then be used to locate the outline in new images. Models exist for hips (both adult and child), knees, hands and cephalometric landmarks on the face. 
IP Reference  
Protection Patent granted
Year Protection Granted 2014
Licensed Yes
Impact Licensed to over 190 research groups in over 35 countries across the world. Licenced for non-commercial evaluation to 15 companies. Commercial license to one company for cephalometric landmark localisation - currently part of a software system being sold globally. By the end of 2020, the system had been licenced to over 900 orthodontists from more than 40 countries.
 
Title Bonefinder 
Description A software tool for automatically locating the outline of certain bones in radiographs, allowing accurate measurement of shape. 
Type Of Technology Software 
Year Produced 2015 
Impact The tool has been made available for research, and has been licensed to over 100 research groups and 15 companies. This has led to about 10 different collaborations with research groups across the world. When applied to the problem of locating landmarks used by orthodontists on cephelograms (radiographs of the skull), it won an international competition for accuracy. This has led to discussions with various companies about licensing the technology for inclusion in dental software suites. It has been licensed to one commercial company, who has integrated the system into their dental software package and is selling it globally. 
URL http://www.bone-finder.com
 
Description Invited talk, University of Bath, Tim Cootes 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Invited talk at the University of Bath.
Gave an overview of our work on locating points on medical images and described several projects including ORCHID and STOpFrac
Year(s) Of Engagement Activity 2019
 
Description Public talk by T.F. Cootes: Pint of Science 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Talk in local pub as part of "Pint of Science" event.
Title: "Why we look at bones" - an overview of work we do analyzing musculoskeletal images.
Approx 40 people attended.
Year(s) Of Engagement Activity 2018
 
Description STEM Exhibition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact In my role as STEM Ambassador, I have had a BoneFinder stall at the STEM Exhibition of Levenshulme High School (Manchester) to raise awareness of STEM subjects and to promote STEM careers to the students of this all girls school.
Year(s) Of Engagement Activity 2018,2019