Robust Systems for Automated Analysis of Structures in 2D Medical Images
Lead Research Organisation:
University of Manchester
Department Name: School of Health Sciences
Abstract
This project aims to develop robust and accurate systems for locating the outlines of bones and other structures in widely used medical images such as radiographs. A key motivation is to provide a set of tools for the clinical research community to help analyse bone shape and thus better understand, monitor and treat musculoskeletal disease. Such diseases affect about 16% of all adults and more than 30% in the over 65s. The annual total cost of arthritis alone is estimated to be more than £30 billion for the UK. There is increasing clinical interest in studying bone shape, but progress is hampered by the time required to annotate the outlines of structures of interest on the large databases now available.
We have recently developed new machine-learning based methods for fitting statistical shape models to images which are achieving state-of-the-art performance. In this project we will build on this approach. Our goal is to provide the clinical research community with a system to allow analysis of large medical image databases as efficiently as possible. To achieve this we will address fundamental research challenges including (i) How to ensure model matching is robust and (ii) How to construct models from un-labelled images efficiently, with minimal human interaction. The project will produce both a practical and useful system for clinicians, and new algorithms and insights into tackling fundamental problems in computer vision and medical image analysis.
We have recently developed new machine-learning based methods for fitting statistical shape models to images which are achieving state-of-the-art performance. In this project we will build on this approach. Our goal is to provide the clinical research community with a system to allow analysis of large medical image databases as efficiently as possible. To achieve this we will address fundamental research challenges including (i) How to ensure model matching is robust and (ii) How to construct models from un-labelled images efficiently, with minimal human interaction. The project will produce both a practical and useful system for clinicians, and new algorithms and insights into tackling fundamental problems in computer vision and medical image analysis.
Planned Impact
This project will benefit the following groups:
Clinicians and Radiologists:
Clinicians will benefit from a better understanding of the range of variation of the human body and the changes caused by disease. The improved information provided by the models will lead to earlier and more accurate diagnosis and more effective choice of treatment and treatment monitoring.
We are working with a range of clinicians in the UK and abroad, who will be able to use the technology developed in the project.
Patients:
In the longer term this will lead to improvements to patients - their diseases will be better understood, diagnosed earlier and more accurately, and it will be easier to select the most appropriate treatment.
A better understanding of who is at risk of disease will lead to more targeted therapies and intervention, helping delay and prevent the worst effects.
Medical Image Analysis Companies:
There are many companies in the UK who develop products to help manage and analyse medical images who will benefit from the algorithms and technology developed in this project. We currently have close links with three local companies working in the field who could license the technology and help take it into wider use.
Pharmaceutical Companies:
Accurately detecting and monitoring the effect of new drugs on disease is of critical importance to pharmaceutical companies
during drug development. The methods developed will be directly applicable to this field, making it easier to study how bones and joints change over time during clinical trials.
Epidemiologists and Health Service Planners:
The project will provide tools to allow the analysis of large groups of images. These will produce valuable data for epidemiologists to understand variations in populations and diseases across populations.
This in turn will allow those in the health service to plan ahead and evaluate the cost and benefits of screening programs and interventions.
Clinicians and Radiologists:
Clinicians will benefit from a better understanding of the range of variation of the human body and the changes caused by disease. The improved information provided by the models will lead to earlier and more accurate diagnosis and more effective choice of treatment and treatment monitoring.
We are working with a range of clinicians in the UK and abroad, who will be able to use the technology developed in the project.
Patients:
In the longer term this will lead to improvements to patients - their diseases will be better understood, diagnosed earlier and more accurately, and it will be easier to select the most appropriate treatment.
A better understanding of who is at risk of disease will lead to more targeted therapies and intervention, helping delay and prevent the worst effects.
Medical Image Analysis Companies:
There are many companies in the UK who develop products to help manage and analyse medical images who will benefit from the algorithms and technology developed in this project. We currently have close links with three local companies working in the field who could license the technology and help take it into wider use.
Pharmaceutical Companies:
Accurately detecting and monitoring the effect of new drugs on disease is of critical importance to pharmaceutical companies
during drug development. The methods developed will be directly applicable to this field, making it easier to study how bones and joints change over time during clinical trials.
Epidemiologists and Health Service Planners:
The project will provide tools to allow the analysis of large groups of images. These will produce valuable data for epidemiologists to understand variations in populations and diseases across populations.
This in turn will allow those in the health service to plan ahead and evaluate the cost and benefits of screening programs and interventions.
Organisations
- University of Manchester (Lead Research Organisation)
- National Taiwan University (Collaboration)
- STOCKPORT NHS FOUNDATION TRUST (Collaboration)
- SRM Dental College (Collaboration)
- ALDER HEY CHILDREN'S NHS FOUNDATION TRUST (Collaboration)
- University Medical Center Utrecht (UMC) (Collaboration)
- University of Bristol (Collaboration)
Publications
Alagundagi DB
(2023)
Exploring breast cancer exosomes for novel biomarkers of potential diagnostic and prognostic importance.
in 3 Biotech
Gielis WP
(2020)
Predicting the mechanical hip-knee-ankle angle accurately from standard knee radiographs: a cross-validation experiment in 100 patients.
in Acta orthopaedica
Gielis WP
(2020)
An automated workflow based on hip shape improves personalized risk prediction for hip osteoarthritis in the CHECK study.
in Osteoarthritis and cartilage
Iwaya LH
(2023)
On the privacy of mental health apps: An empirical investigation and its implications for app development.
in Empirical software engineering
Lindner C
(2016)
Fully Automatic System for Accurate Localisation and Analysis of Cephalometric Landmarks in Lateral Cephalograms
in Scientific Reports
Lindner C
(2015)
Investigation of association between hip osteoarthritis susceptibility loci and radiographic proximal femur shape.
in Arthritis & rheumatology (Hoboken, N.J.)
Lindner C
(2021)
Author Correction: Fully Automatic System for Accurate Localisation and Analysis of Cephalometric Landmarks in Lateral Cephalograms.
in Scientific reports
Description | We have developed software which can accurately and reliably find the outlines of bones in medical images such as X-rays. Given sets of images and associated points (placed by an expert), we can build models which can locate equivalent points automatically on new images. |
Exploitation Route | The software has already been licensed to over 180 different groups (mostly medical) in 35 countries for research. They are using it to make measurements of bone shape in order to look for links between bone shape and signs of disease or to monitor changes in bone shape in response to treatment. Evaluation licences have been signed with five companies. A commercial licence has been signed with one company, who is using the system to help locate landmark points in software for orthodontists. The resulting system has been sold worldwide. We anticipate that the software will be used more widely, and will eventually be adopted by several medical image analysis companies. |
Sectors | Healthcare |
URL | http://www.bone-finder.com |
Description | In the biomedical research community, BoneFinder® is recognised as the leading technology to automatically outline skeletal structures in radiographs. BoneFinder® is freely available (via UMIP) for research purposes. As of October 2020, over 180 research licences had been distributed to over 120 research groups in 40 countries. In dentistry, BoneFinder® has been commercialised to aid and facilitate the diagnosis/assessment of dentoskeletal malformations. As of December 2020, over 930 licences have been issued to orthodontists from 80 countries. |
Sector | Education,Healthcare |
Impact Types | Economic |
Description | Amgen |
Amount | £11,747 (GBP) |
Organisation | Amgen Inc |
Sector | Private |
Country | United States |
Start | 08/2015 |
End | 01/2016 |
Description | An automated system for monitoring developmental dysplasia of the hip in children. Wellcome Trust iTPA Projects for Translation Award |
Amount | £36,000 (GBP) |
Organisation | University of Manchester |
Sector | Academic/University |
Country | United Kingdom |
Start | 03/2021 |
End | 10/2021 |
Description | Collaborative Grant |
Amount | £1,600,000 (GBP) |
Funding ID | 209233/Z/17/Z |
Organisation | Wellcome Trust |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 08/2018 |
End | 08/2022 |
Description | ORCHID STUDY - Outcome Research in Children's HIp Disease |
Amount | £277,643 (GBP) |
Funding ID | 21356 |
Organisation | Versus Arthritis |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 03/2017 |
End | 02/2020 |
Description | Wellcome Trust iTPA Access to Expertise: Establishing a data sharing platform... |
Amount | £25,000 (GBP) |
Organisation | University of Manchester |
Sector | Academic/University |
Country | United Kingdom |
Start | 09/2020 |
End | 05/2021 |
Description | AUGMENT study (Universities of Bristol, Southampton, Aberdeen, Cardiff and Queensland) |
Organisation | University of Bristol |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We are developing methods and software to automatically analyse the hips and knees from DXA images. |
Collaborator Contribution | Partners provide data and clinical expertise/skills, and study the role of bone and joint size and shape in explaining common musculoskeletal diseases. |
Impact | This collaboration has led to a Wellcome Trust Collaborative Award. Three peer-reviewed journal papers (Rheumatology, Osteoarthritis and Cartilage, Bone) and ten peer-reviewed abstract presentations (British Society of Rheumatology Annual Conference: 2022; Osteoarthritis Research Society International World Congress on Osteoarthritis: OARSI-2021, OARSI-2022; American College of Rheumatology Annual Meeting: ACR-Convergence-2021; American Society of Bone and Mineral Research Annual Meeting: ASBMR-2021; Bone Research Society Annual Meeting: BRS-2020) have been published so far. |
Start Year | 2017 |
Description | Automated cephalometric evaluation using BoneFinder |
Organisation | National Taiwan University |
Country | Taiwan, Province of China |
Sector | Academic/University |
PI Contribution | Generation of automatic evaluation software and experimental analyses for accurately locating landmark points on cepalograms. |
Collaborator Contribution | Provision of medical images and manually positioned cephalometric landmarks, and clinical expertise on cephalometric evaluation |
Impact | Journal paper published. Wang et al., "A benchmark for comparison of dental radiography analysis algorithms", Medical Image Analysis 2016, Vol.31, pp.63-76 |
Start Year | 2015 |
Description | Substudy-collaboration with Alder Hey Children's NHS Foundation Trust (AHFT) |
Organisation | Alder Hey Children's NHS Foundation Trust |
Country | United Kingdom |
Sector | Public |
PI Contribution | We are developing methods and software to automatically assess the hips of children from radiographic images and related clinical data. We are publishing results at conferences and in journals. |
Collaborator Contribution | AHFT provides radiographic images and related clinical data as well as clinical expertise. AHFT sets up the technical infrastructure for facilitated data sharing between the University and the Trust. AHFT contributes to all publications. |
Impact | This is a multi-disciplinary collaboration (computer science, radiology, pediatric orthopaedic surgery) that has so far led to three grants as well as three conference publications. Our presentation at the Bone Research Society Annual Meeting 2020 was awarded the Best Clinical Poster Prize. As part of this collaboration we are currently establishing a data sharing platform between the University and the Trust which will be of benefit to any future research studies. |
Start Year | 2016 |
Description | Substudy-collaboration with SRM Dental College & Hospital Chennai |
Organisation | SRM Dental College |
Country | India |
Sector | Academic/University |
PI Contribution | We have developed a software system to evaluate the variation in morphology of the TMJ condyles in orthopantomogram (OPG) images. |
Collaborator Contribution | SRM provided the OPG images and the manual annotations of the outline of the TMJ condyles. The images and annotations were used to develop the system. SRM was leading on writing the journal paper. |
Impact | This is a multi-disciplinary collaboration (computer science, dentistry) that has resulted in one journal paper. |
Start Year | 2015 |
Description | Substudy-collaboration with Stockport NHS Foundation Trust (SFT) |
Organisation | Stockport NHS Foundation Trust |
Department | Stepping Hill Hospital |
Country | United Kingdom |
Sector | Hospitals |
PI Contribution | We are developing methods and software to automatically assess pre-/post-operative knee radiographs and related clinical data (including patient reported outcomes) of patients that underwent knee replacement surgery. |
Collaborator Contribution | SFT is providing the data as well as clinical expertise. |
Impact | This is a multi-disciplinary collaboration (computer science, orthopaedic surgery). We have so far collected over 18000 radiographic images of patients that underwent knee replacement surgery. |
Start Year | 2018 |
Description | UMCU - Bonefinder |
Organisation | University Medical Center Utrecht (UMC) |
Country | Netherlands |
Sector | Academic/University |
PI Contribution | We are building models to automatically locate the outlines of the bones in the hip and the knee in radiographs, and provide the models and software tools. We contribute to all publications. |
Collaborator Contribution | UMCU provides radiographs of the knee and the hip as well as manual bone outlines. UMCU uses the outlines for association studies and publishes the results at conferences and in journals. |
Impact | Three abstracts and two journal papers have been published so far. |
Start Year | 2014 |
Title | Bonefinder system |
Description | An automatic system for locating the outlines of various bones in medical images such as radiographs. It uses statistical shape models and machine learning techniques to create models for each type of bone, which can then be used to locate the outline in new images. Models exist for hips (both adult and child), knees, hands and cephalometric landmarks on the face. |
IP Reference | |
Protection | Patent granted |
Year Protection Granted | 2014 |
Licensed | Yes |
Impact | Licensed to over 190 research groups in over 35 countries across the world. Licenced for non-commercial evaluation to 15 companies. Commercial license to one company for cephalometric landmark localisation - currently part of a software system being sold globally. By the end of 2020, the system had been licenced to over 900 orthodontists from more than 40 countries. |
Title | Bonefinder |
Description | A software tool for automatically locating the outline of certain bones in radiographs, allowing accurate measurement of shape. |
Type Of Technology | Software |
Year Produced | 2015 |
Impact | The tool has been made available for research, and has been licensed to over 100 research groups and 15 companies. This has led to about 10 different collaborations with research groups across the world. When applied to the problem of locating landmarks used by orthodontists on cephelograms (radiographs of the skull), it won an international competition for accuracy. This has led to discussions with various companies about licensing the technology for inclusion in dental software suites. It has been licensed to one commercial company, who has integrated the system into their dental software package and is selling it globally. |
URL | http://www.bone-finder.com |
Description | Invited talk, University of Bath, Tim Cootes 2019 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Postgraduate students |
Results and Impact | Invited talk at the University of Bath. Gave an overview of our work on locating points on medical images and described several projects including ORCHID and STOpFrac |
Year(s) Of Engagement Activity | 2019 |
Description | Public talk by T.F. Cootes: Pint of Science |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Public/other audiences |
Results and Impact | Talk in local pub as part of "Pint of Science" event. Title: "Why we look at bones" - an overview of work we do analyzing musculoskeletal images. Approx 40 people attended. |
Year(s) Of Engagement Activity | 2018 |
Description | STEM Exhibition |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Schools |
Results and Impact | In my role as STEM Ambassador, I have had a BoneFinder stall at the STEM Exhibition of Levenshulme High School (Manchester) to raise awareness of STEM subjects and to promote STEM careers to the students of this all girls school. |
Year(s) Of Engagement Activity | 2018,2019 |