Automating quality assessment of B-mode ultrasound scans (AQABUS)
Lead Research Organisation:
University of Oxford
Department Name: Engineering Science
Abstract
Diagnostic quality of ultrasound (US) scans of the same patient can vary significantly, depending on the level of skill and
expertise of the sonographer and the equipment used. This well documented operator dependence of quality can lead to
misdiagnosis; unnecessary referrals to other imaging modalities; rescans; and results in inefficiencies in clinical workflow
with associated added healthcare costs.
In order to avoid the resultant delayed or incorrect stratification, sonographers are encouraged to follow standardised
acquisition protocol guidelines. Clinical studies show the benefit of this. However, no systematic image reviews are
undertaken. This lack of standardization and monitoring of conformance has implications for the quality of diagnosis and
clinical decisions made in a health service, and is also a key barrier to wider adoption of US as a cost-effective technology
for stratified medicine.
In a collaboration between Intelligent Ultrasound Ltd and Oxford University, the AQABUS project proposes to develop and
evaluate a technology platform for automated quality assessment of US scans. The solution will be delivered as a Quality
Assurance Service (QAS) to imaging departments, to assess whether scans are fit for purpose. Innovations include
creation of the IT infrastructure for delivering such a service; clinical design of QA reporting; and machine learning to
automate image QA which is key to the business proposition.
expertise of the sonographer and the equipment used. This well documented operator dependence of quality can lead to
misdiagnosis; unnecessary referrals to other imaging modalities; rescans; and results in inefficiencies in clinical workflow
with associated added healthcare costs.
In order to avoid the resultant delayed or incorrect stratification, sonographers are encouraged to follow standardised
acquisition protocol guidelines. Clinical studies show the benefit of this. However, no systematic image reviews are
undertaken. This lack of standardization and monitoring of conformance has implications for the quality of diagnosis and
clinical decisions made in a health service, and is also a key barrier to wider adoption of US as a cost-effective technology
for stratified medicine.
In a collaboration between Intelligent Ultrasound Ltd and Oxford University, the AQABUS project proposes to develop and
evaluate a technology platform for automated quality assessment of US scans. The solution will be delivered as a Quality
Assurance Service (QAS) to imaging departments, to assess whether scans are fit for purpose. Innovations include
creation of the IT infrastructure for delivering such a service; clinical design of QA reporting; and machine learning to
automate image QA which is key to the business proposition.
Planned Impact
Beyond academic engineering and clinical medicine researchers, beneficiaries of the research are:
1. Clinicians - feedback on scan quality will help imaging leads to identify sonographers who may need remedial training,
and ensure scanning is of a consistently high standard. Monitoring should reduce the number of recalls, lead to fewer
missed diagnoses, and hence reduce costs and increase department efficiency. Obstetrics is the first clinical area that will
benefit (the area chosen for this project) but if successful, other areas of medicine may adopt similar practices.
2. NHS/hospital managers - feedback on imaging department performance will ensure hospital imaging services are of the
highest level, identify areas of improvement, and reduce the likely of litigation due to "wrongful birth" (estimated to cost the
NHS around £60 million annually).
3. Patients - monitoring of scanning quality may reduce the number of recalls and lead to fewer missed diagnoses, thus
improving patient management and reducing pregnancy risk.
4. Industry - Intelligent Ultrasound will be the immediate commercial beneficiary of the outputs of this research and use it to
inform new products. Product sales will allow the company to grow and creates jobs. Ensuring ultrasound imaging is of
consistently high quality has been a barrier to introduction of ultrasound imaging in some areas of medicine. Ultrasound
equipment manufacturers (eg. Philips, GE, Siemens, Toshiba etc), may benefit by increased sales of equipment if quality
issues can be overcome by using such a QA service.
5. Beneficiaries are not restricted to the UK health service, and the outputs of this project has the potential to impact
ultrasound-based healthcare delivery worldwide, including in the developing world where ultrasound is one of a few lowcost
imaging solutions.
The timescale of demonstrated impact are likely to be on the short to medium scale, post project end. Short term impact
will be demonstrated in obstetrics. Medium term impacts are likely to be demonstration in other areas of medicine, and
eventually through adoption of the QA approach as a NICE guideline or equivalent.
1. Clinicians - feedback on scan quality will help imaging leads to identify sonographers who may need remedial training,
and ensure scanning is of a consistently high standard. Monitoring should reduce the number of recalls, lead to fewer
missed diagnoses, and hence reduce costs and increase department efficiency. Obstetrics is the first clinical area that will
benefit (the area chosen for this project) but if successful, other areas of medicine may adopt similar practices.
2. NHS/hospital managers - feedback on imaging department performance will ensure hospital imaging services are of the
highest level, identify areas of improvement, and reduce the likely of litigation due to "wrongful birth" (estimated to cost the
NHS around £60 million annually).
3. Patients - monitoring of scanning quality may reduce the number of recalls and lead to fewer missed diagnoses, thus
improving patient management and reducing pregnancy risk.
4. Industry - Intelligent Ultrasound will be the immediate commercial beneficiary of the outputs of this research and use it to
inform new products. Product sales will allow the company to grow and creates jobs. Ensuring ultrasound imaging is of
consistently high quality has been a barrier to introduction of ultrasound imaging in some areas of medicine. Ultrasound
equipment manufacturers (eg. Philips, GE, Siemens, Toshiba etc), may benefit by increased sales of equipment if quality
issues can be overcome by using such a QA service.
5. Beneficiaries are not restricted to the UK health service, and the outputs of this project has the potential to impact
ultrasound-based healthcare delivery worldwide, including in the developing world where ultrasound is one of a few lowcost
imaging solutions.
The timescale of demonstrated impact are likely to be on the short to medium scale, post project end. Short term impact
will be demonstrated in obstetrics. Medium term impacts are likely to be demonstration in other areas of medicine, and
eventually through adoption of the QA approach as a NICE guideline or equivalent.
Publications
Alison Noble J
(2016)
Reflections on ultrasound image analysis.
in Medical image analysis
Coffey S
(2017)
Protocol and quality assurance for carotid imaging in 100,000 participants of UK Biobank: development and assessment.
in European journal of preventive cardiology
Maraci MA
(2020)
Toward point-of-care ultrasound estimation of fetal gestational age from the trans-cerebellar diameter using CNN-based ultrasound image analysis.
in Journal of medical imaging (Bellingham, Wash.)
Namburete AIL
(2018)
Fully-automated alignment of 3D fetal brain ultrasound to a canonical reference space using multi-task learning.
in Medical image analysis
Papageorghiou A
(2023)
EP02.46: An AI system (SonoLyst) achieves expert level performance when categorising images for adherence to ISUOG mid-trimester screening guidelines
in Ultrasound in Obstetrics & Gynecology
Ryou H
(2019)
Automated 3D ultrasound image analysis for first trimester assessment of fetal health.
in Physics in medicine and biology
Xu Y
(2020)
Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation.
in Journal of medical imaging (Bellingham, Wash.)
Yaqub M
(2019)
OC23.05: Effect of maternal BMI on ultrasound image quality at the routine anatomy scan
in Ultrasound in Obstetrics & Gynecology
Yaqub M
(2021)
491 ScanNav® audit: an AI-powered screening assistant for fetal anatomical ultrasound
in American Journal of Obstetrics and Gynecology
Yaqub M
(2017)
A Deep Learning Solution for Automatic Fetal Neurosonographic Diagnostic Plane Verification Using Clinical Standard Constraints.
in Ultrasound in medicine & biology
Yaqub M
(2019)
Quality-improvement program for ultrasound-based fetal anatomy screening using large-scale clinical audit.
in Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology
Description | Automated methods were developed and validated to detect and grade obstetric ultrasound images using machine learning methods. Methods were validated on a large hospital database of images and agreement with experts assessed. Methods have been licensed to the business partner to be taken forward towards commercialization. This is a "world first" in this area addressing an important issue in medical imaging of how to ensure images are "fit for purpose". The business partner has subsequently significantly further developed the algorithms into medically-approved software for checking that quality of obstetric ultrasound images for the anomaly (second trimester) scan which was released in a new ultrasound machine in October 2021. A side project looked at, through an EPSRC Impact Acceleration Award, how related ideas might be adapted to improve manual quality assessment of UK Biobank carotid ultrasound data. This led to an academic publication. |
Exploitation Route | The business partner took this work forward for commercialization. We have also learned a lot about real world sonography in hospitals and how to assess sonographer performance which has informed our current academic research. We have discussed our findings with international clinical experts in the field. We are now looking into other related problems to advance uses of ultrasound in clinical practice. This has included securing NIHR Biomedical Research Centre funding for a further applied project looking at the third trimester scan, and NIHR funding investigating the economic case for adapting the second trimester anomaly scan screening to the first trimester. |
Sectors | Healthcare |
Description | Methods were licensed to the business partner to be taken forward towards commercialization. The business partner went on to significantly improve on the methods and original research ideas which were developed into a CE marked device (ScanNav) which underwent evaluation in two UK hospitals in 2019. Subsequently the software was embedded in a third-party ultrasound machine and is being sold commercially as of September 2020. |
First Year Of Impact | 2017 |
Sector | Healthcare |
Impact Types | Societal Economic |
Description | ERC Advanced Grant |
Amount | € 2,500,000 (EUR) |
Organisation | European Research Council (ERC) |
Sector | Public |
Country | Belgium |
Start | 11/2016 |
End | 10/2021 |
Description | GCRF: Growing Research Capability Call |
Amount | £8,000,000 (GBP) |
Funding ID | MR/P027938/1 |
Organisation | Medical Research Council (MRC) |
Sector | Public |
Country | United Kingdom |
Start | 09/2017 |
End | 09/2021 |
Description | Grand Challenge Exploration Phase 1 Round 14 |
Amount | $100,000 (USD) |
Funding ID | OPP1128941 |
Organisation | Bill and Melinda Gates Foundation |
Sector | Charity/Non Profit |
Country | United States |
Start | 11/2015 |
End | 04/2017 |
Description | Oxford EPSRC Impact Acceleration Award (Technology award: AQABUS extension to UK Biobank QA assessment) |
Amount | £70,000 (GBP) |
Funding ID | EP/K503769/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 07/2015 |
End | 07/2016 |
Description | Oxford EPSRC Impact Acceleration Award (Technology award: Six-steps) |
Amount | £70,000 (GBP) |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 01/2015 |
End | 03/2016 |
Description | Oxford NIHR Biomedical Research Centre |
Amount | £250,000 (GBP) |
Organisation | National Institute for Health Research |
Department | NIHR Biomedical Research Centre |
Sector | Public |
Country | United Kingdom |
Start | 03/2017 |
End | 03/2022 |
Description | CCRF University of Oxford |
Organisation | University of Oxford |
Department | Oxford Cardiovascular Clinical Research Facility (CCRF) |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | A collaboration funded by the Oxford Impact Acceleration Award fund has explored the feasibility of using the software auditing system developed for this project on echocardiography and carotid ultrasound data. Specifically the IAA project funded a clinical research fellow to manually audit data and the viability of automating analysis in the future. |
Collaborator Contribution | The partner performed manual analysis and contributed to discussions on how analysis might be automated. Data came from CCRF clinical research trials and pilot data from the UK Biobank carotid ultrasound study. |
Impact | The output was the report on findings which suggest directions in which the work might be taken forward. Options are being explored that might fund future work. |
Start Year | 2015 |
Description | Intelligent Ultrasound Ltd |
Organisation | Intelligent Ultrasound Ltd |
Country | United Kingdom |
Sector | Private |
PI Contribution | Partner on EPSRC/Innovate UK SMI Project. |
Collaborator Contribution | Licence agreement to use IDF software for research purposes. Licence to use AQABUS software for research purposes. IU is an medical image analysis SME and is also part of the MedIAN Network. |
Impact | See separate entry |
Start Year | 2013 |
Description | Oxford University Hospitals NHS Foundation Trust - Nuffield Department of Women's and Reproductive Health |
Organisation | Oxford University Hospitals NHS Foundation Trust |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Engineering research expertise, intellectual input. |
Collaborator Contribution | Access to real-world data (data acquired in clinic), knowledge of clinical sonography. |
Impact | Multiple research grants, trained personnel (clinical and engineering), supported also by academic outputs in engineering and medicine. WRH is the academic department associated with the hospital's Women's Centre. This is an inter-disciplinary collaboration between engineering and medicine. The collaboration has also led to a spin-out (Noble and Papageorghiou are co-founders of Intelligent Ultrasound Ltd founded in 2012). |
Start Year | 2008 |
Title | Aqabus software |
Description | Automated image analysis for describing ultrasound scans. |
IP Reference | |
Protection | Copyrighted (e.g. software) |
Year Protection Granted | 2016 |
Licensed | Commercial In Confidence |
Impact | Licensing the impact. |
Title | System and methods for structures detection and multi-class image categorization in medical imaging |
Description | A system and method are provided to automatically categorize biological and medical images. The new system and method can incorporate a machine learning classifier in which novel ideas are provided to guide the classifier to focus on regions of interest (ROI) within medical images for categorizing or classifying the images. The system and method can ignore regions when misleading structures exist. The detection and classification of one or more features of interest within a discriminative region of interest within an image are rendered invariant to differences in translation, orientation and/or scaling of the one or more features of interest within the medical image (s). The system and method allow a processor to more quickly, efficiently and accurately process and categorize medical images. |
IP Reference | US patent 15744371 |
Protection | Patent granted |
Year Protection Granted | 2020 |
Licensed | Commercial In Confidence |
Impact | Used to influence design of versions of a prototype product.. |
Company Name | Intelligent Ultrasound |
Description | Intelligent Ultrasound, a spin-out of Oxford University, develops software that aims to improve the quality, reliability and diagnostic power of a medical ultrasound scan. |
Year Established | 2012 |
Impact | Founded by Professor Alison Noble, Dr Aris Papageorghiou and Mr Andy Hill in 2012. Founding investment included funds from the NHS. First product launched 2014. Winner OBN Best Emerging Medtech Company Award 2014. Intelligent Ultrasound Ltd was bought by MedaPhor Group PLC in 2017. IU's AI-based ScanNav product was CE marked in 2017 and the company announced the first evaluation of ScaNav in a UK hospital (St George's London) in February 2018. IU held a InnovateUK project in partnership with Oxford University from 2014-16 (AQABUS). |
Website | http://www.intelligentultrasound.com |
Description | Inspirational Engineer Talk - University of Cambridge |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Invited talk on my career and research given in at in invited lecture at the Department of Engineering, University of Cambridge. |
Year(s) Of Engagement Activity | 2019 |
Description | Invited talk at CVPR 2017 workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Gave an invited talk as part of a workshop at CVPR 2017 (Hawaii) which aimed to give an overview to computer vision researchers or problems and state-of-the-art research in medical image analysis. A little follow-up but discussion was quite passive (we might have been competing with the weather on the last day of the meeting!) |
Year(s) Of Engagement Activity | 2017 |
Description | Invited talk at CVPR 2019 workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk given at CVPR 2019 workshop |
Year(s) Of Engagement Activity | 2019 |
Description | Invited talk at International Ultrasonics Symposium |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited key note talk in the inaugural session on machine learning in ultrasonics. The session was packed reflecting the interest in not only my group's work but the interest in machine learning as well. |
Year(s) Of Engagement Activity | 2017 |
Description | MISS Summer School |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Invited lecturer at international summer school. |
Year(s) Of Engagement Activity | 2016 |
Description | Microsoft Postgraduate Summer School |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Invited talk at Microsoft Summer school |
Year(s) Of Engagement Activity | 2016 |
Description | NIHR Point of Care Ultrasound workshop (Birmingham) |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | I gave a talk at a clinical workshop looking at how point of care US might be introduced outside of traditional hospital uses. Meeting aimed to educate clinical practioners working in primary care to think about how US might be used in the future and encourage them to consider being part of trials of new innovations. |
Year(s) Of Engagement Activity | 2019 |
Description | OxFest (Oxford) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Postgraduate students |
Results and Impact | Careers talk as part of OxFest which included presentation of outputs of this research. |
Year(s) Of Engagement Activity | 2016 |
Description | PRAIRIE Summer School talk (Paris) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Gave a talk at the postgraduate summer school attended largely by computer science students interested in AI and not with a healthcare focus with questions afterwards. |
Year(s) Of Engagement Activity | 2019 |
Description | Placenta workshop (New Delhi, India) |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Study participants or study members |
Results and Impact | The international workshop explored the state-of-art understanding in placental biology through to clinical management of conditions related to abnormal placental function with a focus on understanding implications to the Indian population. International leaders in the field were invited to attend. There was interest in better understanding the state-of-art in placental and fetal imaging and image analysis and I gave a talk on this topic. A number of directions of follow-on were discussed as follow-on. |
Year(s) Of Engagement Activity | 2015 |
Description | Samsung Satellitte Symposium, European Congress in Radiology |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Talk was part of a lunch symposium presenting latest research in AI applied to radiology |
Year(s) Of Engagement Activity | 2017 |
Description | School Visit (Oxford) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Schools |
Results and Impact | Talk to year 9-12 Biomedical Sciences Society on biomedical engineering which including presenting work from this grant. |
Year(s) Of Engagement Activity | 2016 |
Description | School talk |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Schools |
Results and Impact | Talk at Headington School as the Key Note speaker for their Year of Science. |
Year(s) Of Engagement Activity | 2017 |
Description | Schools Careers Evening (Oxford) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Schools |
Results and Impact | Speaker as careers evening for Girls School. Discussions with the school girls including discussion of my research. |
Year(s) Of Engagement Activity | 2015,2016 |
Description | Speaker - International Women in Engineering Day 2017 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Schools |
Results and Impact | Secondary school girls from a number of local schools visited the department to see different areas of engineering and do some simple activities related to engineering. I gave the short talk at tea on some of emerging areas of engineering (wacky engineering) and talked a little about my own research and my field. Feedback from schools was positive for the whole event. |
Year(s) Of Engagement Activity | 2017 |
Description | Special Interest Ultrasound Group Annual meeting (Oslo) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | National ultrasound specialist interest group annual meeting in Oslo with participants from medical and NDE backgrounds, primarily industry focussed. I was one of two overseas guest speaks at the two day event. |
Year(s) Of Engagement Activity | 2019 |
Description | St Hughs College Donors Dinner |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Supporters |
Results and Impact | Short talk as part of college donors dinner event which sparked questions and discussions afterwards. |
Year(s) Of Engagement Activity | 2020 |
Description | Talk (AI@Oxford) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | Speaker in a healthcare session of AI@Oxford conference and discussions with some participants afterwards |
Year(s) Of Engagement Activity | 2019 |
Description | University of Bath Annual Athena Swan Lecture |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Other audiences |
Results and Impact | Careers talk which included presentation of work from the research grant. |
Year(s) Of Engagement Activity | 2015 |