Development & Clinical Translation of Scalable HPC Ultrasound Models

Lead Research Organisation: University College London
Department Name: Medical Physics and Biomedical Eng

Abstract

The use of ultrasound as a diagnostic imaging tool is well known, particularly during pregnancy where ultrasound is used to create images of the developing foetus. In recent years, a growing number of therapeutic applications of ultrasound have also been demonstrated. The goal of therapeutic ultrasound is to modify the function or structure of the tissue, rather than produce an anatomical image. This is possible because the mechanical vibrations caused by the ultrasound waves can affect tissue in different ways, for example, by causing the tissue to heat up, or by generating internal radiation forces that can agitate the cells or tissue scaffolding. These ultrasound bioeffects offer a huge potential to develop new ways to treat major diseases such as cancer, to improve the delivery of drugs while minimising side-effects, and to treat a wide spectrum of neurological and psychiatric conditions.

The fundamental challenge shared by all applications of therapeutic ultrasound is that the ultrasound energy must be delivered accurately, safely, and non-invasively to the target region within the body. This is difficult because bones and other tissue interfaces can severely distort the shape of the ultrasound beam. This has a significant impact on the safety and effectiveness of therapeutic ultrasound, and presents a major hurdle for the wider clinical acceptance of these exciting technologies. In principle, any distortions to the ultrasound beam could be accounted for using advanced computer models. However, the underlying physics is complex, and the scale of the modelling problem requires extremely large amounts of computer memory. Using existing software, a single simulation running on a supercomputer can take many days to complete, which is too long to be clinically useful.

The aim of this proposal is to develop more efficient computer models to accurately predict how ultrasound waves travel through the human body. This will involve implementing new approaches that efficiently divide the computational problem across large numbers of interconnected computer cores on a supercomputer. New approaches to reduce the huge quantity of output data will also be implemented, including calculating clinically important parameters while the simulation runs, and optimising how the data is stored to disk. We will also develop a professional user interface and package the code within the regulatory framework required for medical software. This will allow end-users, such as doctors, to easily use the code for applications in therapeutic ultrasound without needing to be an expert in computer science. In collaboration with our clinical partners, the computer models will then be applied to different applications of therapeutic ultrasound to allow the precise delivery of ultrasound energy to be predicted for the individual patient.

Planned Impact

The novel software developed as part of this proposal will generate impact in two different ways. First, there is considerable potential for the computational models to impact clinical practice by extending and expediting the use of ultrasound as a tool to treat cancer and other diseases. Second, the computational performance of the models is likely to have significant impact in the many allied areas of research and industry in which waves play a role.

Clinical Impact: Over the last decade, novel applications of therapeutic ultrasound have excited much interest within the research community. However, wider clinical acceptance has been modest due to challenges in controlling the ultrasound fields with the required accuracy. The scalable k-space models and clinical user interface proposed in this application could overcome this barrier and act as a catalyst for the widespread clinical application of these technologies. Impact will arise from (i) the significantly enhanced model accuracy, (ii) the unprecedented levels of computational performance which will allow large-scale ultrasound simulations in previously unrealisable compute times, (iii) the low barrier to entry for end-users that are not experts in HPC, and (iv) adherence to the regulatory framework required for the clinical application of scientific software.

Academic and Industrial Impact: Acoustic models are used for many applications outside biomedical ultrasound, including non-destructive evaluation, seismology, oil and gas exploration, architectural acoustics, ultrasonic welding, and ocean acoustics. Our efficient large-scale models will be of direct relevance to these fields, offering improved modelling accuracy and unprecedented computational performance. The techniques for domain decomposition and the optimisation of the code for emerging many-core HPC architectures will also be of interest to those modelling waves in other contexts where spectral methods are commonly used, for example in computational electromagnetics and computational quantum chemistry.

Economic Impact: The enhanced computational performance, clinician-led user interface design, and regulatory compliance of the software developed during this project will provide a significant competitive edge over the simulation packages currently used in academia and industry. These advances will make the software commercially attractive to medical device companies. It is expected the generated IP will lead to licensing agreements or the development of new spinout companies, with the UK becoming a base for future international investment into computational acoustics and clinical applications of therapeutic ultrasound.

Publications

10 25 50
publication icon
Arridge S (2016) On the adjoint operator in photoacoustic tomography in Inverse Problems

publication icon
Jaros J (2015) Large-scale Ultrasound Simulations Using the Hybrid OpenMP/MPI Decomposition in Proceedings of the 3rd International Conference on Exascale Applications and Software

publication icon
Jaros J (2015) Large-Scale Ultrasound Simulations with Local Fourier Basis Decomposition in SC15: International Conference for High Performance Computing, Networking, Storage and Analysis

publication icon
Jaros J (2015) Full-wave nonlinear ultrasound simulation on distributed clusters with applications in high-intensity focused ultrasound in The International Journal of High Performance Computing Applications

publication icon
Martin E (2016) Simulating Focused Ultrasound Transducers Using Discrete Sources on Regular Cartesian Grids. in IEEE transactions on ultrasonics, ferroelectrics, and frequency control

publication icon
Martin E (2017) Rapid Spatial Mapping of Focused Ultrasound Fields Using a Planar Fabry-Pérot Sensor. in IEEE transactions on ultrasonics, ferroelectrics, and frequency control

 
Description The aim of this project is to develop next generation modelling tools for therapeutic ultrasound that scale efficiently across large computer clusters, and can be easily accessed by end users such as clinicians. The project is now at the end of its third year, and significant progress has been made towards these objectives. First, new domain decomposition techniques have been developed that allow the computational problem to be more efficiently mapped onto supercomputing resources. These include pencil and local domain decomposition techniques for global domain decomposition, and highly efficient local domain decomposition based on local Fourier basis for running simulations on a cluster of graphical processing units (GPUs). Second, the backend of a clinical interface has been established, including pipelines for secure data transfer between local and remote compute resources, new techniques for on-the-fly visualisation and compression, and the development of a transducer library for several single element therapy transducers. Third, a regulatory QMS framework has been put in place for the clinical user interface. This allows the use of the software within a medical context, and for the software to be later CE-marked as a medical device. Finally, the models have been validated using clinical data from several partner sites.
Exploitation Route The developed software tools are likely to generate impact both on the targeted delivery of ultrasound therapy, as well as academics working to understand wave physics and ultrasound metrology. This impact is likely to come about through our open-source software releases, as well as the development of new modelling and domain decomposition approaches that can be applied in many other fields.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare

 
Description The acoustic models being developed as part of this grant are being used by ultrasound and acoustic companies in the private sector, including for transducer design, image reconstruction, and ultrasound dose calculations.
First Year Of Impact 2016
Sector Digital/Communication/Information Technologies (including Software),Education,Healthcare,Manufacturing, including Industrial Biotechology
Impact Types Economic

 
Description ThUNDDAR Network Pilot Funding
Amount £49,298 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 09/2018 
End 02/2019
 
Description UCL Knowledge Exchange and Innovation Fund
Amount £28,163 (GBP)
Organisation UCL Business 
Sector Private
Country United Kingdom
Start 03/2018 
End 09/2018
 
Title k-Wave Acoustics Toolbox 
Description k-Wave is an open source MATLAB toolbox designed for the time-domain simulation of propagating acoustic waves in 1D, 2D, or 3D. The toolbox has a wide range of functionality, but at its heart is an advanced numerical model that can account for both linear and nonlinear wave propagation, an arbitrary distribution of heterogeneous material parameters, and power law acoustic absorption. The numerical model is based on the solution of three coupled first-order partial differential equations which are equivalent to a generalised form of the Westervelt equation. The equations are solved using a k-space pseudospectral method, where spatial gradients are calculated using a Fourier collocation scheme, and temporal gradients are calculated using a k-space corrected finite-difference scheme. The temporal scheme is exact in the limit of linear wave propagation in a homogeneous and lossless medium, and significantly reduces numerical dispersion in the more general case. Power law acoustic absorption is accounted for using a linear integro-differential operator based on the fractional Laplacian. A split-field perfectly matched layer (PML) is used to absorb the waves at the edges of the computational domain. The main advantage of the numerical model used in k-Wave compared to models based on finite-difference time domain (FDTD) schemes is that fewer spatial and temporal grid points are needed for accurate simulations. This means the models run faster and use less memory. A detailed description of the model is given in the k-Wave User Manual and the references below. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact The toolbox is widely used in academia and industry, and has been used for research into transcranial ultrasound, ultrasound therapy, the development of novel ultrasound sources, photoacoustic imaging, and many other applications. There have been eight major releases of the toolbox. It currently has more than 9500 registered users in 70 countries. A 2010 paper describing the first release of the toolbox has >500 citations, and the active online user forum has >2500 posts.