Generation of echocardiogram images for 3D image enhancement and localisation

Echocardiography (Echo) is key to the assessment and management of all cardiac diseases. Echocardiograms are produced using ultrasound waves to create a moving picture of the heart. At the power levels used in the clinic, the use of sound waves is painless and harmless, and the devices required to generate them are low cost and portable which provides convenience [Potter]. These advantages of echocardiography contribute to its widespread clinical use today. 3D Echocardiography (3D Echo) allows the quantification of absolute cardiac chamber volumes and visualisation of the 3D structure and dynamic motion images of the heart, especially heart valve structures [Shiota]. It offers significant additional clinical information to traditional 2D echo, and has been identified as the best echocardiographic method for sequential quantification of left ventricle volumes and ejection fractions in patients with cancer undergoing chemotherapy. Moreover, unlike the 2D version, 3D echo is not reliant on plane positioning, does not require geometric modelling and does not make assumptions about the shapes of the chambers of the heart [Cheng], furthering its reproducibility and accuracy. However, the main limitation of 3D echo has been and still is the inferior image quality compared with today's 2D imaging technology [Lang]. Currently, the spatial resolution is limited by the number of beams and sweeps that the probe can send and receive. This limits lateral resolution specifically and as a result, also weakens image contrast. Higher resolution 3D images would enable more accuracy when calculating salient chamber volumes and improve the visualisation capabilities of this method. Attempts to improve image quality have mostly focused on changes to the transducer itself [Casas]. An important issue with the echo modality is that it is difficult to use without significant training and experience, especially in transoesophageal echocardiography (TEE) imaging. It can be difficult to know the probe's position and understand what exactly is being imaged. Automatic localisation of the probe would ease the process of performing TEE by helping the user identify the probe pose within the body. Furthermore, automatic localisation creates scope for more automation in performing the echo itself. For example, robotic actuators could be called on at certain stages of the echo to perform tasks that they would be better suited to compared to a medical professional. In order to tackle these issues, we plan to develop a pipeline that determines the probe's position and orientation, within the body, from 2D echo image inputs. This will be achieved by firstly developing an algorithm that, when given a 2D echo image, identifies its most likely location in a 3D anatomical model of the heart. Once developed, we will use this algorithm to predict the probe's position and orientation within the body, and hence develop methods to support 3D image guidance of TEE imaging, using 2D slices. Finally, in the final part of the DPhil we will use this pipeline to support the leveraging of the higher quality 2D echo images to enhance the 3D echo images, overcoming one of the main obstacles to 3D TEE. The methodologies used throughout the project will be mostly based on state-of-the-art machine learning tools, in particular Convolutional Neural Networks. This project is undertaken in partnership with GE Healthcare and falls within the EPSRC Medical Imaging research area. References Potter, A., Pearce, K., & Hilmy, N. (2019). The benefits of echocardiography in primary care. British Journal of General Practice, 69(684), 358-359. https://doi.org/10.3399/BJGP19X704513 Shiota, T. (2008). 3D echocardiography: The present and the future. Journal of Cardiology, 52(3), 169-185. https://doi.org/10.1016/J.JJCC.2008.09.004

The UK's world-leading position in biomedical research is critically dependent upon training scientists with the cutting-edge research skills and technological know-how needed to drive future scientific advances. Since 2009, the EPSRC and MRC CDT in Systems Approaches to Biomedical Science (SABS) has been working with its consortium of 22 industrial and institutional partners to meet this training need.

Over this period, our partners have identified a growing training need caused by the increasing reliance on computational approaches and research software. The new EPSRC CDT in Sustainable Approaches to Biomedical Science: Responsible and Reproducible Research - SABS:R^3 will address this need. By embedding a sustainable approach to software and computational model development into all aspects of the existing SABS training programme, we aim to foster a culture change in how the computational tools and research software that now underpin much of biomedical research are developed, and hence how quantitative and predictive translational biomedical research is undertaken.

As with all CDT Programmes, the future impact of SABS:R^3 will be through its alumni, and by the culture change that its training engenders. By these measures, our existing SABS CDT is already proving remarkably successful. Our alumni have gone on to a wide range of successful careers, 21 in academic research, 19 in industry (including 5 in SABS partner companies) and the other 10 working in organisations from the Office of National Statistics to the EPSRC. SABS' unique Open Innovation framework has facilitated new company connections and a high level of operational freedom, facilitating 14 multi-company, pre-competitive, collaborative doctoral research projects between 11 companies, each focused on a SABS student.

The impact of sustainable and open computational approaches on biomedical research is clear from existing SABS' student projects. Examples include SAbDab which resulted from the first-ever co-sponsored doctorate in SABS, by UCB and Roche. It was released as open source software, is embedded in the pipelines of several pharmaceutical companies (including UCB, Medimmune, GSK, and Lonza) and has resulted in 13 papers. The SABS student who developed SAbDab was initially seconded to MedImmune, sponsored by EPSRC IAA funding; he went on to work at Roche, and is now at BenevolentAI. Similarly, PanDDA, multi-dataset X-ray crystallographic software to detect ligand-bound states in protein complexes is in CCP4 and is an integral part of Diamond Light Source's XChem Pipeline. The SABS student who developed PanDDA was awarded an EMBO Fellowship.

Future SABS:R^3 students will undertake research supported by both our industrial partners and academic supervisors. These supervisors have a strong track record of high impact research through the release of open source software, computational tools, and databases, and through commercialisation and licensing of their research. All of this research has been undertaken in collaboration with industrial partners, with many examples of these tools now in routine use within partner companies.

The newly focused SABS:R^3 will permit new industrial collaborations. Six new partners have joined the consortium to support this new bid, ranging from major multinationals (e.g. Unilever) to SMEs (e.g. Lhasa). SABS:R^3 will continue to make all of its research and teaching resources publicly available and will continue to help to create other centres with similar aims. To promote a wider cultural change, the SABS:R^3 will also engage with the academic publishing industry (Elsevier, OUP, and Taylor & Francis). We will explore novel ways of disseminating the outputs of computational biomedical research, to engender trust in the released tools and software, facilitate more uptake and re-use.