Content Based Image Retrieval For Real-Time Registration In Image-Guided Interventions

Ultrasound imaging provides important real-time information to guide surgical interventions. Intra-operative Laparoscopic UltraSound (LUS) has been widely used in hepatic surgery, and may improve the detection, characterisation and localisation of hepatic tumours. However, it can be difficult to use LUS as the surgeon is normally required to look at both the laparoscopic video on one monitor and then the UltraSound (US) images on a different monitor, while performing their duties. Furthermore it is difficult to relate live LUS images to pre-operative images such as Magnetic Resonance (MR) or Computed Tomography (CT) scans, as the ultrasound is 2 dimensional, and the MR/CT scans are mostly 3 dimensional, and both look very different to the patients anatomy.

So, various authors including ourselves have developed image-guided laparoscopic surgery systems to provide an integrated computer system to guide the surgeon, where the aim is to provide an intuitive display of all the available imaging data, in a convenient and easy to use fashion. However, while many advancements in guidance technology have been developed by the academic community over the last 20 years, only a very small proportion has transitioned to clinical use. Challenges include the lack of space, the need for sterility, the requirement for as little user-interaction as possible and the time constraints imposed by the cost of theatre time. Many research prototypes simply do not meet these criteria and fail to translate to a clinically useful product.

Typically, current methods of registering (aligning) data are often too slow or too awkward to be used in real-time. I propose a feasibility study to build a new framework for real-time registration that can be applied to applications as diverse as laparoscopic surgery, endoscopic fetal surgery, robot-assisted surgery and image-guided ablation of tumours.

The proposed method will make use of image simulation prior to surgery. The computer will simulate and store a large quantity of ultrasound images, from a variety of different positions and orientations. Then during surgery, the live ultrasound feed will be matched to the images pre-operative, simulated, database. This will enable real-time alignment of the pre-operative data, thereby providing a much easier to use system.

My role as principal investigator is to maximise the likelihood of the techniques developed through this research being exploited for positive benefits.

A particular benefit of the proposed project is that there are potentially two different routes to a product. If the technology can be made to work without electro-magnetic (EM) tracking, then it will be immediately attractive to a wide variety of ultrasound manufacturers, and will be easy to translate to the clinic, as it will not require significant hardware changes, or changes to clinical workflow. If however, the EM tracking is a necessity, then we will be well placed to translate the technology as we will be communicating with BK Medical and using their new probes.

Impact On Clinical Practice: Minimally invasive procedures offer significant patient, clinical and cost benefits over conventional open surgical procedures. The adoption of such techniques is accelerating in areas such as urology and hepatobiliary surgery. However the more complex procedures place increasing demands on the surgeon and drive the use of more advanced technology such as image guidance to reduce cognitive load. But these technologies will only have real clinical impact if they can work seamlessly in the operating room, do not hinder the normal workflow and hence can be easily adopted as a standard of care. In this project, we will first target laparoscopic ultrasound devices, for the liver and kidney, but in principal, the technology could be applicable to a wide range of imaging devices and interventional procedures.

Commercialisation: In order to have real impact, the technology must be commercialised. I have already started conversations with BK Medical who would be well placed to take advantage of the output. This could take the form of a licensing deal. An alternative is to develop the technology into a product ourselves. Recently CMIC span-out Smart Target (Dr. Dean Barratt), which involved implementing the necessary ISO-13485 quality management system and developing the software to IEC-62304. I was closely involved with setting up and establishing this framework, and my previous commercial experience means I am well placed for such an endeavour.

Intellectual Property: The UCL Business (UCLB) team will be involved throughout the project duration to ensure the timely generation of IP when techniques with commercial exploitation potential arise. UCLB have already been involved in setting up non-disclosure agreements with BK Medical.

Steps To Impact: Starting with the ideas proposed here, I will take the following steps to maximise the impact of the new techniques developed, translate them to the clinic and facilitate industrial exploitation.
1. Development of real-time simulation software and submit a paper to IPCAI 2018 (deadline Nov 2017), a prestigious conference in our field.
2. Establish a research software framework with Dr. Philip Pratt, Imperial College London.
3. Prepare a test data set from our previous porcine experiments and publish online.
4. Develop the main CBIR approach, and submit a paper to MICCAI 2018 (deadline Feb 2018), the premier conference in our field.
5. Collect human data from the RFH cases, and discuss results with BK Medical.
6. Develop a high quality software library with the key algorithms to be either licensed or open-sourced as appropriate, and as advised by UCLB at the time in question.
7. Investigate the interest of BK Medical, and pursue a combined project with BK Medical.
8. If successful, investigate start-up funding for a potential spin-out.

An ESPRC first grant is critical for steps 1 to 6 and to provide the results to support further grant funding and collaborative clinical, academic and industrial partnerships for steps 5-8. Funding for further clinical studies will be sought from the MRC and Wellcome Trust in joint proposals with RFH clinicians.