Augmented-Reality for Guiding Laparoscopic Surgery

1. Brief description of the context of the research including potential impact

Gallstone disease is one of the most common surgical problems worldwide and can cause a variety of different complications varying from biliary colic to life-threatening conditions such as ascending cholangitis and pancreatitis. Symptomatic gallstones are a common indication for surgery. Magnetic Resonance Cholangiopancreatography (MRCP) is a non-invasive imaging test that is used to detect biliary obstruction and has been recommended as a good way of detecting common bile duct stones. However, during surgery for removing gallbladder (cholecystectomy) it is difficult to identify biliary anatomy, and errors can occur with dire consequences.

This research has been initiated within the wider project of the NIHR Patient Safety Research Collaboration, and we will aim to make this extremely common surgery safer and to reduce risks.

We have previously delivered a system for Augmented Reality (AR) liver surgery, that displays a 3D model from pre-operative Computed Tomography (CT) scans on top of laparoscopic video, to guide the surgeon. We will extend this system for the purpose of gall bladder surgery. In the longer term, the impact of such a system would be to make surgery safer for patients undergoing gallbladder removal, liver resection or kidney excision.

2. Aims and Objectives

The aim is to provide image-guidance to enable the surgeon to identify biliary structures with more confidence. The specific aims of the project may include:

Image processing of MRCP scans and image-alignment with other pre-operative imaging

Real-time identification of biliary anatomy from laparoscopic video

Reconstruction of internal anatomy from laparoscopic video or ultrasound

Alignment of pre-operative data to intra-operative imaging

Provision of quantifiable metrics to aid during surgery, e.g. depth to target

3. Novelty of Research Methodology

These objectives will require the novel application of machine learning methods and particularly deep learning. We will adapt popular transformer networks for video image-processing and reinforcement learning for the alignment problem. Deep learning in medical imaging is a fast moving field, with novel algorithms appearing daily. We will work with other researchers at the Wellcome/EPSRC Centre of Interventional and Surgical Sciences (WEISS) and Centre for Medical Image Computing (CMIC) to develop state of the art methods.

4. Alignment to EPSRC's strategies and research areas

The project is well aligned with EPSRC Healthcare Technologies Strategy, and specifically focusses on the "discovering and accelerating the development of new interventions". The use of AR technologies for surgery will provide safer and more targeted interventions.

5. Any companies or collaborators involved

The project is co-funded by the National Institute of Health Research (NIHR) under the NIHR Central London Patient Safety Research Collaboration grant.