Human-centred Machine Intelligence to optimise Robotic Surgical Training (HuMIRoS)
Lead Research Organisation:
University College London
Department Name: Medical Physics and Biomedical Eng
Abstract
Robotic-assisted surgery (RAS) is carried out in almost 2M cases worldwide, delivering advantages to both patients and surgeons. Its adoption in the UK is limited by the availability of training: RAS requires high levels of skill and rigorous training, which is not widely available. One critical aspect of training is the performance evaluation of trainees, which is currently done through manual review by senior surgeons: it is inefficient and difficult to standardise. To address this issue the multidisciplinary HuMIRoS project will pioneer AI technology to automatically evaluate the performance of robotic surgeon trainees and provide them meaningful feedback. This will lead to improved training experience and accelerated learning curve.
Following a user-centred approach, the project will develop the first AI-based system for automated action quality assessment (AQA): automatically measure the quality of actions carried out with the surgical robot. The core idea for AQA is to fuse information from heterogeneous data sources (video, robot joint kinematics, semantic information) and pioneer multimodal artificial intelligence (MMAI) methods that link performance assessment to surgical actions (e.g., instrument movements) and consequences (e.g., errors). AQA outcomes will be presented to RAS trainees as constructive feedback through a user interface (UI), co-designed with users and stakeholders to deliver optimum training experience and outcomes. Closing the loop, machine understanding will be rationalised and made accountable on the UI using eXplainable AI (XAI), to facilitate the formulation of causal interpretation of the observed surgeons' actions and derived performance estimates.
HuMIRoS has the following key objectives:
To architect new MMAI technology for estimating and classifying RAS performance by representing core skill attributes (tool manipulation, camera (endoscope) navigation, respect for tissue, surgical outcome) and detecting instances of surgical errors (or suboptimal execution).
To develop novel human-computer interactions (HCI) around the MMAI, including the development of novel XAI methods, to communicate AQA predictions to end-users and make them accountable, thereby optimising RAS training.
To validate the HuMIRoS approach with experiments in RAS training courses and real cases, demonstrate and quantify the benefits introduced, release datasets to promote AQA research.
The research will leverage datasets collected from the largest UK RAS training hub (project partner The Griffin Institute, Northwick Park Institute for Medical Research) and annotated with clinically validated performance metrics and error description tools. Experimentation in structured training sessions, end-user workshops, and real RAS cases will evaluate the face and construct validity and ability to generalise, of developed solutions. The project will regularly engage patient advisory groups to inform AI development and purpose.
Direct beneficiaries of project outcomes include:
Patients and healthcare systems: Accelerate the learning curve of new surgeons trained faster and more efficiently. Increase uptake and democratisation of RAS as it will be accessible in more surgical sites.
RAS trainees: Optimise the training experience for trainee surgeons, now able to receive expert-level feedback readily and develop skills more efficiently. Faculty time and resources will also be streamlined. Facilitate integration of new technologies (e.g., telementoring) in RAS training.
RAS surgeons: Decrease intraoperative complications by identifying moments of high risk for error.
Following a user-centred approach, the project will develop the first AI-based system for automated action quality assessment (AQA): automatically measure the quality of actions carried out with the surgical robot. The core idea for AQA is to fuse information from heterogeneous data sources (video, robot joint kinematics, semantic information) and pioneer multimodal artificial intelligence (MMAI) methods that link performance assessment to surgical actions (e.g., instrument movements) and consequences (e.g., errors). AQA outcomes will be presented to RAS trainees as constructive feedback through a user interface (UI), co-designed with users and stakeholders to deliver optimum training experience and outcomes. Closing the loop, machine understanding will be rationalised and made accountable on the UI using eXplainable AI (XAI), to facilitate the formulation of causal interpretation of the observed surgeons' actions and derived performance estimates.
HuMIRoS has the following key objectives:
To architect new MMAI technology for estimating and classifying RAS performance by representing core skill attributes (tool manipulation, camera (endoscope) navigation, respect for tissue, surgical outcome) and detecting instances of surgical errors (or suboptimal execution).
To develop novel human-computer interactions (HCI) around the MMAI, including the development of novel XAI methods, to communicate AQA predictions to end-users and make them accountable, thereby optimising RAS training.
To validate the HuMIRoS approach with experiments in RAS training courses and real cases, demonstrate and quantify the benefits introduced, release datasets to promote AQA research.
The research will leverage datasets collected from the largest UK RAS training hub (project partner The Griffin Institute, Northwick Park Institute for Medical Research) and annotated with clinically validated performance metrics and error description tools. Experimentation in structured training sessions, end-user workshops, and real RAS cases will evaluate the face and construct validity and ability to generalise, of developed solutions. The project will regularly engage patient advisory groups to inform AI development and purpose.
Direct beneficiaries of project outcomes include:
Patients and healthcare systems: Accelerate the learning curve of new surgeons trained faster and more efficiently. Increase uptake and democratisation of RAS as it will be accessible in more surgical sites.
RAS trainees: Optimise the training experience for trainee surgeons, now able to receive expert-level feedback readily and develop skills more efficiently. Faculty time and resources will also be streamlined. Facilitate integration of new technologies (e.g., telementoring) in RAS training.
RAS surgeons: Decrease intraoperative complications by identifying moments of high risk for error.