MAESTRO Jr. - Multi-sensing AI Environment for Surgical Task & Role Optimisation
Lead Research Organisation:
Imperial College London
Department Name: Surgery and Cancer
Abstract
This project is about devising and implementing a smart operating room environment, powered by trustable, human-understanding artificial intelligence, able to continually adapt and learn the best way to optimise safety, efficacy, teamwork, economy, and clinical outcomes.
We call this concept MAESTRO.
A fitting analogy for MAESTRO is that of an orchestra conductor, a 'maestro', who oversees, overhears and directs a group of people on a common task, towards a common goal: a masterful musical performance. Although the music score is identical for all orchestras, there is no doubt that they all perform it in different ways and some significantly better than others. Although the quality and personality of orchestra musicians is very important, it is widely accepted that the role of the maestro is crucial, and extends beyond the duration of the musical performance to rehearsals and understanding of the context behind the music score.
Thus, while it is possible for orchestras to perform without conductors, most cannot function without one.
Our proposed MAESTRO AI-powered operating room of the future rotates around four key elements:
(a) The holistic sensing of patient, staff, operating room environment and equipment through an array of diverse sensor devices.
(b) Artificial intelligence focused on humans (human-centric), able to continually understand situations and actions developing in the operating room, and of intervening when necessary.
(c) The use of advanced human-machine user interfaces for augmenting task performance.
(d) A secure device interconnectivity platform, allowing the full integration of all above key elements.
As in our orchestra analogy, our envisioned MAESTRO directs the OR staff and surgical devices before, during and after a surgical procedure by:
(1) Sensing surgical procedures in all their aspects, including those which are currently neglected such as the physiological responses of staff (e.g., heart rate, blood pressure, sweating, pupil dilation), focus of attention, brain activity, as well as harmful events that may escape the attention of the clinical team.
(2) Overseeing individual and team performance in real-time, throughout the operation and across different types of surgeries and different teams.
(3) Guiding and assisting the surgical team via automated checkpoints, virtual and augmented visualisations, warnings, individualised and broadcasted alerts, automation, semi-automation, robotics, and other aids and factors that can affect performance in the operating room.
(4) Augmenting and optimising individual and collective operational capabilities, skills, and task ergonomics, through novel human-machine interaction and interfacing modalities.
The project is designed to have a significant societal, economic and technological impact, and to establish the NHS as a leading healthcare paradigm worldwide. MAESTRO leverages the expertise of top researchers in the areas of robotics, sensing, artificial intelligence, human factors, health policies and patient safety. It is co-designed in collaboration with top clinicians, one of the largest NHS Trusts in England, patient groups, performing artists, and several small and medium-sized enterprises and large multinational industries operating in the areas of artificial intelligence, medical devices, digital health, large networks, cloud services, cyber security.
We call this concept MAESTRO.
A fitting analogy for MAESTRO is that of an orchestra conductor, a 'maestro', who oversees, overhears and directs a group of people on a common task, towards a common goal: a masterful musical performance. Although the music score is identical for all orchestras, there is no doubt that they all perform it in different ways and some significantly better than others. Although the quality and personality of orchestra musicians is very important, it is widely accepted that the role of the maestro is crucial, and extends beyond the duration of the musical performance to rehearsals and understanding of the context behind the music score.
Thus, while it is possible for orchestras to perform without conductors, most cannot function without one.
Our proposed MAESTRO AI-powered operating room of the future rotates around four key elements:
(a) The holistic sensing of patient, staff, operating room environment and equipment through an array of diverse sensor devices.
(b) Artificial intelligence focused on humans (human-centric), able to continually understand situations and actions developing in the operating room, and of intervening when necessary.
(c) The use of advanced human-machine user interfaces for augmenting task performance.
(d) A secure device interconnectivity platform, allowing the full integration of all above key elements.
As in our orchestra analogy, our envisioned MAESTRO directs the OR staff and surgical devices before, during and after a surgical procedure by:
(1) Sensing surgical procedures in all their aspects, including those which are currently neglected such as the physiological responses of staff (e.g., heart rate, blood pressure, sweating, pupil dilation), focus of attention, brain activity, as well as harmful events that may escape the attention of the clinical team.
(2) Overseeing individual and team performance in real-time, throughout the operation and across different types of surgeries and different teams.
(3) Guiding and assisting the surgical team via automated checkpoints, virtual and augmented visualisations, warnings, individualised and broadcasted alerts, automation, semi-automation, robotics, and other aids and factors that can affect performance in the operating room.
(4) Augmenting and optimising individual and collective operational capabilities, skills, and task ergonomics, through novel human-machine interaction and interfacing modalities.
The project is designed to have a significant societal, economic and technological impact, and to establish the NHS as a leading healthcare paradigm worldwide. MAESTRO leverages the expertise of top researchers in the areas of robotics, sensing, artificial intelligence, human factors, health policies and patient safety. It is co-designed in collaboration with top clinicians, one of the largest NHS Trusts in England, patient groups, performing artists, and several small and medium-sized enterprises and large multinational industries operating in the areas of artificial intelligence, medical devices, digital health, large networks, cloud services, cyber security.
Publications
Lam K
(2022)
Machine learning for technical skill assessment in surgery: a systematic review
in npj Digital Medicine
Naik R
(2022)
The Measurement of Cognitive Workload in Surgery Using Pupil Metrics: A Systematic Review and Narrative Analysis.
in The Journal of surgical research
| Description | We have shown that we can use data from physiological sensors (eg, brain activity, heart rate, pupil tracking) to reveal the level of cognitive effort associated with a surgical task. In other words, we are able to tell the difficulty a task imposes on an idividual (surgery in this case) by using AI and wearable sensors. |
| Exploitation Route | We are already in the process of publishin our findings (ArxiV preprints already available and a few paper under review). Our findings may be avaiable in other domains, beyond surgery |
| Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Education,Healthcare |
| Description | Formation and Engagement with Clinical Advisory Board |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited key opinion leaders in this surgical field, in an organised hybrid workshop to discuss the MAESTRO vision and to set the foundations for further involvement as a partner, advisor and potential early adopter of the technology. Twelve surgeons covering pretty much the entire spectrum of surgical specialties. The long-term goals through this engagement activity where: 1. Demonstrate impact on real and co-designed clinical cases, through pilots with partnering clinicians across the UK 2. Explore and understand the implications and the ethical aspects, and boost trust and acceptability among surgical teams 3. Develop an ecosystem, both nationally and internationally, involving all relevant stakeholders including researchers, clinicians, patients, managers, regulators and industry to support this vision |
| Year(s) Of Engagement Activity | 2022 |
| Description | Hamlyn Symposium workshop on Human-Centric Data-Driven Perception, Cognition and Action in the Operating Theatre |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The aim of this workshop was to examine the state-of-the-art and unite under the same roof the different efforts in holistic sensing, AI, and robotics in the operating theatre. A major outcome is the expected cross-fertilisation between the diverse spectrum of relevant stakeholders, which include engineers, clinicians, scientist and industry |
| Year(s) Of Engagement Activity | 2022 |