Interaction Analytics for Automatic Assessment of Communication Quality in Primary Care
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Molecular. Genetics & Pop Health
Abstract
The goal of this PhD project is to investigate the interaction and communication of between doctors and patients.
Studies have demonstrated the effect of doctors' interpersonal skills on concrete medical outcomes, both while interacting with patients and within teams. Recent work in the domain of argumentation and negotiation modelling has shown that natural language processing (NLP) and machine learning can be used to monitor and assess surface and cognitive communication skills. In addition, the same techniques have started to be deployed in the medical domain, e.g. tests in mental health conditions screening for assessment and diagnosis of Alzheihmer's disease and dementia.
In the scope of this PhD, two scenarios are being investigated for their complementarity. The first scenario features one-to-one communication, i.e. doctor-patient during medical interview in a GP's office setting. In this scenario, the focus is to monitor the different aspects of the consultation, based on state of the art practices and guidelines. In addition to the monitoring of the practitioner, it involves the analysis of the patient's discourse and behaviour. The reference for this scenario are the phases of the Cambridge guide to the medical interview: gathering information, providing structure, relationship building, explanation and planning, tracking information and closing the session.
The second scenario features many-to-many communication in medical emergency settings, e.g. intensive care units. The patient, being in critical condition, will not be directly tracked but the contribution of indirect data acquisition in the form of vital signs will be explored. Doctors and nurses interventions in critical and stressful situations involve an additional, different set of skills. In this context, the focus is set on crisis resource management (CRM), a concept emerging from, and in use in the aviation industry. This is centred around non-technical skills (NTS), a set of general cognitive and interpersonal skills. NTS complements domain skills and is crucial to allow safe and efficient interventions.
The core analysis in these two scenarios share similar attributes centred around planning, communication, assertiveness and utilisation of information. New models of multimodal interpretation will be developed based on the different inputs modalities and their combination. A research platform will be developed in parallel to include and test the models.
The first part Video and 3D recognition will be used for posture, gesture and facial expressions interpretation. The second part of the interpretation will use a complete Natural Language Understanding (NLU) pipeline, including prosodic features analysis and semantic processing based on automatic speech recognition and speaker diarisation. Discourse modelling, i.e. high level interpretation of contextual information, will then be developed based on the fused low level interpretations for the exploration of more complex patterns and structures, grounded in conclusions and directions of studies in communication performance.
The current work is set on the definition of the requirements and the specification of the recording setups based on the analysis of data collections previously performed in the domain. The objective is to offer guidelines for the acquisition of high quality anonymised dataset that would benefit both manual and automatic analysis and would be suitable for advanced processing.
Studies have demonstrated the effect of doctors' interpersonal skills on concrete medical outcomes, both while interacting with patients and within teams. Recent work in the domain of argumentation and negotiation modelling has shown that natural language processing (NLP) and machine learning can be used to monitor and assess surface and cognitive communication skills. In addition, the same techniques have started to be deployed in the medical domain, e.g. tests in mental health conditions screening for assessment and diagnosis of Alzheihmer's disease and dementia.
In the scope of this PhD, two scenarios are being investigated for their complementarity. The first scenario features one-to-one communication, i.e. doctor-patient during medical interview in a GP's office setting. In this scenario, the focus is to monitor the different aspects of the consultation, based on state of the art practices and guidelines. In addition to the monitoring of the practitioner, it involves the analysis of the patient's discourse and behaviour. The reference for this scenario are the phases of the Cambridge guide to the medical interview: gathering information, providing structure, relationship building, explanation and planning, tracking information and closing the session.
The second scenario features many-to-many communication in medical emergency settings, e.g. intensive care units. The patient, being in critical condition, will not be directly tracked but the contribution of indirect data acquisition in the form of vital signs will be explored. Doctors and nurses interventions in critical and stressful situations involve an additional, different set of skills. In this context, the focus is set on crisis resource management (CRM), a concept emerging from, and in use in the aviation industry. This is centred around non-technical skills (NTS), a set of general cognitive and interpersonal skills. NTS complements domain skills and is crucial to allow safe and efficient interventions.
The core analysis in these two scenarios share similar attributes centred around planning, communication, assertiveness and utilisation of information. New models of multimodal interpretation will be developed based on the different inputs modalities and their combination. A research platform will be developed in parallel to include and test the models.
The first part Video and 3D recognition will be used for posture, gesture and facial expressions interpretation. The second part of the interpretation will use a complete Natural Language Understanding (NLU) pipeline, including prosodic features analysis and semantic processing based on automatic speech recognition and speaker diarisation. Discourse modelling, i.e. high level interpretation of contextual information, will then be developed based on the fused low level interpretations for the exploration of more complex patterns and structures, grounded in conclusions and directions of studies in communication performance.
The current work is set on the definition of the requirements and the specification of the recording setups based on the analysis of data collections previously performed in the domain. The objective is to offer guidelines for the acquisition of high quality anonymised dataset that would benefit both manual and automatic analysis and would be suitable for advanced processing.
Publications
Ryan P
(2019)
Using artificial intelligence to assess clinicians' communication skills.
in BMJ (Clinical research ed.)
Studentship Projects
Project Reference | Relationship | Related To | Start | End | Student Name |
---|---|---|---|---|---|
MR/N013166/1 | 30/09/2016 | 29/09/2025 | |||
1805087 | Studentship | MR/N013166/1 | 31/08/2016 | 29/02/2020 | Pierre Albert |
Title | INCA GP |
Description | Real world recordings of medical consultations in GP offices (audio, angle of speech, 3D stream). |
Type Of Material | Database/Collection of data |
Year Produced | 2018 |
Provided To Others? | No |
Impact | Demonstration of feasibility of sensible data collection for the ICGP. |
Description | Data collection |
Organisation | Heriot-Watt University |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Provision and adaptation of the recording system created during this PhD. |
Collaborator Contribution | Feedback from the researcher's and experimenter's perspectives for the use of the system in collecting sensible data. |
Impact | Joint publication of the process expected this year. Multi-disciplinarity: Speech processing, Medicine (Dementia detection). |
Start Year | 2019 |
Description | ISO 24617-2 revision meeting |
Organisation | Saarland University |
Country | Germany |
Sector | Academic/University |
PI Contribution | On 2018-04-06.07 Following investigation on the application of ISO 24617-2 dialogue act taxonomy to the doctor-patient communication, an invitation was received to participate in the revision meeting of ISO standard 24617-2 for dialogue act annotation. A presentation was done to the members of the committee on the specificities of the communication in the medical consultation, characteristics of the existing annotation frameworks, and the use of dialogue acts in the analysis of the medical communication. |
Collaborator Contribution | Discussion of the specificities of assessment of medical communication requiring extension and integration into current revision of ISO 24617-2. Expertise in dialogue analysis and talks to pursue cooperation within a formal project. |
Impact | No outcome yet, revision of the standard (ISO/CD 24617-2) in development. Disciplines: dialogue analysis, health communication. |
Start Year | 2018 |
Description | ISO 24617-2 revision meeting |
Organisation | University of Tilburg |
Country | Netherlands |
Sector | Academic/University |
PI Contribution | On 2018-04-06.07 Following investigation on the application of ISO 24617-2 dialogue act taxonomy to the doctor-patient communication, an invitation was received to participate in the revision meeting of ISO standard 24617-2 for dialogue act annotation. A presentation was done to the members of the committee on the specificities of the communication in the medical consultation, characteristics of the existing annotation frameworks, and the use of dialogue acts in the analysis of the medical communication. |
Collaborator Contribution | Discussion of the specificities of assessment of medical communication requiring extension and integration into current revision of ISO 24617-2. Expertise in dialogue analysis and talks to pursue cooperation within a formal project. |
Impact | No outcome yet, revision of the standard (ISO/CD 24617-2) in development. Disciplines: dialogue analysis, health communication. |
Start Year | 2018 |
Title | Hardware - data collection system |
Description | Recording device with a microphone array and a 3D/video camera, with encryption of collected data. Two devices can be paired and different microphones can be used. |
Type Of Technology | Systems, Materials & Instrumental Engineering |
Year Produced | 2020 |
Impact | Data collection collaboration with Heriot-Watt |
Title | Medical consultation analysis tool |
Description | Analysis tool that processes metrics (turn taking, speech distribution) and extract features (egemaps) from audio recordings of medical consultation. Continuous development. Not publicly released. |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | Analysis tools and API to support the work of the PhD. |