HOD2: Toward Holistic Approaches to Clinical Prediction of Multi-Morbidity: A Dynamic Synergy of Inter-Connected Risk Models
Lead Research Organisation:
University of Manchester
Department Name: School of Health Sciences
Abstract
Individuals with multiple medical conditions are more likely to die earlier and have lower quality of life. Despite the high number of people who are diagnosed with multiple conditions, clinical practice tends to operate within distinct areas of individual conditions. Several measures have been developed that attempt to quantify the overall complexity of such comorbidity burden, but such metrics cannot predict multiple outcomes to help guide decision-making.
To this end, clinical prediction models (CPMs) are mathematical tools/algorithms that aim to support clinical decision-making by predicting the likelihood that a clinical event of interest will occur given a set of characteristics about the individual (e.g. their age, gender, weight, etc.). However, CPMs also operate in pockets of individual diseases, where different CPMs are made to predict the likelihood of a single adverse clinical outcome. However, this fails to respect the way medical practice works and is unhelpful for the patient who is likely interested in their whole healthcare and care planning, rather than risks of developing individual/specific conditions. Ignoring the relationships between different conditions can lead to an under-estimation of risk, which can have consequences for care-planning and treatment decision-making.
Therefore, this proposal will aim to develop a "CPM-Network" environment, where models will be developed to predict the likelihood of a patient developing different (but potentially related) events. For example, this is classically achieved by predicting the risk of diagnosis A from one CPM, the risk of diagnosis B from another CPM, and then combining these risks by assuming the diagnoses are not related to each other (independent). The key point of this proposal is that these are not independent events, and our CPM-Network will capture this appropriately. Clinically, this means that patients will be managed differently by knowing that the actual probabilities (from the CPM-Network) are higher.
There are emerging modelling techniques that can be used to formulate such a CPM-Network, but methodological challenges currently prohibit them being used in such a capacity. This proposal will address these challenges and aim to develop methods that relax previous modelling assumptions, to allow development of CPMs that reflect a more realistic and holistic view of a patient's health and care.
In this project, we have the following objectives:
1) To develop methods that fit multiple CPMs simultaneously to allow CPMs to predict risks of multiple events across different disease areas in a computationally feasible manner.
2) Investigate validation (testing) of CPM-Networks, including extending methods from Objective 1 to consider penalisation/shrinkage to mitigate the dangers of overfitting.
3) To examine the feasibility of applying our CPM-Network to proof-of-concept clinical examples of: coronary heart disease, atrial fibrillation, stroke, chronic kidney disease and type-II diabetes mellitus, compared to conventional approaches.
4) Explore strategies for communicating risks from a CPM-Network through public and stakeholder engagement, and develop software to disseminate the CPM-Network approach.
There are a range of potential applications and benefits arising from this work, since tackling multi-morbidity (patients with multiple medical conditions) is a high priority for the NHS. For example, accurately predicting multi-morbid risk through a CPM-Network can aid clinical decision-making through appropriate multi-morbidity planning. This project directly challenges historic approaches to doing this, to produce models that can better inform care needs, aid patients understand future prognosis, inform healthcare professionals, and guide service provision.
To this end, clinical prediction models (CPMs) are mathematical tools/algorithms that aim to support clinical decision-making by predicting the likelihood that a clinical event of interest will occur given a set of characteristics about the individual (e.g. their age, gender, weight, etc.). However, CPMs also operate in pockets of individual diseases, where different CPMs are made to predict the likelihood of a single adverse clinical outcome. However, this fails to respect the way medical practice works and is unhelpful for the patient who is likely interested in their whole healthcare and care planning, rather than risks of developing individual/specific conditions. Ignoring the relationships between different conditions can lead to an under-estimation of risk, which can have consequences for care-planning and treatment decision-making.
Therefore, this proposal will aim to develop a "CPM-Network" environment, where models will be developed to predict the likelihood of a patient developing different (but potentially related) events. For example, this is classically achieved by predicting the risk of diagnosis A from one CPM, the risk of diagnosis B from another CPM, and then combining these risks by assuming the diagnoses are not related to each other (independent). The key point of this proposal is that these are not independent events, and our CPM-Network will capture this appropriately. Clinically, this means that patients will be managed differently by knowing that the actual probabilities (from the CPM-Network) are higher.
There are emerging modelling techniques that can be used to formulate such a CPM-Network, but methodological challenges currently prohibit them being used in such a capacity. This proposal will address these challenges and aim to develop methods that relax previous modelling assumptions, to allow development of CPMs that reflect a more realistic and holistic view of a patient's health and care.
In this project, we have the following objectives:
1) To develop methods that fit multiple CPMs simultaneously to allow CPMs to predict risks of multiple events across different disease areas in a computationally feasible manner.
2) Investigate validation (testing) of CPM-Networks, including extending methods from Objective 1 to consider penalisation/shrinkage to mitigate the dangers of overfitting.
3) To examine the feasibility of applying our CPM-Network to proof-of-concept clinical examples of: coronary heart disease, atrial fibrillation, stroke, chronic kidney disease and type-II diabetes mellitus, compared to conventional approaches.
4) Explore strategies for communicating risks from a CPM-Network through public and stakeholder engagement, and develop software to disseminate the CPM-Network approach.
There are a range of potential applications and benefits arising from this work, since tackling multi-morbidity (patients with multiple medical conditions) is a high priority for the NHS. For example, accurately predicting multi-morbid risk through a CPM-Network can aid clinical decision-making through appropriate multi-morbidity planning. This project directly challenges historic approaches to doing this, to produce models that can better inform care needs, aid patients understand future prognosis, inform healthcare professionals, and guide service provision.
Technical Summary
Clinical prediction models (CPMs) are models that are used to predict the risk of clinically relevant outcomes in individuals with a particular disease or health condition. They are increasingly used to support clinical decisions, yet they seldom reflect the interplay between developing multiple comorbidities. Specifically, CPMs operate in silos of disease risk, where different (and independent) models are derived to predict the risk of a single adverse clinical outcome. Current approaches to predicting multi-morbidity through CPMs incorrectly assume that the probability of patients developing multi-morbidity is the product of the risks from individual models. Our proposed solution is to advance multi-state modelling, to develop a "CPM-Network" that can appropriately predict the risk of different (but related) clinical outcomes, with the associated benefits to patient care, healthcare planning and service utilisation.
Multi-state survival models present a way of predicting risk of a patient moving between different 'states'. While such multi-state CPMs are appealing since they account for competing risks and can predict risks of different clinical events within a given disease area, they do not scale to model the risks of different clinical events across disease areas, in a cumulative sense. As we move from a single event (i.e. two-state) CPM, to a multi-disease (i.e. multi-state) CPM, we find combinatorial complexities that cannot be overcome in the existing multi-state modelling framework. As such, this proposal will extend competing risk and multi-state techniques, to allow development of CPMs that reflect a realistic and holistic view of a patient's health and care.
Our research question is: what is the feasibility and potential benefits (compared with existing approaches to predict multi-morbidity) of developing a CPM-Network? We will address methodological challenges in this space such as computational complexity, model overfitting and model validation.
Multi-state survival models present a way of predicting risk of a patient moving between different 'states'. While such multi-state CPMs are appealing since they account for competing risks and can predict risks of different clinical events within a given disease area, they do not scale to model the risks of different clinical events across disease areas, in a cumulative sense. As we move from a single event (i.e. two-state) CPM, to a multi-disease (i.e. multi-state) CPM, we find combinatorial complexities that cannot be overcome in the existing multi-state modelling framework. As such, this proposal will extend competing risk and multi-state techniques, to allow development of CPMs that reflect a realistic and holistic view of a patient's health and care.
Our research question is: what is the feasibility and potential benefits (compared with existing approaches to predict multi-morbidity) of developing a CPM-Network? We will address methodological challenges in this space such as computational complexity, model overfitting and model validation.
Planned Impact
Stakeholders of the outcomes from this project might include: academic researchers (methodological and applied), healthcare professionals (e.g. GPs), patients/public (through tailoring predictions from models to individual patients), and professional organisations. Indeed, managing and treating multi-morbidity is at the forefront of healthcare systems globally. To this end, the work outlined in this proposal has the potential to inform (accurate) multi-morbidity risk estimation at both a population- and patient-level. Hence, GPs could use models developed using the methodology derived from this project to inform care-planning. For an individual patient, optimising multi-morbidity care planning is key to improve their quality of life and future prognosis.
Additionally, there are potential academic beneficiaries, since the rapidly expanding use of routinely collected health data for research presents a number of methodological challenges that need to be addressed to ensure that these uses are robust. This project tackles some of these key challenges (as described in the case for support). Greater understanding and methodological solutions for working with routine health data will be key to their wider usage and for the ability for such analyses to answer questions of public health importance without significant bias. The methodological product created is likely to have impact throughout the academic communities. As prediction models are used increasingly as decision support tools, it is important that they are appropriately developed, validated and implemented.
The use of prediction models in clinical practice, decision support, audit and stratified medicine is rapidly growing. Improvements in the methodology for these models therefore has great potential to improve the efficiency of the healthcare system, and ensure that resource is appropriately targeted. Current use of CPMs that operate in isolation fails to allow appropriate and valid inferences about multi-morbid risks for healthcare planning, evaluation and service provision. We are interested in estimating the probability of patients developing multi-morbidity, which is not the product of the risks from individual models since they are not independent events.
The specific application that we consider (predicting risks of coronary heart disease, atrial fibrillation, stroke, chronic kidney disease and type-II diabetes mellitus) will provide proof-of-concept information on the feasibility of using our methods in practice. These conditions are chosen for their high population burden of disease, availability of data and strong expertise among the investigators. However, to reiterate, our principal aim is to develop methodology that could be applied across a range of clinical settings.
The dissemination of the results of this project to the aforementioned beneficiaries is described in the pathways to impact document, where we outline our approaches to realise the impacts that we mention here.
Additionally, there are potential academic beneficiaries, since the rapidly expanding use of routinely collected health data for research presents a number of methodological challenges that need to be addressed to ensure that these uses are robust. This project tackles some of these key challenges (as described in the case for support). Greater understanding and methodological solutions for working with routine health data will be key to their wider usage and for the ability for such analyses to answer questions of public health importance without significant bias. The methodological product created is likely to have impact throughout the academic communities. As prediction models are used increasingly as decision support tools, it is important that they are appropriately developed, validated and implemented.
The use of prediction models in clinical practice, decision support, audit and stratified medicine is rapidly growing. Improvements in the methodology for these models therefore has great potential to improve the efficiency of the healthcare system, and ensure that resource is appropriately targeted. Current use of CPMs that operate in isolation fails to allow appropriate and valid inferences about multi-morbid risks for healthcare planning, evaluation and service provision. We are interested in estimating the probability of patients developing multi-morbidity, which is not the product of the risks from individual models since they are not independent events.
The specific application that we consider (predicting risks of coronary heart disease, atrial fibrillation, stroke, chronic kidney disease and type-II diabetes mellitus) will provide proof-of-concept information on the feasibility of using our methods in practice. These conditions are chosen for their high population burden of disease, availability of data and strong expertise among the investigators. However, to reiterate, our principal aim is to develop methodology that could be applied across a range of clinical settings.
The dissemination of the results of this project to the aforementioned beneficiaries is described in the pathways to impact document, where we outline our approaches to realise the impacts that we mention here.
Publications
Gehringer CK
(2023)
Developing and externally validating multinomial prediction models for methotrexate treatment outcomes in patients with rheumatoid arthritis: results from an international collaboration.
in Journal of clinical epidemiology
Martin GP
(2021)
Developing clinical prediction models when adhering to minimum sample size recommendations: The importance of quantifying bootstrap variability in tuning parameters and predictive performance.
in Statistical methods in medical research
Martin GP
(2021)
Clinical prediction models to predict the risk of multiple binary outcomes: a comparison of approaches.
in Statistics in medicine
Pate A
(2024)
Calibration plots for multistate risk predictions models.
in Statistics in medicine
Pate A
(2023)
Minimum sample size for developing a multivariable prediction model using multinomial logistic regression.
in Statistical methods in medical research
Pate A
(2023)
Developing prediction models to estimate the risk of two survival outcomes both occurring: A comparison of techniques.
in Statistics in medicine
Title | R package for CPM Development |
Description | Open source R package to enable researchers to apply the methods in this award and other related projects. |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2022 |
Provided To Others? | Yes |
Impact | Production of open-source software is vital to enable the methodology to be used/implemented and have a positive impact on research in this space. Given the nature of this output, it is difficult to quantify this information, but we do know it has been downloaded around 2000 times. |
URL | https://github.com/GlenMartin31/predRupdate |
Title | R package for testing calibration of a multi-state model |
Description | Open source R package to enable researchers to apply the methods that have been developed in this award around how to assess the calibration of a multi-state model (for multiple outcome prediction; e.g., multimorbidity). |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2023 |
Provided To Others? | Yes |
Impact | Production of open-source software is vital to enable the methodology to be used/implemented and have a positive impact on research in this space. Given the nature of this output, it is difficult to quantify this information, but we do know it has been downloaded around 1000 times. |
URL | https://cran.r-project.org/web/packages/calibmsm/index.html |
Description | Advancing understanding of multimorbidity in metabolic disease through innovation in statistical machine learning |
Organisation | Novo Nordisk |
Country | Denmark |
Sector | Private |
PI Contribution | This collaboration seeks to explore methods within the area of multimorbidity. The methods developed in my MRC award for multimorbidity prediction using multi-state models are forming the foundation of the research with Novo Nordisk, advancing these methods within a counterfactual framework to enable counterfactual/causal predictions about multimorbidity planning/treatment. Prof Niels Peek (Co-I on my MRC award) is leading the collaboration with Novo Nordisk and was awarded funding as part of that collaboration. |
Collaborator Contribution | Their expertise in this area and access to some of their trial data in this space to apply the methods we develop. |
Impact | This was a multi-disciplinary collaboration between methodologists, clinicians and epideiologists. Outputs from this collaboration are ongoing research networks, which will build towards publications in the future. |
Start Year | 2021 |
Description | Non-Technical Blog Posts |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | We aimed to translate the research from this award into non-technical summaries to enable wider interaction. Specifically, we wrote blog posts about the research work, and created summary videos. |
Year(s) Of Engagement Activity | 2021,2022,2023 |
URL | https://glenmartin31.github.io/ |