Novel methods for optimising health systems payment for performance interventions to improve maternal and child health in low-resource settings

Lead Research Organisation: London School of Hygiene & Tropical Medicine
Department Name: Public Health and Policy

Abstract

There are different ways to pay health care providers. They can be paid based on the resources they use (e.g. staff, drugs) and the size of their population. They can also be paid based on what outcomes they achieve, which is termed payment for performance (P4P). P4P schemes are aimed at influencing the behaviour of health workers and their managers to deliver better quality health care. P4P schemes are currently being implemented in many low- and middle-income countries to improve maternal, newborn and child health.

Many studies have focused on assessing these P4P programmes' impact on the performance targets. More recently, studies have considered the effect of P4P on the inputs and infrastructure required to deliver health care services (i.e. the health system). However, these studies have mainly focused on effects on a single element within the health system, e.g. drugs or staff, rather than looking at all these factors in an interconnected, comprehensive manner. Mathematical models enable the analysis of how interconnected systems such as the health system function and respond to change. While models have been built and used to look at health programmes, to date there has been very limited use of these models to study health systems in low- and middle-income countries and their response to programmes such as P4P. One of the advantages of models is that they can also be used to anticipate the likely effects of programme changes before these changes are actually made. This can be helpful to those designing programmes to help them work out which design would work best.

This study will use two types of models to understand the effects of P4P: a model of how a health facility functions in terms of the overall flows of patients and drugs and supplies across the facility (i.e. a system dynamic model) and a model of the way health care workers, their managers and patients interact and behave within the health facility (i.e. an agent-based model). The models will be developed in two different settings: Tanzania and Zambia. Tanzania and Zambia were chosen as they have made mixed progress in terms of maternal, newborn and child health outcomes and they both made the decision to introduce P4P to try to improve maternal and child health. These two models will provide us with an understanding of how health systems work and respond to the P4P programme in these settings, and how P4P can be best designed for maximum impact on the health of mothers, newborns and children. To construct models of the Tanzanian health care system, we will use information from a previous study of the impact of P4P in Tanzania together with interviews with programme implementers. The models will be used in Zambia to see if they can accurately predict the effects of P4P on the Zambian health system or if changes in the model structure are needed. The models will be used to understand the effects (both intended and unintended) of P4P in each country and explore how changes in the design of the Tanzanian and Zambian P4P programmes may affect health and health systems outcomes. Results will be used to improve P4P programme design in each setting. We will also develop a toolkit for how to develop and use system dynamics and agent-based models to analyse health system response to interventions such as P4P for use in other countries, and training activities to support their uptake. This project will support knowledge sharing and learning across partners in the United Kingdom, Tanzania, Zambia and Uganda.

Technical Summary

This project will apply novel methods rooted in complexity science to understand and optimise health systems payment for performance (P4P) interventions to maximise impact on maternal, newborn and child health (MNCH) in low-resource settings. P4P is a mechanism to improve the availability, quality and utilisation of essential health services through financial incentives to providers and healthcare managers. It has been applied widely in high-income settings and low and middle income countries (LMIC) have also started its implementation to improve MNCH.

This research focuses on Tanzania and Zambia - a low- and a lower-middle income country, respectively, which have been implementing P4P schemes to improve MNCH outcomes since 2011. This study will build a system dynamic model (SDM) and an agent-based model (ABM) of the Tanzanian health system and its response to P4P. System dynamics represents a top-down approach to model the complex macro behaviour of the health system. ABM, on the other hand, is a bottom-up approach which can model emergent behaviour resulting from interactions between health workers, health care managers and patients. These models will be developed based on in-depth interviews, available data and literature reviews, and calibrated using 85%-90% data from the impact and process evaluations of P4P in each country. We will validate the SDM and ABM in the Tanzanian and compare ABM and SDM in characterising the P4P scheme in Tanzania. We will examine the generalisability of the model to Zambia. We will use these novel models to examine the intended and unintended effects of the P4P scheme that were not previously measured or observed, and to predict the effects of feasible changes to the design of the P4P scheme and examine what changes to the programme design would be needed to reach the desired output/outcome level. We will also build capacity among decision makers and researchers on SDM and ABM to study health systems using models in LMIC.

Planned Impact

This research is intended to benefit three main groups: a) national stakeholders in low- and middle-income country settings; b) international donors and agencies investing in health systems; c) the academic and researcher community. In addition, as these groups act on the study's findings, there is clear potential for health workers and their managers to benefit in terms of a better work environment and for patients to benefit in terms of increased access to higher quality and more effective care, which will result in better health. Here, we set out how each group is expected to benefit from the research, and the Pathways to Impact section sets out the strategies for achieving this benefit.

Study findings will be of interest to national governments, payment for performance (P4P) implementers and donors supporting these programmes in Tanzania and Zambia, by helping to better understand how such investments affect health systems and maternal and child health and how to optimise impact from these investments and maximise value for money. The models will be developed with a user-friendly software interface, and can provide the foundation for models considering the effects of other health system strengthening interventions in the future. The methodological toolkit developed can guide other countries on design choices, where to implement and how to monitor effects over time.

This research will interest the international community of donors and agencies financing and supporting the implementation of P4P, as the findings may help to shape their own investments in other countries. Furthermore, they may be able to adapt and use the models in other settings.

The academic beneficiaries and the benefits are outlined in the Academic beneficiaries section.
 
Title COSMIC - a film about the project in Tanzania 
Description We created a short 8 minute film introducing the COSMIC project to stakeholders. 
Type Of Art Film/Video/Animation 
Year Produced 2020 
Impact This film has been circulated among policy stakeholders in Tanzania. 
 
Title The COSMIC project - Zambia 
Description We created an 8 minute film to introduce Zambian policy makers to the project. 
Type Of Art Film/Video/Animation 
Year Produced 2020 
Impact This video has been circulated to policy makers in Zambia. 
 
Description Advancing methods for building global system resilience to health and climate-related shocks
Amount £6,000 (GBP)
Funding ID IF-2021-038 
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2021 
End 08/2023
 
Description Applying a complex adaptive systems lens to the global health system
Amount £26,130 (GBP)
Funding ID IF-2021-038 
Organisation The Leverhulme Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2021 
End 08/2022
 
Description UKRI Covid-19 Grant Extension Allocation
Amount £87,583 (GBP)
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 09/2020 
End 09/2021
 
Title Creation of Causal Loop Diagram for P4P in Tanzania 
Description Using secondary qualitative data, Rachel Cassidy has created a causal loop diagram of the Tanzanian health system and P4P which will be the foundation for the system dynamic model. 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? No  
Impact This will be used to build the system dynamic model. 
 
Title Semi-structured interview guide for data collection in Zambia to inform the causal loop diagram 
Description A semi-structured interview guide was developed for use in Zambia to explore the generalisability of the Tanzanian causal loop diagram and adapt it to Zambia. 
Type Of Material Improvements to research infrastructure 
Year Produced 2021 
Provided To Others? No  
Impact The tool was used to gather data in Zambia. 
 
Title Semi-structured interview guide for qualitative interviews with stakeholders involved in implementation or evaluation of P4P in Tanzania 
Description This tool was used to obtain qualitative data in relation to the project in order to validate the causal loop diagram. 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? No  
Impact The tool gathered data that was used to finalise the causal loop diagram and draft an academic paper. 
 
Title Agent based model 
Description We have built an agent based model of delivery care in Tanzania, focused on community demand dynamics. 
Type Of Material Computer model/algorithm 
Year Produced 2023 
Provided To Others? No  
Impact We are in the process of completing model documentation and will then make it publicly available. 
 
Title Data on stakeholder experience of P4P in Tanzania 
Description In stakeholder interviews to validate the developed causal loop diagram, we asked participants if they recognised the system structures and elements in the diagram and if changes needed to be made to reflect their experience of the health system and P4P. A semi-structured interview tool was used, where we presented segments of the causal loop diagram to stakeholders using question prompts to illicit feedback. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? No  
Impact Analysis of data is complete, with paper drafted and under review at a journal. 
 
Title Data on stakeholder perceptions of RBF in Zambia 
Description Data based on semi-structured interviews with national and subnational level stakeholders in the health sector. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? No  
Impact None yet 
 
Title System dynamic model 
Description The system dynamic model explores how pay for performance affects the delivery of antenatal and delivery care in Tanzania. The model can also explore how contextual factors change outcomes, and how delays in payment and a shift in the allocation of funding between providers and facilities affect outcomes. The model is built using Stella Architect. 
Type Of Material Computer model/algorithm 
Year Produced 2022 
Provided To Others? No  
Impact The model is finalised and is in the process of being uploaded onto a web platform together with model documentation to enable model re-use and adaptation. by others. 
 
Description Collaboration with IIASA 
Organisation International Institute for Applied Systems Analysis
Country Austria 
Sector Academic/University 
PI Contribution JB is being hosted by the IIASA for a one year sabbatical funded by the Leverhulme Trust. JB is exploring the interconnections between systems thinking approaches in health and socio-ecological systems research.
Collaborator Contribution Reinhard Mechler is hosting JB during her sabbatical at the Systemic Risk and Resilience Research group. Nikita Strelkovskii supported COSMIC by undertaking a formal analysis of causal loop diagrams using network analysis methods.
Impact Two joint proposals: Leverhulme Trust International Exchanges fellowship; and a Royal Society International Exchanges fellowship.
Start Year 2021
 
Description Ifakara Health Institute - Dr Peter Binyaruka 
Organisation Ifakara Health Institute
Country Tanzania, United Republic of 
Sector Charity/Non Profit 
PI Contribution Quarterly in person/Skype meetings, discuss the current progress made with the project and providing short seminars in SD modelling methods.
Collaborator Contribution Dr Binyaruka has participated in a model validation session, using his expert knowledge and experience of the P4P initiate in Tanzania to refine the conceptual system dynamics model (causal loop diagram). He has attended project meetings in person (in the UK) and over Skype. In the next stage of the project, Dr Binyaruka will be applying for project ethics approval in Tanzania and contributing to the development of the system dynamics model with his expert knowledge of the P4P initiative and pilot P4P initiative data that has been collected in Tanzania.
Impact Conceptual system dynamics model of the P4P initiative and impact on key performance indicators in MNCH services in Tanzania.
Start Year 2018
 
Description Involved in London-based system dynamic methods network 
Organisation Royal Veterinary College (RVC)
Country United Kingdom 
Sector Academic/University 
PI Contribution Rachel Cassidy is a member of a System Dynamics Modelling Group, led by Dr Kevin Queenan (The Royal Veterinary College, University of London). The working group, consisting of researchers from London-based institutions including LSHTM, SOAS and RVC, meet at least once a month to work through examples of SDM in the wider literature and discuss issues with modelling we have encountered in our own work.
Collaborator Contribution Partners have given feedback on the research.
Impact None yet.
Start Year 2019
 
Description Makerere University - Dr Agnes Rwashana Semwanga 
Organisation Makerere University
Country Uganda 
Sector Academic/University 
PI Contribution Quarterly in person/Skype meetings, discuss the current progress made with the project.
Collaborator Contribution Dr Rwashana Semwanga has participated in a model validation session, using her expert knowledge and experience of SD modelling to explore the impact of health policy on MNCH services and health outcomes to refine the conceptual system dynamics model (causal loop diagram). She has attended project meetings in person (in the UK) and over Skype. In the next stage of the project, Dr Rwashana Semwanga will be contributing to the development of the system dynamics model with her expert knowledge of health system modelling, particularly simulating MNCH services and 'what-if' scenario policy testing.
Impact Conceptual system dynamics model of the P4P initiative and impact on key performance indicators in MNCH services in Tanzania.
Start Year 2018
 
Description Partnership with Andrada Tomoaia-Cotisel at RAND cooperation. 
Organisation RAND Europe
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Rachel Cassidy has been involved in regular discussions with other PhD students using system dynamic modelling methods, facilitated by Andrada.
Collaborator Contribution Andrada has provided guidance to the project, by acting as a mentor to Rachel.
Impact Conference presentations have been submitted.
Start Year 2018
 
Description Partnership with Zaid Chalabi 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Zaid agreed to continue to advise on the SDM and ABM within the project after he left LSHTM.
Collaborator Contribution Zaid supports Rachel Cassidy and Anna Foss.
Impact Zaid provided input into the systematic review which was published by the project.
Start Year 2018
 
Description Research network on application of complexity science methods to study the global health system 
Organisation Graduate Institute of International and Development Studies
Country Switzerland 
Sector Academic/University 
PI Contribution I jointly organised a workshop on What can we learn from adopting a complex adaptive systems lens to the global health system and how can we do it? with Suerie Moon. Two follow up workshops have been held by the Graduate Institute. We are in the process of developing a collaborative proposal.
Collaborator Contribution Agnes Semwanga from Uganda will also join the workshop.
Impact A workshop which is planned in March 2021. Jo Borghi and Reinhard Mechler have put forward two funding applications, one for a Leverhulme Trust grant supporting study leave for Jo Borghi, and another Royal Society international exchanges grant. They were both successful. LSHTM, IIASA and Graduate Institute are jointly applying for funding to apply these ideas to the global vaccine system.
Start Year 2021
 
Description Research network on application of complexity science methods to study the global health system 
Organisation International Institute for Applied Systems Analysis
Country Austria 
Sector Academic/University 
PI Contribution I jointly organised a workshop on What can we learn from adopting a complex adaptive systems lens to the global health system and how can we do it? with Suerie Moon. Two follow up workshops have been held by the Graduate Institute. We are in the process of developing a collaborative proposal.
Collaborator Contribution Agnes Semwanga from Uganda will also join the workshop.
Impact A workshop which is planned in March 2021. Jo Borghi and Reinhard Mechler have put forward two funding applications, one for a Leverhulme Trust grant supporting study leave for Jo Borghi, and another Royal Society international exchanges grant. They were both successful. LSHTM, IIASA and Graduate Institute are jointly applying for funding to apply these ideas to the global vaccine system.
Start Year 2021
 
Description Research network on application of complexity science methods to study the global health system 
Organisation Open University
Country United Kingdom 
Sector Academic/University 
PI Contribution I jointly organised a workshop on What can we learn from adopting a complex adaptive systems lens to the global health system and how can we do it? with Suerie Moon. Two follow up workshops have been held by the Graduate Institute. We are in the process of developing a collaborative proposal.
Collaborator Contribution Agnes Semwanga from Uganda will also join the workshop.
Impact A workshop which is planned in March 2021. Jo Borghi and Reinhard Mechler have put forward two funding applications, one for a Leverhulme Trust grant supporting study leave for Jo Borghi, and another Royal Society international exchanges grant. They were both successful. LSHTM, IIASA and Graduate Institute are jointly applying for funding to apply these ideas to the global vaccine system.
Start Year 2021
 
Description University of Zambia 
Organisation University of Zambia
Country Zambia 
Sector Academic/University 
PI Contribution We have had regular meetings with colleagues at the University of Zambia regarding the project and model generalisability.
Collaborator Contribution Partners have provided valuable input into the study design for Zambia.
Impact Not yet.
Start Year 2018
 
Description Asked about potential to adapt model to explore new primary care financing approach 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact We had a meeting with Unicef Tanzania about how we could adapt the COSMIC models to inform the design of a new primary care financing approach in Tanzania. We are developing a proposal for this work now.
Year(s) Of Engagement Activity 2023
 
Description Dissemination workshop in Dar es Salaam 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact We organised a dissemination workshop with policy makers from Tanzania and Zambia to share the main study findings and provide policy makers with the opportunity to explore the models and provide feedback on their improvement.
Year(s) Of Engagement Activity 2022
 
Description Presentation on system dynamic methods for health systems research by Rachel Cassidy 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact Rachel was invited to present at the Centre for Health Economics Economic Evaluation theme meeting to introduce the system dynamics methods and their use in health systems research, to a group of researchers with an interest in this field.
Year(s) Of Engagement Activity 2020
 
Description Validation of Agent Based Model in Tanzania 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact A series of interviews were held with stakeholders in Tanzania to inform the design and help validate the structure of the Agent Based Model.
Year(s) Of Engagement Activity 2021
 
Description Validation of causal loop diagrams in Tanzania 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The causal loop diagram we developed using secondary data collected in a previous project on impact of P4P was validated through a series of new stakeholder interviews. Three rounds of data collection took place (in March, June and December 2020) with twenty-one stakeholders who were closely involved with the evaluation and implementation of P4P in Tanzania.

As a route to stakeholder engagement and informing stakeholders on the purpose of these interviews in the wider context of our research project, we sent stakeholders a flyer and link to a short film introducing the research project and purpose of the interviews.
Year(s) Of Engagement Activity 2020