A comparative study of effectiveness and cost-effectiveness of alternative poverty eradication interventions in extremely poor areas of Northern Ghana

Lead Research Organisation: University of Essex
Department Name: Economics

Abstract

Our research addresses directly the following overarching question: What factors shape pathways into and out of poverty and people's experience of these, and how can policy create sustained routes out of extreme poverty in ways that can be replicated and scaled up?

The study compares the effectiveness and cost-effectiveness of alternative poverty eradication projects and assesses their scalability and sustainability. In particular, it answers the following questions:

- What is the most effective poverty eradication strategy for an extremely deprived rural area: an integrated development project, a poverty graduation programme, or a cash transfer programme?
- Which strategy is more cost-effective at poverty reduction and for what reasons?
- What are effectiveness and cost-effectiveness of these strategies once they are scaled up at the national level and implemented over the long term?

The study will assess effectiveness and cost-effectiveness of three poverty reduction programmes implemented in deprived areas of Northern Ghana, which share the goal of eradicating poverty over the long term by unlocking poverty traps: 1) the Livelihood Empowerment Against Poverty (LEAP) programme implemented by the Government of Ghana, a cash transfer programme targeted to extremely poor households; 2) the SADA-Millennium Village Project implemented by the Earth Institute, consisting of a package of interventions directed to address multiple failures in the health, education and production domains; 3) the BRAC-inspired Poverty graduation programme implemented by local NGOs, which targets the poorest of the poor and provides households with a productive asset grant, cash consumption support, and other services.

The research has three main goals. First, we intend to inform choices between alternative poverty eradication programmes. This information will particularly benefit the funders and the implementers of the projects considered, but the lessons learned will have a wider application to similar projects implemented in low-income countries. Second, we intend to make a methodological contribution by developing methods of cost-effectiveness analysis of complex interventions. These methods are currently under-developed and no obvious methodology exist to account for the complexity of the institutional environments, the presence of synergies among different project components, and the presence of economies of scale when interventions are scaled up. Third, we intend to improve our understanding of poverty by investigating the mechanisms underlying extreme poverty and by analysing under what conditions specific interventions can unlock poverty traps.

The research will first analyse the effectiveness of the interventions using well-established impact evaluation methodologies relying on rigorous evaluation designs and solid control groups. The analysis will be applied to extraordinarily rich panel household datasets which allow the analysis of poverty over multiple dimensions in addition to standard monetary poverty. The second phase of the research will go beyond the analysis of effectiveness and statistical significance of project effects, by looking at the cost-effectiveness of the interventions. Cost data will be obtained and cost-effectiveness ratios will be calculated for main poverty targets, such as reduction in poverty head-count and malnutrition rates. In a third phase we will assess the scalability and the sustainability of the interventions. To this aim we will extrapolate the effects of the intervention over other geographic areas and over time using novel statistical techniques, and we will simulate different cost structures in the presence of economies of scale and different modalities of project implementation. Simulations, and related sensitivity analyses, will generate impact predictions of the effectiveness and cost-effectiveness of the interventions over the long term and at the national and regional levels.

Planned Impact

There is an increasing demand - from policy-makers, donors and academia - for impact evaluations that go beyond statistical significance and effect sizes and that address cost-effectiveness, scalability and sustainability of interventions. Our research will first benefit the organisations implementing and funding the development projects considered: DFID, the Earth Institute and the Government of Ghana. The results of our study will also benefit other implementers, including NGOs but also major organisations such as the World Bank and IFAD, that operate at the local level in poverty reduction efforts in West Africa and other low-income countries. Our methodological innovations will benefit research and funding institutions dedicated to the evaluation of poverty reduction project such as IFPRI, J-Pal, 3ie and the World Bank. Finally, we will benefit scholars and students in international development in the UK and Ghana. We will reach these multiple audiences with the following dissemination and capacity building activities:

- An inception workshop will be held in Accra at the University of Ghana with all the stake-holders of the three projects considered. In this workshop we will discuss an activity plan and stakeholders will contribute to defining the objectives of the study;
- Two sensemaking workshops will be conducted in Ghana at the end of the study. The first will be held in Accra and will target the Government of Ghana, the funders and the implementers of the project considered and of similar poverty reduction projects. The second workshop will be held at the University of Development Studies in Tamale and will target members of NGOs and local government officers that are implementing the interventions in the field. Both workshops will be learning events in which researchers and policy-makers will reconcile multiple sources of evidence and policy goals;
- Three papers on the methodological contributions of the study and its applications will be published in peer-reviewed journal such as the Journal of Development Studies, the World Bank Economic Review and the Journal of Development Economics. The papers will be presented at ISSER in Accra and at least one event in London to be organised jointly with 3ie;
- Summary briefs of the effectiveness and cost-effectiveness studies will be produced by the IDS communication department and will be particularly targeted to senior officers of the organizations involved. The investigators will also develop a methodological toolkit on cost-effectiveness analysis of complex interventions. The toolkit and the case studies provided by the research will be used in Impact Evaluation Course offered by one of the investigators at the IDS thus reaching a wider audience of development professionals working in evaluation;
- There will be three capacity building events (one held at the IDS and two at ISSER) for graduate students and officers of governments, NGOs and international organisations. Capacity building events will occur in occasion of research team meetings and will consist of teaching exchanges in the form of master classes, workshops and short courses. The event in the UK will be organised by ISSER and will focus on data collection methods and analysis of household surveys in difficult field contexts. The events in Ghana will focus on quasi-experimental impact methodologies and cost-effectiveness analysis of complex interventions and will be led by IDS and Essex;
- Following IDS Open Access policy, all research output will be made available on a project webpage on the IDS website. The page will contain summaries of the research projects, audiovisuals of the workshops and conferences, a downloadable version of the cost-effectiveness toolkit, the stata computer codes used in the empirical analysis, and pdf versions of the reports and papers produced by the team. The website will also offer an opportunity to visitors and events' participants to provide feedback via a blog page.

Publications

10 25 50
 
Description This award has supported the creation of new knowledge, increased research capability, created findings that can impact policy work of NGOs and the non-profit sector, and developed new research collaborations. We will focus on discussing only a few of the achievements here.

The first achievement is that of significant new knowledge regarding cost effectiveness of policies aiming to reduce poverty. Policy makers use many methods to fight poverty but three key ones are: (i) 'big push' interventions (such as with Millennium Villages); (ii) cash transfers (conditional or unconditional transfers); and (iii) graduation from poverty programs (combine a cash transfer along with a productive asset being given to a household). From our research, it is clear that, regarding poverty, graduation from poverty programs are the most cost effective in places such as the SADA area of northern Ghana. There is the caveat that the cash transfer program we looked at (LEAP) provided too small of a transfer, in real terms, to have had the effects for which policy makers had hoped. Thus, a cash transfer program with larger sums given to families, could still have reduced poverty.

In Northern Ghana, at roughly the same time, all three of these key poverty alleviation programs were implemented. We looked at data from the SADA Northern Ghana Millennium Village (MV), the LEAP program implemented by the Government of Ghana, and BRAC's graduation from poverty program. The MV and LEAP program had no impact on the level of poverty in the area. This made it clear that only the graduation from poverty program was cost-effective. However, graduation from poverty programs are not cheap and discussions with policy makers has led to research questions regarding how to reduce the costs.

The second achievement was the increased research capability. We held three workshops in Ghana with PhD students where we provided training on new methods of empirical identification. We also worked with PhD and master students on their research projects by meeting with them one-on-one and walking through their approach to answering their proposed research questions. The Ghana based Co-I also was able to attend a key workshop in Kenya. The grant also funded a southern based RA at the University of Ghana that was able to attend the same workshop in Kenya and present her own work. Besides increased research capacity, though, the PIs met with many government officials and NGOs - such as DFID and the MV project - to provide sensemaking of the results and discuss the findings from the papers produced by the investigators.

A final achievement that this award was that it was able to support a new collaboration between the investigators and researchers that had worked on other poverty impact evaluations in Ghana. For example, we have begun working with one of the authors that evaluated the BRAC graduation from poverty program for the Science publication. That work is important because it involves trying to find ways to reduce the cost of the graduation program; making it more cost-effective.
Exploitation Route The key findings of the research conducted with the support of this award are: (i) in extremely poor, semi-arid areas such as Northern Ghana, the most effective, and cost-effective way of reducing poverty are graduation from poverty programs; (ii) well-funded big push programs have no/little impact on poverty; and (iii) cash transfers have to be much larger than the government of Ghana has thus far been willing to provide if a cash transfer is to have any effect on poverty.

Given the clear finding that graduation programs are the most cost-effective at reducing poverty, it is likely that DFID Ghana and other NGOs focused on reducing poverty in Ghana, will focus on such programs. Through sensemaking meetings with policy makers and continued conversations with funders, we are working to discuss this research with the widest applicable audience. The key next step, though, is to determine how to reduce the cost of implementing a graduation from poverty program. Through conversations with researchers and policy makers, we hope our research on cost-effectiveness will have an impact on those discussions as well.
Sectors Agriculture, Food and Drink,Education,Government, Democracy and Justice

 
Description A key part of our award was to have sensemaking discussions with policy makers. We have met with many policy makers associated with the three programs we looked at - the MV, LEAP, and the BRAC graduation from poverty program. The other key stakeholder that we have met with is DFID. In Ghana we held discussions with two DFID representatives and hope to keep those discussions on-going. Our hope is to explain our findings - that graduation from poverty programs are the most cost effective - and provide support in determining how to bring down the cost of implementing graduation from poverty programs. The findings on the effectiveness and cost-effectiveness of the SADA-MVP project were widely publicised and debated. The project was found not to be successful and ultimately scaled down. The intervention in Northern Ghana was discontinued and other millennium village interventions in Africa were downsized, while similar interventions in other countries were modified in size and scope. While we cannot directly claim that our work led directly to those decisions, we believe it was an important determining factor. We would also like to mention that the Earth Institute embarked on a major evaluation of the MV programme after our work and that it is now taking impact evaluation very seriously. This is a change in the operations of the organisation that we believe we contributed to produce. The findings relating to the LEAP programme have been the subject of discussions with the Government of Ghana, DFID, the World Bank and UNICEF. Although cash transfer programmes are normally successful, our analysis and others showed that much could be improved in the delivery of the intervention in Ghana. Our findings sparked discussions regarding the size of the transfers and the targeting of the intervention in order to maximise impact. The GOG introduced changes to the programme to make the intervention more successful that are aligned with the conclusions of our evaluations. The Government of Ghana in collaboration with the World Bank, UNICEF and other development partners are now supporting further evaluations of LEAP and LIPW with the goal of further testing and investigating the areas of attention identified by our studies. Impact on the poverty graduation programme is less obvious. The project is largely regarded as successful and therefore the attention of researchers and policy-makers has been directed to ways of replicating and adapting the intervention rather than modifying or discontinuing project components.
First Year Of Impact 2019
Sector Education,Government, Democracy and Justice
Impact Types Policy & public services

 
Title GSPS Panel Data 
Description To look at the effects of the LEAP programme we tried to follow the instructions in the LEAP impact evaluation (see: https://assets.publishing.service.gov.uk/media/57a089ad40f0b649740001f2/LEAP_Impact_Evaluation_FINAL2014march17.pdf). Unfortunately, this requires access to the newest wave of the GSPS which is proprietary. We have been told they will make it publicly available soon. Therefore, below, is a list to where the currently, earlier, versions of the GSPS are available. Hopefully the new data will be available soon. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? No  
Impact This is the only dataset that will allow one to look at the impact of LEAP at the individual level. The government is now starting a LEAP 1000 programme so these results have been very important. 
URL https://assets.publishing.service.gov.uk/media/57a089ad40f0b649740001f2/LEAP_Impact_Evaluation_FINAL...
 
Title Graduating from Poverty Dataset 
Description We compared three poverty alleviation programmes and had to use the publicly available data from each. For the BRAC implemented programme, we used IPA's publicly available data from their Science paper. One can extract just the Ghana portion if one would like. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact This dataset allowed us to clearly see, of the three poverty alleviation programmes at which we were looking, on this BRAC project really made an impact on poverty. One can easily replicate the Science paper results and standardize definitions of poverty to see the impact. 
URL https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/NHIXNT
 
Title SADA MV Dataset 
Description We used data that was made publicly available regarding the DFID funded SADA MV in Northern Ghana. This dataset is available through the UK Data Archive but is extremely hard to obtain. We would be willing to help other researchers interested in using the dataset to navigate any issues they may face obtaining the data. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact We were able to replicate the ITAD report results but, in particular, we were able to standardize measures of poverty that were used across datasets such as BRAC and the Government of Ghana. 
URL https://beta.ukdataservice.ac.uk/datacatalogue/studies/study?id=7734
 
Description ISSER Collaboration 
Organisation Institute of Statistical, Social and Economic Research
Country Ghana 
Sector Academic/University 
PI Contribution ISSER is a premier research institute in Ghana. We have organized two conferences with them: one to discuss the project and one to provide practical training to students at the University of Ghana.
Collaborator Contribution They have organized the space and facilities for the conference.
Impact Two conferences.
Start Year 2017
 
Description CEDIL Talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Jeffrey Sachs debated the importance of MVs in alleviating poverty and mentioned the work of E Masset, a Co-PI in this group. The event was not organized by us, nor were we invited to participate, but the event itself relied on a lot of work that Dr Masset and Dr Nolen had done regarding the MV and its impact in Northern Ghana. (If this should not be here, let me know!)
Year(s) Of Engagement Activity 2020
 
Description NGO Meetings 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Met with DFID Ghana, BRAC, the MV, and members of the Government of Ghana. In these meetings discussions were held on what we had found during our research and on what the organizations were currently working. The goal of these meetings was to make our output and advice as relevant as possible to potential users of the output.
Year(s) Of Engagement Activity 2019
 
Description NGO Meetings 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Met with members of DFID Ghana, the Millennium Village (MV) project, and BRAC. These are the three main funders of the poverty alleviation programmes we examined in our study. We met to discuss their research questions and what output they would be most interested in hearing about.
Year(s) Of Engagement Activity 2017
 
Description Presentation and Training 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact One Co-I, Dr I Osei-Akoto and one UoG RA, Ms C Nyan, attending the "Translating Evidence to Policy and Action: Lessons and Experiences from the Region" in Nairobi and presented work relating to the three poverty programmes we were examining and gained training relevant to our research.
Year(s) Of Engagement Activity 2018
 
Description Teaching Exchange 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact As part of the grant we setup a teaching exchange in Accra. At the exchange we invited PhD students, practitioners, researchers from other schools, etc. We then went over methods for impact evaluation and how there were accomplished in practice.
Year(s) Of Engagement Activity 2017
 
Description Teaching Exchange 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact As part of the grant we setup a teaching exchange in Accra. At the exchange we invited PhD students, practitioners, researchers from other schools, etc. We then went over methods for impact evaluation and how there were accomplished in practice.
Year(s) Of Engagement Activity 2018