Measuring complex outcomes of environment and development interventions

How do we know if development aid is spent effectively and delivering results? Aid effectiveness has become an important discussion point in international development in recent years, leading to renewed focus on how to measure the performance of development programmes, which is Theme 3 of the ESRC-DFID call. Measuring results is difficult because the results depend on what you chose to measure, when, for whom, and how it is measured. A second difficulty is attribution: how do you know that changes are due to development interventions, as opposed to other ongoing processes. Rigorous impact evaluation methodologies have been developed to address the problem of attribution by comparing intervention sites to controls. However, impact evaluation methods have tended to focus on standard economic measures of human wellbeing (e.g. income), which may not capture outcomes considered relevant by local people - such as resource tenure, education, ability to insure against shocks, or political power.

This project focuses on interventions designed to counter increasing environmental resource scarcity, caused by unsustainable use of biodiversity, deforestation, and degradation of ecosystem services, arguably some of the greatest long-term threats to human wellbeing. Understanding the processes that cause resource scarcity and what are effective interventions to address resource scarcity are relevant to Theme 1 of the call. Environment-development interventions are also some of the hardest to assess, since the processes that cause environmental resource scarcity are often very complex, and potentially affect multiple aspects of human wellbeing (not just income). For this reason, credible evaluations of their medium to long-term impacts on human wellbeing are few, providing little evidence to inform decision-makers about what works, for whom, and why.

The project aims to directly improve the implementation of policies in the environment-development sector through enhanced understanding of what works, based on more appropriate measurement of results. It brings together the Wildlife Conservation Society (WCS, a science-based environmental organization working in over 60 countries), and two of the UK's foremost academics in the field from University College London and Imperial College, as well as local and national partners. The project has three components:
(1) a methodological research component, which focuses on how to measure the development impacts of environment-development interventions on the poor
(2) a field research component, which will test the different methodological approaches with developing country researchers at 3 sites (2 in Africa and 1 in Asia) that have very different social, cultural and political settings, to see how methods perform in different circumstances and to specifically evaluate 3 field interventions focused on Payments for Ecosystem Services
(3) a practical component, which will use the results of the research to directly inform how environment-development projects are implemented in the field, building upon the consortium's networks of partners and programmes.

The research responds directly to the growing demand for knowledge about how to measure the performance of development programmes, and the specific paucity of impact evaluations in the environment-development sector. The research will therefore be high impact: both academically, in terms of publications, and for policymakers and practitioners because it responds to already-identified needs for information. The field component focuses on incentive mechanisms, such as Payments for Ecosystem Services, which have been widely promoted over the past decade as a more equitable way to ensure poverty alleviation and sustainable environmental resource management. Through the project's networks and partners, the results will directly impact how current environment-development policies are implemented, and will inform the design of future initiatives.

Impact: This project addresses a central development policy issue: how can poor people's experience of change be factored more directly into policy evaluation and formulation? It delivers innovative ways to capture poor people's experienced wellbeing, enabling comparison with more established measurements of change, and building a more robust evidence base about what interventions work, how, and for whom. The project has been specifically designed with the three research sites to address issues that are a priority for local people and national governments. Through the consortium and our national partners, the project is well-placed to ensure that the results of the research are used to influence local decision-making and national policy.

Who will benefit from this research?
Besides researchers (see Academic Beneficiaries statement) other beneficiaries include: 1) national-level policymakers in the focal countries; 2) implementing organisations and practitioners; 3) local resource users; 4) the wider global network of organisations through WCS's networks and the consortium members; and 5) donor organisations and development policy initiatives.

How will they benefit from this research?
(1) National-level policymakers in Cambodia, Tanzania and Uganda will benefit from increased understanding of what works, how to measure results, and the implications of the research for existing policies (such as revenue-sharing from protected areas) and new policies (such as PES and REDD). The project sites have been chosen for their potential policy impact, looking at how insecurity of local land tenure can drive land-use change and affect poverty (Cambodia and Tanzania), how to share revenue from protected areas (Uganda), and how best to structure PES/REDD interventions to generate dual development and environment benefits (Cambodia and Tanzania).

(2) Implementing organisations and practitioners in Cambodia, Tanzania and Uganda will benefit from increased understanding of how environment and development projects are implemented and evaluated, both now and in the long-term.

(3) Local resource users will become better heard and represented, and be more willing to engage, thus improving intervention effectiveness and contributions to poverty alleviation. In all three sites, conditional, market-based, incentive payments are being used to encourage local resource users to reduce pressures on environmental resources, and increase the benefits they derive from these resources. The research will help to understand the extent to which payment interventions are achieving their stated goals, and will propose changes to improve implementation.

(4) Global networks of policy makers and practitioners will benefit from new knowledge. Project briefings, methodologies, results and publications will be distributed through consortium partners' networks with reach to >60 countries worldwide. The results will be of direct relevance to programmes in Gabon, Guatemala (funded by DFID) and Bolivia, where project partners are also measuring long-term changes in wellbeing and governance as a consequence of environment-development interventions.

(5) Donor organisations and development policy initiatives will benefit with greater understanding of what 'works' and how to measure results. The need for such information has been recognized by the High Level Forum on Aid Effectiveness, and is necessary to measure progress towards the Millennium Development Goals.

The project will build local capacity through supporting national post-graduate students with scholarships and research project supervision. All the project partners have a strong record at building national capacity, and many of these students and researchers then go on to fill high-level Government, policy and practitioner positions, ensuring lasting impact.