Indicators for Measuring, Monitoring and Evaluating Post-Disaster Recovery

Lead Research Organisation: University of Cambridge
Department Name: Architecture

Abstract

This proposal is to produce guidelines for measuring, monitoring and evaluating recovery after a major disaster. The aim is to develop indicators of recovery that can exploit the wealth of data now available, including that from satellite imagery, internet-based statistics and advanced field survey techniques. Two key aspects of the proposal are that the indicators will be based on a user needs survey of aid agencies and will be validated against published data and field survey observations. The proposal is important because currently there is no agreed standard approach to evaluating the effectiveness of recovery aid. A simple way of using satellite imagery, statistics and digital in-field data to measure and monitor the extent of recovery over time would be of immense benefit to aid agencies in targeting resources effectively and evaluating the effectiveness of their expenditure.
 
Description Monitoring and evaluation of recovery and reconstruction after natural disasters can assist
on-going aid effort, provide accountability and guide aid policy. The project has developed new guidelines which provide a systematic, independent and replicable approach to monitoring and evaluating the recoveryprocess. The indicators proposed in the guidelines were tested in case study research of Ban Nam Khem, Thailand that was hit by the 2004 Indian Ocean tsunami, and Chella Bandi, Muzaffarabad, AJK Pakistan, that was struck by the 2005 Kashmir earthquake.

The indicators adopted by the Recovery Project encompass a range of physical,
environmental, social and economic factors that combine to give an accurate picture of the
reconstruction process at specific intervals. Both the speed and quality of recovery can be
monitored and evaluated by comparing key indicators to base-line statistics also acquired
with satellite imagery. Remote sensing-based performance indicators offer national governments and donor agencies a systematic and independent framework for accurately and comprehensively
monitoring and evaluating recovery and reconstruction. From a time-series of satellite
images a comprehensive set of indicators bring together pieces of the recovery jigsaw that
would otherwise take considerable time and resources to assemble.

Our indicators of recovery are based on the experience of the project team and feedback from users andstakeholders. They revolve around three key questions.
What to measure?
Knowing which indicators to measure and when to acquire satellite imagery is dependent on
the nature of the disaster, the needs of the users and the limitations of the satellite imagery
available. The Recovery Project created a list of indicators after consulting with users via a
user-needs survey. The results of this survey suggest there is a strong preference for a
comprehensive approach to monitoring recovery encompassing multiple sectors. This is the
approach we took. The proposed M&E methodology therefore covers a manageable number
of indicators that encompass all sectors of recovery. Users can choose which indicators to
monitor according to their own needs and resources.
When to measure?
Because of the dynamic nature of recovery, the timing and duration of different recovery
processes is likely to differ and the importance of different indicators is likely to vary. The
frequency that images are acquired also depends on whether the evaluation is ongoing or of
completed projects. To monitor the on-going progress of recovery, data should ideally be
acquired a few weeks after a disaster and every 6-12 months thereafter, while completed
project evaluations can be carried out using just two images: a pre-project image (baseline)
and a post-project image. Figure 1.2 shows some of the key processes and events visible in
satellite imagery with estimations of when they are likely to appear, relative to each other. In
some cases, the overall progress of recovery may be inferred by monitoring proxy indicators,
such as the presence or absence of transitional shelters.
How to measure?
Measurement tools may be divided into two broad categories: Direct Observation (e.g.
remote sensing and ground survey) and Social-Audit (e.g. focus group meetings, household
surveys and key informant interviews). The tools may be used independently of each other,
but are usually more effective if used in a complementary way.
This approach to monitoring and evaluation using satellite imagery was designed to be noncountry
or hazard specific. To test the transferability of the Indicators similar techniques were
applied to two case studies: Ban Nam Khem in Thailand and Muzaffarabad in Pakistan. The
case study sites differed markedly. Nevertheless, the case studies demonstrated that the
same indicators and similar approach could be applied in widely differing situations.
Exploitation Route see narrative impact summary
Sectors Digital/Communication/Information Technologies (including Software),Environment,Government, Democracy and Justice
URL http://www.martincentre.arct.cam.ac.uk/research/risk/rebuildd
 
Description Monitoring and evaluation of recovery and reconstruction after natural disasters can assist on-going aid effort, provide accountability and guide aid policy. The EPSRC-funded project Indicators for Measuring, Monitoring and Evaluating Post-Disaster Recovery (EP/F015232/1) , (2008-2011) (hereafter the Indicators Project) developed new guidelines which provide a systematic, independent and replicable approach to monitoring and evaluating the post-disaster recovery process. The indicators proposed in the guidelines were tested in case study research of Ban Nam Khem, Thailand which was hit by the 2004 Indian Ocean tsunami, and Chella Bandi, Muzaffarabad, AJK Pakistan, which was struck by the 2005 Kashmir earthquake. The indicators developed by the Indicators Project derive from a combination of earth observation using satellite images, ground surveys and interviews with affected populations. They encompass a range of physical, environmental, social and economic factors that combine to give an accurate picture of the reconstruction process at specific intervals. The Project resulted in the publication of a set of Guidelines which were presented at the Word Bank/ GFDRR World Reconstruction Conference in Geneva in May 2011, and subsequently made available to a wide range of stakeholders among them International Organisations (World Bank, UN-Habitat), national governments, and NGOs concerned with relief and reconstruction (International Committee of the Red Cross, British Red Cross, CARE etc). The impact of the Indicators study up to February 2016 is summarised in this report. The impacts include participation of the Indicators Project team in a number of further research projects to better understand the information needs of different stakeholders in the reconstruction process over time, to develop further the measurement tools proposed in the Indicators Project, and to review recovery and reconstruction efforts in different contexts. They include some significant applications of the monitoring tools developed in the context of both post-disaster recovery (Haiti and Pakistan) and in the monitoring and evaluation by humanitarian organisations of refugee camps resulting from conflicts in East Africa. They also include devising and planning scenario workshops for Emergency Management Departments in three countries (Turkey, Kyrgyzstan and Tajikistan) to help them better understand their information needs in early post-disaster situations. The work has had a positive impact on the attitude of the World Bank GFDRR to the application of geospatial tools in their risk management activities which can expect to be implemented in the future but, at this date, is yet to be fully implemented. This remainder of this report is divided into three sections: • further academic studies wholly or partly deriving from the Indicators Project • applications in practice • prospects for further application 2. Follow-on academic studies 2.1 EPSRC one-year (2011-2012) follow-on study "Commercialisation of satellite-based monitoring and evaluation data for long-term recovery" (EP/28609/1). This project (the RebuilDD project) aimed to bring the outcomes of the Indicators project closer to market application. Two field pilot projects in Haiti took place in the context of this project and two field studies on reconstruction in Chile and New Zealand (reported in Sections 3.1, 3.3 below). The project also included an intenational workshop on Geospatial tools and information needs in post-disaster recovery,held in Cambridge, which was attended by a variety of stakeholders from UN Agencies, NGOs and the Insurance Sector. The project also led to the establishment of the RebuilDD Consortium of UCAM, CAR Ltd and ImageCat Ltd which collaborated in formulating the SENSUM Project work programme. 2.2 EPSRC Urgent Application "Field survey following the 12.5.08 Wenchuan Earthquake in Sichuan Province, China (Urgent Application)" EP/G036365/1 (2008-09). Following the magnitude 7.9 Wenchuan earthquake, China, on 12 May 2008, the Cambridge members of the Indicators project team carried out damage/recovery surveys in October 2008, 6 months after the earthquake, and March 2009. The survey was carried out in collaboration with colleagues from Chongqing University, China, in seven locations: Dujiangyan, Yinxiu, Wenchuan, Pengzhou, Shifang, Hongbai and Beichuan. The damage observed in the towns of Yinxiu and Beichuan, which could be described as total destruction, was the most severe. The towns are located on the two ends of the ruptured fault line that extends approximately 240km. Beichuan, already damaged by the ground shaking, was hit by a devastating landslide three months after the earthquake which almost completely buried the town. Direct access to ongoing recovery of a major earthquake allowed the team to apply research from the Indicators project as the recovery and research were underway. http://link.springer.com/article/10.1007/s10518-011-9318-7 2.3 EU SENSUM Project (FP7, Seventh Framework Programme, THEME [SPA.2012.1.1-04] Support to emergency response management - Grant agreement no: 312972). The 2-year EU-funded collaborative SENSUM project (2013-2014) aimed to "deliver innovative methodologies and software tools for dynamic, multi-resolution monitoring of pre-disaster vulnerability and preparedness and post-disaster recovery planning and monitoring, based on current and future space-based products". The aims of Work Package 5 , led by UCAM, required the identification of various indicators relevant to earthquake risk that could be identified via earth observation products, as well as those that would need in-situ observations. A related aspect was the development of tools for change detection, which not only would be exploited for gauging changes in the built environment over time (i.e., a result of increasing urbanisation for example), but also for monitoring recovery operations during the post-event period (e.g., extent and changes in tent camps, road accessibility, etc.). In SENSUM the UCAM team used and further developed the same recovery indicators identified in the EPSRC Indicators Project. For the purpose of post-disaster recovery monitoring, UCAM developed semi-automated methods for users to use temporal analyses to identify focus areas followed by high-resolution data techniques to track specific changes for recovery and building vulnerability analyses. Based on the changes within the indicator-specific GIS objects in a high resolution image, indices can be calculated and plotted over time. The SENSUM method detects change, which is identified as the absence of similarity between gradient, roughness, texture and edges. For the purposes of recovery evaluation and monitoring, the method can be used to detect changes to buildings over time and therefore track the progress of initial damage, clearing, reconstruction and recovery due to an earthquake. One of the sites used in SENSUM for testing these algorithms was Muzaffarabad in Pakistan which was also studied in the Indicators project. The SENSUM Project and its results are elaborated at the project website: http://www.sensum-project.eu/ The CAR contributions to this project concerned the design and organisation of scenario planning exercises to identify user needs for information, which also derived directly from the work of the Indicators Project. Several of the CAR outcomes consisted of practical applications (reported below, Section 3.3). ImageCat's involvement in the SENSUM project focused on the development of data collection guidelines and protocols for risk and recovery mapping exercises. ImageCat's testing of the algorithms developed by UCAM and other project partners built on the work done in the Indicators Project through testing of semi-automated techniques on Muzzaffarabad for the extraction of buildings, roads and assessing multi-temporal changes. ImageCat's researchers also aided in the Pavia 2014 International Summer School on Data Fusion of Risk-related Remotely Sensed and Geospatial Data - a major dissemination and capacity-building event from SENSUM. ImageCat were close collaborators with its RebuilDD collaborators, CAR and UCAM, on the definition and running of the user needs workshops in Kyrgyzstan and Turkey. 2.2 Cambridge Architectural Research Ltd work on Recovery. Developing indicators to track recovery after major disasters in Pakistan and Thailand prompted CAR to attempt to better understand what is involved in successful recovery and to identify the underlying common challenges and success factors in a wider range of cases. Dr Platt from CAR took various opportunities to visit seven other countries, typically a year after the event. to observe progress, to interview key players and to conduct surveys of stakeholders and residents. In all, 10 recovery cases studies have been done after earthquake related events in Pakistan, Thailand, Chile, Italy, New Zealand, Japan, China, Turkey, Iran and the USA. This has led to a book to be published by Springer in 2016. Using remote sensing to monitor recovery also led on to theoretical work on how to conceptualize resilience and recovery, refining remote sensing analysis, including semi-automatic change detection and volunteered geographic information (crowdsourcing), and developing alternative approaches to monitoring recovery including ground based surveys, key informant interviewing, official reports and statistics and building control and insurance data. http://www.carltd.com/services/disaster-recovery 2.3 Related studies on Recovery at ImageCat The Indicators Project enabled ImageCat and ReBuilDD partners, CAR and UCAM, to develop a recovery-focused work package for the EC FP7 project, SENSUM. In addition to the technical and commercial inputs for this project, a number of other related projects have been undertaken. Following the 2010 Haiti earthquake, ImageCat were Principal Investigators on a RAPID grant from the US National Science Foundation, collaborating with researchers at the University of British Columbia, University of Delaware and University of Memphis: "Assessing Community-Scale Damage, Disruption, and Early Recovery in Post-Earthquake Haiti" Award No. CMMI - 1034876. This study used remote sensing to identify community disruption in seven affected communities of Haiti following the 2010 earthquake and establish the progress of recovery in the first five months following the disaster. The findings were presented on Capitol Hill in September 2011 as part of an NSF Hazards Research Showcase to the Senate and Congress. ImageCat was the industrial partner on two related MSc projects as part of the MSc Applied Geographical Information Science and Remote Sensing MSc course at the University of Southampton. ImageCat provided mentoring, technical guidance, access to data and domain expertise to support the projects. The Indicators Project provided theoretical background to both studies, which built on the practise of recovery monitoring using remote sensing in Pakistan and Haiti by Matthew Simpson (2011) and Sarah Pyatt (2010) respectively (project details are provided in reference section). Both students achieved distinctions in their dissertations. Two ImageCat researchers joined the Earthquake Engineering Field Investigation Team's (EEFIT) return mission to L'Aquila, Italy in 2012. Much of this was field work to identify progress in recovery. The EEFIT mission fed into studies at the University of Pavia into L'Aquila recovery, in which remote sensing has played a central part. ImageCat organised a joint GEM-SENSUM event at the 2014 Understanding Risk Forum in London in July 2014 where a cut down version of the scenario planning exercise was used to establish the use of geospatial data to support post-earthquake recovery. In 2014, Dr John Bevington from ImageCat was an invited participant in a consultation on a Guide to Developing Disaster Recovery Frameworks as part of a World Bank-EC-UNDP initiative to provide guidelines for developing frameworks for measuring and monitoring recovery (https://www.gfdrr.org/recoveryframework). To this point, the issue of evidence-based tracking of recovery using geospatial data had not been considered in consultations, and the work done in the Indicators Project and SENSUM were described. The use of such data is now being considered as a component of resilient recovery. These guidelines are currently being proposed as a method for initiating a measured approach to recovery following the April 2015 Nepal earthquake. 2.4 Follow-up studies at the University of Peshawar, Pakistan Several academic studies of the reconstruction following the 2005 Kashmir Earthquake (one of the two case-studies in the Indicators Project) have been carried out at the University of Peshawar under Professor Amir Khan who was a member of the Indicators Project Team. One of his students has recently completed an M.Phil on "Post October 2005 earthquake: Evaluation of housing reconstruction in District Mansehra, Pakistan", 2015. Two PhD scholars will soon complete their dissertations on the following topics: • Monitoring livelihood recovery in October 2005 Earthquake affected area of Northern Pakistan • An evaluation of housing reconstruction programme in 2009-10 conflict affected area of District swat, Khyber Pakhtunkhwa, Pakistan. 2.5 Follow-up studies at Memorial University, Canada. Dr Ratana Chuenpagdee at Memorial University who was a member of the project team in Ban Nam Khem in Thailand has incorporated findings from the Indicators project in presentations about coastal resilience and uses it as an example to talk about the importance of mangrove forests. In a current project, one research cluster focusses on how fishing communities respond to change (environmental and others). The outcomes of the Indicators project on environmental indicators has influenced the thinking on this topic as well. 3. Applications in practice 3.1 Application to humanitarian post-earthquake recovery work in Haiti In the year following the conclusion of the Indicators Project, the Project Officer, Daniel Brown worked in Haiti with both the British Red Cross and with UN-Habitat, in the context of recovery following the 2010 Haiti earthquake, on separate projects each of which made direct applications of the outcomes of the Indicators Project. For the British Red Cross the work was to help BRC survey the site, Demas19, which was the focus of its recovery and reconstruction efforts, and help compile social, physical and legal datasets about the settlement and its housing, making use of a combination of earth observation and ground survey techniques. More on this project can be found through these links: http://blogs.redcross.org.uk/emergencies/2014/01/haiti-earthquake-interactive-map/ http://www.redcross.org.uk/What-we-do/Emergency-response/Recovering-from-disasters/Haiti-earthquake-2010/Recovery/Shelter With UN-Habitat Daniel Brown used automated remote sensing techniques to monitor spontaneous settlement across a large area called Canaan to the north of the capital Port-au-Prince. Following this study agencies, notably American Red Cross, were able to commit many millions of $US ($12 million by the US Red Cross alone) to support self-build reconstruction in this area. The following links describe this work in general terms: http://unhabitat.org/haiti/ http://www.redcross.org/what-we-do/international-services/haiti-assistance-program 3.2 Application to humanitarian work in Somalia and Southern Sudan In 2012-2015, Daniel Brown was working for REACH-Initiatives on projects in East Africa: the work concerned the monitoring of refugee camps in Southern Sudan and Somalia. REACH is set up to support the humanitarian sector using modern information technology with tailored assessment methodologies, tools and processes that are adapted to specific emergency contexts and to the requirements of humanitarian organisations. http://www.reach-initiative.org/reach/about-reach/what-we-do In South Sudan, he used the knowledge gained from the Indicators Project to support the monitoring and assessment of refugee camps using a combination of remote sensing analysis and ground field work. Outputs from this work included maps, census data and thematic reports to support decision-making. This work supported organisations such as UNHCR, Danish Refugee Council, IOM, Solidarites International and others. http://www.reachresourcecentre.info/system/files/resource-documents/reach_ssd_report_ajoung_thok_relocation_dec2014_1.pdf Whilst in a regional role in Nairobi, Kenya Brown used the knowledge and techniques gained from the project to continue support to South Sudan throughout the conflict that began in 2013. http://www.reach-initiative.org/where-we-work/ongoing-field-presence/south-sudan Brown also supported rapid, remote assessments of camps in Somalia with remote sensing, and led a project leading to the development of a Monitoring & Evaluation (M&E) framework and tools for the Somalia Shelter Cluster (headed by UNHCR and DRC). This helped to standardise the M&E process for many small, Somalian agencies operating in the country. A detailed evaluation of a permanent shelter project was also conducted, again using remote sensing and also mobile phone technology. http://www.reach-initiative.org/where-we-work/ongoing-field-presence/somalia http://www.reachresourcecentre.info/system/files/resource-documents/reach_som_map_somaliadisplacementcrisis_luuq_waterdistance_26sep2014_a3_v2.pdf 3.3 Scenario Planning exercises for Government Emergency Management Departments in Turkey, Kyrgyzstan and Tajikistan The SENSUM project analysed the data needs of the end users through scenario planning exercises with three civil protection partners, AFAD in Turkey and the Government Emergency Management Departments in Kyrgyzstan and Tajikistan. Dr Stephen Platt of CAR Ltd ran the two-day scenario planning exercises with emergency management personnel in Kyrgyzstan/Tajikistan and with AFAD personnel and municipal urban planners in Turkey. These workshops led to a much richer understanding of these users' information needs in managing response and steering early recovery, both of which have a big influence on long-term recovery and development. The following links indicate how AFAD's subsequent work has been influenced by these exercises: http://www.ipkb.gov.tr/Dosyalar/EBulten/ISMEP_BULTEN_07_ENG.pdf https://www.afad.gov.tr/en/HaberYazdir.aspx?ID=5&IcerikID=5106 3.4 Multi-hazard and risk assessment for Provincial Disaster Management Authority, Khyber Pakhtunkhwa, Pakistan Professor Amir Khan is working with UN-Habitat Pakistan in collaboration with the Provincial Disaster Management Authority (PDMA) Khyber Pakhtunkhwa , which has engaged the Asian Disaster Preparedness Center (ADPC) and its partner, Community Resilience Initiatives (CRI) to provide consulting services for supporting the Government of Khyber Pakhtunkhwa in developing more synergized, cohesive and inclusive disaster risk management (DRM) policies. As a team member of CRI team, he is carrying out a Multi-Hazard Vulnerability and Risk Assessment (MHVRA) of two selected Districts namely Chitral and Dera Ismail Khan of Khyber Pakhtunkhwa Province. The project is being carried out with the involvement of government, academic and scientific offices. The project activities also adapted strategies for transfer of knowledge and experience to local counterparts through the meetings and workshops. Many of the techniques developed through the Indicators Project are being put into practice in this work. 4. Prospects for further application 4.1 GFDRR, the World Bank. GFDRR, The Global Facility for Disaster Risk Reduction is the World Bank's specialist team on all aspects of Disaster Risk Reduction. GFDRR wrote a letter of support for the application for the Indicators Project, and its results were reported at a World Bank/ UN workshop in Geneva in April 2011. Keiko Saito, who was RA in the Indicators Project team is now a Staff member of the GFDRR in Washington, where her role is to provide technical assistance to client countries in leveraging latest technologies and innovation, as well as on the effective application of geospatial data and analysis for informed decision making in disaster risk management. https://www.gfdrr.org/meet-our-team Keiko Saito reports that the outputs from the Indicators and Indicators project were very useful in demonstrating the potentials of what can be done with satellite images and geospatial data for the longer term recovery monitoring and planning to an audience that have minimal understanding of the technology. Based on this, GFDRR Sustainable recovery team is currently engaged with the geospatial industry, in particular the CEOS space agencies in scaling up the usage of satellite imagery for the longer term reconstruction planning, monitoring and evaluation. This initiative, called the Recovery Observatory (RO),started in 2015 and is led by the French Space Agency CNES. Due to budgeting issues the pilot project is being delayed. The Recovery Indicators developed through this project funded by EPSRC are likely to become the basis of the RO project. Through further demonstration, opportunities will be sought. As the World Bank operates on a demand basis rather than supply driven, further demonstration opportunities will be sought to generate client country's needs for such tools through our operations. As there is little room to undertake research at the World Bank, these opportunities to pursue new ideas and demonstrate their potential through research projects are very much appreciated by GFDRR. The World Bank DRM community is also increasingly using geospatial data for its operations, including ex-ante risk assessments as well as ex-post disaster impact assessments, for which the longer term recovery monitoring demonstrated by the outputs from the EPSRC funded Indicators and RebuilDD projects will be the next area of focus. (RS 22.2.2016)
First Year Of Impact 2010
Sector Construction,Digital/Communication/Information Technologies (including Software),Environment,Government, Democracy and Justice
Impact Types Cultural,Societal,Policy & public services
 
Description EPSRC
Amount £173,972 (GBP)
Funding ID RG61366 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start