Impact assessment based on self-reported attribution in complex contexts of rural livelihood transformations in Africa.
Lead Research Organisation:
University of Bath
Department Name: Social and Policy Sciences
Abstract
There is a growing demand for impact evidence within development agencies, both to support their own learning and to account responsibly to other stakeholders. Internal M&E systems are an important foundation for such assessment but it is widely recognised that they can usefully be complemented by periodic independent impact assessment (IA). However, doubts remain over the reliability and/or cost-effectiveness of much IA, prompting ongoing experimentation and academic debate over alternative approaches.
The proposed research addresses the particularly difficult issue of how to assess the impact of development activities intended to benefit poor men and women caught up in complex processes of rural transformation in Africa. The core issue addressed is how to attribute change specific interventions in a way that is reliable, timely and cost-effective to different stakeholders, without at the same time distorting or constraining the development activity being assessed.
While the impact attribution problem is pervasive in development practice, the research will be focus particularly on the work of two NGOs: Self Help Africa (SHA) and Farm Africa (FA), and their ongoing activities in Malawi and Ethiopia. More specifically the research will assess scope in such contexts for rigorous and cost-effective impact assessment based on (a) systematic monitoring of economic security at the household level combined (b) with in-depth interviewing designed to elicit self-reported attribution by respondents. This approach can be contrasted with both participatory approaches to impact assessment, and those that rely on statistically inferred attribution using either randomised control trials or quasi-experimental methods.
The proposed research comprises three complementary strands.
Strand 1: four baseline studies (two in Malawi, two in Ethiopia) followed by two rounds of programme monitoring studies conducted by the SHA and FA with specialist technical support from the NGO Evidence for Development (EfD). This will draw upon and enhance EfD's pioneering work in developing the Integrated Household Method (IHM) for measuring and modelling the reslience of household level cash income and food consumption.
Strand 2: two rounds of impact studies by independent consultants, using a qualitative individual impact protocol (QUIP) that combines Strand 1 monitoring with semi-structured interviews based on a self-reported attribution methodology.
Strand 3: two rounds of meta-evaluation, through which the quality of evidence obtained through Strands 1 and 2 will be reviewed, and additional evidence collectied of how it was interpreted and utilised by different stakeholders, and how it affected their working relationships. A particular focus here is on what evidence obtained through Strand 2 added to that obtained from Strand 1 and whether this justified the extra cost.
The research will thereby contribute to improving feedback loops in support of improved development interventions to support rural livelihood transformation in Africa, with potential methodological benefits in support of aid effectiveness more generally.
In addition to the core methodological goals of the research, the research will also address immediate and substantive questions facing the two participating NGOs in their work in Ethiopia and Malawi, relevant also to other organisations carrying out similar activities. More specifically, SHA and FA are seeking to improve their understanding of what combination of interventions are most effective in improving the economic and food security of poor rural households in the two countries. Activities to be investigated include promoting improved seeds and cultivation practices, animal husbandry, supply chain upgrading, natural resource management, microfinance, and diversification into non-farm activities.
The proposed research addresses the particularly difficult issue of how to assess the impact of development activities intended to benefit poor men and women caught up in complex processes of rural transformation in Africa. The core issue addressed is how to attribute change specific interventions in a way that is reliable, timely and cost-effective to different stakeholders, without at the same time distorting or constraining the development activity being assessed.
While the impact attribution problem is pervasive in development practice, the research will be focus particularly on the work of two NGOs: Self Help Africa (SHA) and Farm Africa (FA), and their ongoing activities in Malawi and Ethiopia. More specifically the research will assess scope in such contexts for rigorous and cost-effective impact assessment based on (a) systematic monitoring of economic security at the household level combined (b) with in-depth interviewing designed to elicit self-reported attribution by respondents. This approach can be contrasted with both participatory approaches to impact assessment, and those that rely on statistically inferred attribution using either randomised control trials or quasi-experimental methods.
The proposed research comprises three complementary strands.
Strand 1: four baseline studies (two in Malawi, two in Ethiopia) followed by two rounds of programme monitoring studies conducted by the SHA and FA with specialist technical support from the NGO Evidence for Development (EfD). This will draw upon and enhance EfD's pioneering work in developing the Integrated Household Method (IHM) for measuring and modelling the reslience of household level cash income and food consumption.
Strand 2: two rounds of impact studies by independent consultants, using a qualitative individual impact protocol (QUIP) that combines Strand 1 monitoring with semi-structured interviews based on a self-reported attribution methodology.
Strand 3: two rounds of meta-evaluation, through which the quality of evidence obtained through Strands 1 and 2 will be reviewed, and additional evidence collectied of how it was interpreted and utilised by different stakeholders, and how it affected their working relationships. A particular focus here is on what evidence obtained through Strand 2 added to that obtained from Strand 1 and whether this justified the extra cost.
The research will thereby contribute to improving feedback loops in support of improved development interventions to support rural livelihood transformation in Africa, with potential methodological benefits in support of aid effectiveness more generally.
In addition to the core methodological goals of the research, the research will also address immediate and substantive questions facing the two participating NGOs in their work in Ethiopia and Malawi, relevant also to other organisations carrying out similar activities. More specifically, SHA and FA are seeking to improve their understanding of what combination of interventions are most effective in improving the economic and food security of poor rural households in the two countries. Activities to be investigated include promoting improved seeds and cultivation practices, animal husbandry, supply chain upgrading, natural resource management, microfinance, and diversification into non-farm activities.
Planned Impact
There is continued and growing demand for high quality evidence of the impact of development activities, both in countries where such activities are taking place and in those that provide financial and technical support. In the context of some scepticism about "what works" demand from the general public is also important (e.g. Horton, 2010).
The project aims to have an impact on methodological debates over appropriate forms of M&E and impact assessment at various levels: (a) within SHA and FA, and through them within the NGO networks in which they participate; (b) among other development agencies engaged in work similar to SHA and FA; (c) within academia - see note on academic beneficiaries; (d) within aid policy networks within the UK and globally - e.g. through policy forums run by ODI, 3ie, Oxfam; (e) at the national level within the two countries where field work will be undertaken; (f) among the general public.
Positive impact on methodologies for strengthening feedback and organisational learning within development agencies and along the aid chain should result in improved rural development practices and outcomes both within the countries where the research is undertaken and potentially more widely.
There is widespread uncertainty about the effectiveness of alternative strategies and modalities for promoting household level economic security in the context of complex rural livelihood transformations: over the terms of smallholder participation in national and global value chains, for example. While the primary goal of the research is methodological, the pilot studies will also deliver more immediate and tangible benefits in terms of better understanding of the impact of the specific rural development activities of SHA and Farm Africa. These insights will impact most immediately on their own programmes in each country and more widely across the other countries across Africa where they work. They should also contribute to national and global debates over rural development strategies more widely.
The impact of both methodological and substantive empirical findings will be maximised through networks at global, national and local levels. Given the primacy that SRA based IA places on the validity of beneficiary perceptions of change and attribution of impact, ensuring feedback and enabling dialogue within the communities where the studies are undertaken will also be important. The NGOs conducting the research will lead in supporting communities to use the information to increase understanding of the relative impact of specific development activities in their locality for different community members. This will help inform decisions and be used in participatory planning processes.
The attached "Pathways to impact" plan sets out target audiences, forms of engagement, outputs and potential impacts more fully.
The project aims to have an impact on methodological debates over appropriate forms of M&E and impact assessment at various levels: (a) within SHA and FA, and through them within the NGO networks in which they participate; (b) among other development agencies engaged in work similar to SHA and FA; (c) within academia - see note on academic beneficiaries; (d) within aid policy networks within the UK and globally - e.g. through policy forums run by ODI, 3ie, Oxfam; (e) at the national level within the two countries where field work will be undertaken; (f) among the general public.
Positive impact on methodologies for strengthening feedback and organisational learning within development agencies and along the aid chain should result in improved rural development practices and outcomes both within the countries where the research is undertaken and potentially more widely.
There is widespread uncertainty about the effectiveness of alternative strategies and modalities for promoting household level economic security in the context of complex rural livelihood transformations: over the terms of smallholder participation in national and global value chains, for example. While the primary goal of the research is methodological, the pilot studies will also deliver more immediate and tangible benefits in terms of better understanding of the impact of the specific rural development activities of SHA and Farm Africa. These insights will impact most immediately on their own programmes in each country and more widely across the other countries across Africa where they work. They should also contribute to national and global debates over rural development strategies more widely.
The impact of both methodological and substantive empirical findings will be maximised through networks at global, national and local levels. Given the primacy that SRA based IA places on the validity of beneficiary perceptions of change and attribution of impact, ensuring feedback and enabling dialogue within the communities where the studies are undertaken will also be important. The NGOs conducting the research will lead in supporting communities to use the information to increase understanding of the relative impact of specific development activities in their locality for different community members. This will help inform decisions and be used in participatory planning processes.
The attached "Pathways to impact" plan sets out target audiences, forms of engagement, outputs and potential impacts more fully.
Organisations
- University of Bath (Lead Research Organisation)
- Tearfund (Collaboration)
- Evidence for Development (Collaboration)
- Save the Children (Collaboration)
- Bristol City Council (Collaboration)
- Oxfam GB (Collaboration)
- Bath Social & Development Research Ltd (Collaboration)
- Habitat for Humanity (Collaboration)
- Aga Khan Foundation (Collaboration)
- FARM AFRICA (Collaboration)
- F1F9 India Private Limited (Collaboration)
- SELF HELP AFRICA (UK) (Collaboration)
- SETsquared Partnership (Collaboration)
- Diageo (Collaboration)
- C and A Foundation (Collaboration)
People |
ORCID iD |
| James Copestake (Principal Investigator) |
Publications
Copestake J
(2020)
Case selection for robust generalisation: lessons from QuIP impact evaluation studies
in Development in Practice
Copestake J
(2014)
Credible impact evaluation in complex contexts: Confirmatory and exploratory approaches
in Evaluation
Copestake J
(2019)
Attributing Development Impact
Copestake J
(2019)
The Veil of Ignorance Process Tracing (VoiPT) methodology
Copestake J
(2015)
Mixed Methods Research in Poverty and Vulnerability
Copestake J
(2020)
Recently graduated midwives in Uganda: Self-perceived achievement, wellbeing and work prospects.
in Midwifery
Copestake J
(2018)
Managing relationships in qualitative impact evaluation of international development: QuIP choreography as a case study
in Evaluation
Copestake, J
(2016)
QuIP: Directices para el administrador
| Description | Summary. As planned, we designed an innovative qualitative impact assessment methodology, called the QuIP, for impact assessment of NGO interventions on rural livelihoods and food security at the household level in the context of rapid and complex rural transformations taking place across Africa. The methodology was successfully tested on four projects alongside project monitoring using the individual household method (IHM) designed by Evidence for Development (EfD). This revealed household level changes and impacts that were highly context dependent and heterogeneous. What are the four most significant achievements from the grant? Achievement 1. New knowledge generated about changes in livelihoods and wellbeing in complex rural contexts in Ethiopia and Malawi, and their causes. Project changes over a two year period in livelihood and food security indicators were obtained for a longitudinal survey sample of 172 households using the IHM. Plot QUIP studies included semi-structured interviews with 48 households and 16 focus group discussions (four per project). The final round of QUIP studies covered 98 households, all of whom were also part of the baseline IHM sample survey. Detailed findings are summarised in working papers. For each project these comprise an IHM baseline survey report, an IHM endline report, Pilot and Final QUIP reports. The data reveals very wide variation in households' livelihood characteristics and wellbeing indicators between projects, within them and over time. They also reported on a wider range of positive and negative drivers of change during the two year period. Evidence on short-term project impact (both explicit and implicitly supporting project theories of change) was largely positive, but not always sufficient to offset exogenous negative drivers . Achievement 2. Development of the QUIP - a new and improved approach to qualitative impact assessment of rural development interventions in complex contexts The QUIP methodology was successful in generating substantial impact evidence. While broadly based on a combination of contribution analysis and process tracing, the research pioneered innovative features. First, the project successfully tested the feasibility of blindfolding field researchers and respondents from knowing the precise purpose of interviews (beyond finding out about local livelihood changes), thereby reducing the threat of pro-project confirmation bias associated with self-reported attribution methodologies. Second, a substantial amount of work went into developing a highly transparent, auditable and efficient way of organising the impact evidence obtained so as to facilitate quick and easy interpretation and lesson learning. This entailed substantial investment in development of Excel spreadsheets and associated protocols to ensure consistent and useful coding. This investment goes a long way to reduce the 'black box' characteristic of much qualitative research. Achievement 3. Insights gained into implementation of mixed methodologies: how and how not to integrate quantitative and qualitative approaches to monitoring and impact assessment. The project was successful in generating a longitudinal dataset that enables data from the IHM and QUIP studies to be combined and cross-analysed (work that is still ongoing). However, the two methods proved more complicated to integrate than hoped. While the data is useful in highlighting heterogeneity at the household level, it is harder to assess the relevance of the findings to the wider population of intended beneficiaries of the projects than hoped. Lighter touch quantitative monitoring of key proxy indicators could have established a stronger frame from which to select QUIP samples Achievement 4. Successful model identified for building within-country collaborative relationships for impact evaluation. An unanticipated lesson gained from the project relates to the process of commissioning impact evaluation and building the pool of researchers available to conduct them. The QuIP studies required that a third party (in this case the University of Bath) recruits local researchers who were not known to the evaluand. However, once data collection and analysis was completed the dialogue between field researchers, project staff and analysts (which took place at country level workshops) proved both fruitful in itself and effective in forging new relationships between externally funded NGOs and local universities. |
| Exploitation Route | Findings have been taken forward and used by others in three ways. FIRST substantive findings from the four pilot studies were internalised by the participating NGOs: forming part of their final evaluations to donors and contributing to future programming. During the research period the PI joined the Food, Nutrition and Income Security Advisory Group of one of the collaborating NGOs (Self Help Africa) and has played a role in advising it in design, implementation and evaluation of a range of subsequent rural development programmes across Africa. SHA has also employed the QuIP in subsequent impact evaluations in Kenya, Zambia and Burkina Faso. SECOND development of the QUIP has proved timely in relation to growing interest in qualitative impact evaluation as both complement and alternative to more established quantitative approaches. Interest has centred particularly on the wider relevance of our experiences with blindfolded data collection, and semi-automation of thematic data analysis and visualisation. The PI has continued to contribute to debate over how to impact evaluation, not least as a member of the intellectual leadership team of the Centre for Development Impact and Learning, launched in 2017 with funding from the UK Department of International Development. Interest in the QUIP has been reflected in substantial demand to share our experiences with researchers, evaluators and development agencies who are also exploring scope for improved qualitative impact evaluation. Conference presentations, seminars, workshops and meetings (in Canada, Ethiopia, France, Greece, Ireland, Malawi, Netherlands, South Africa, Sweden, Switzerland, the UK and Zambia) are listed separately and have led to discussions how to adapt and use the QuIP with many NGOs, aid agencies, consultancies and firms. Global virtual seminars have also been organised with M&E staff of the Aga Khan Development Foundation, DFID and Save the Children. Partly as a result other agencies have adopted all or part of the QuIP methodology - leading examples being Acumen, OPM and WYG. THIRD In response to interest in the QuIP the original research led directly to formation of Bath Social and Development Research - as a spin-off social enterprise (see www.bathsdr.org). In its first three years of operation it completed more than forty QuIP studies for more than fifteen development organisations across more than fifteen countries. A book and several journal articles have also been produced that draw on the original research as well as this follow up work. |
| Sectors | Agriculture Food and Drink Communities and Social Services/Policy Digital/Communication/Information Technologies (including Software) Education Environment Financial Services and Management Consultancy Healthcare Other |
| URL | http://go.bath.ac.uk/art |
| Description | We are beginning to have a significant impact on thinking and practice concerning the use of qualitative methods to assess the impact of development programmes and projects. Our initial target was to do this among NGOs engaged in rural livelihood promotion, including climate adaptation projects. Further financial support was obtained to enable us to engage also with impact investors (see ES/NO15649/1) In addition to direct engagement through conference presentations, seminars, workshops and consultations (including the Evaluation Dept of DFID) we have set up a non-profit spin-off company called Bath SDR Ltd to promote use of the qualitative impact protocol (QuIP) which was the main methodological product of the project. This mainstreaming work is gaining momentum - see entry for Bath SDR under Collaborations for more details of the many studies it has completed. This company has now conducted nearly forty QuIP evaluation studies in nearly twenty countries, and the QuIP has also influenced thinking about impact evaluation methodology more widely. |
| First Year Of Impact | 2017 |
| Sector | Agriculture, Food and Drink,Communities and Social Services/Policy,Education,Environment,Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice |
| Impact Types | Societal Economic Policy & public services |
| Description | Adoption of QuIP by OPM in concurrent evaluation in India |
| Geographic Reach | Asia |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | OPM are lead consultants for concurrent evaluation of a DFID programme to promote socially responsible financial inclusion across priority states in Northern India. The team decided to incorporate aspects of the QuIP in design of mid-line and end-line impact evaluation. |
| Description | Citation in UN Working paper on evaluation of the SDGs (page 26, paragraph 125) |
| Geographic Reach | Multiple continents/international |
| Policy Influence Type | Implementation circular/rapid advice/letter to e.g. Ministry of Health |
| URL | http://www.uneval.org/document/detail/1912 |
| Description | E-mail endorsement of QUIP by DFID staff 13 October 2015 |
| Geographic Reach | National |
| Policy Influence Type | Implementation circular/rapid advice/letter to e.g. Ministry of Health |
| Description | INTRAC Board Membership |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Invited membership of the Intellectual Leadership Team of DFID's new Centre for Excellence in Development Impact and Learning (CEDIL) |
| Geographic Reach | National |
| Policy Influence Type | Membership of a guideline committee |
| Description | Membership of Programme Advisory Board of Gorta Self Help Africa |
| Geographic Reach | Africa |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Ongoing role as advisor to Gorta Self Help Africa, an Irish and UK registered development agency dedicated to improving livelihoods and food security of poor and vulnerable households across Sub-Saharan Africa. Quarterly meetings with responsibility to review strategy papers, proposals, evaluations and reviews. Spin-offs have included brokering research collaboration between GSHA and the Dept of Biology and Biochemistry at the University of Bath. |
| URL | https://selfhelpafrica.org/ie/ |
| Description | Participation in Wilton Park meeting on new frontiers in evaluation |
| Geographic Reach | Multiple continents/international |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Invited participant at an international workshop on "new frontiers for evaluation in an era of market-oriented development" Wilton Park, 20-22 July. |
| Description | ESRC-DFID Impact and Engagement |
| Amount | £73,782 (GBP) |
| Funding ID | ES/N015649/1 |
| Organisation | Economic and Social Research Council |
| Department | ESRC-DFID Joint Fund |
| Sector | Public |
| Country | United Kingdom |
| Start | 12/2015 |
| End | 03/2016 |
| Description | ICURe |
| Amount | £35,000 (GBP) |
| Funding ID | QUIP 514603306 |
| Organisation | Innovate UK |
| Sector | Public |
| Country | United Kingdom |
| Start | 08/2015 |
| End | 04/2016 |
| Title | Qualitative impact Protocol (QuIP) |
| Description | The QuIP is primarily an evaluation method, which might also be classified as a form of applied social science research incorporating aspects of process tracing and contribution analysis in particular. However, as both an exploratory and confirmatory evaluation method it does incorporate some innovative features that are relevant to purer or academic social science research methods (e.g. in relation to methods to reduce confirmation bias and improve data coding). In addition to pursuing opportunities for commercialising the QuIP in development practice and among impact investors, we have also presented it to more academic audiences, and are planning applications to more academic work (e.g. through Maconachie's new project for Humanity United) on assessing wellbeing along informal mining value chains in West Africa). |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2015 |
| Provided To Others? | Yes |
| Impact | The QuIP has attracted a lot of interest among other researchers, and been presented at several conferences (including in Bordeaux, London, Manchester, Pretoria, Stockholm) and through invited seminars (including for DFID, Save the Children, Oxfam UK, OPM Ltd, the National Audit Office and the Aga Khan Development Network). Other research bodies have incorporated it into research proposals, including teams at the Overseas Development Institute, OPM, the International Institute for Environment and Development. A non-profit company Bath Social and Development Research Ltd, has been set up specifically to promote QuIP and has used it to conduct commissioned impact evaluation studies for Self Help Africa (in Kenya), Oxfam GB (in Ethiopia) and Diageo (in Ethiopia). More are planned |
| URL | http://qualitysocialimpact.org/ |
| Title | ART project data archive UK Data (QuIP) |
| Description | Offical and comprehensive archive of QuIP data collected under the ART project for UK Data |
| Type Of Material | Database/Collection of data |
| Year Produced | 2016 |
| Provided To Others? | Yes |
| Impact | None to date |
| URL | http://reshare.ukdataservice.ac.uk/852065/ |
| Title | ART project data archive UK data (IHM) |
| Description | Complete archive of "Individual Household Method" data collected under the ART project. Submitted to UK Data in line with required guidelines |
| Type Of Material | Database/Collection of data |
| Year Produced | 2016 |
| Provided To Others? | Yes |
| Impact | IHM data from ART project |
| URL | http://reshare.ukdataservice.ac.uk/852064/ |
| Title | Impact assessment complex contexts of rural livelihood transformations in Africa. Part 1- Longitudinal household income data |
| Description | |
| Type Of Material | Database/Collection of data |
| Year Produced | 2016 |
| Provided To Others? | Yes |
| Title | Impact assessment in complex contexts of rural livelihood transformations in Africa. Part 2- Interview data |
| Description | |
| Type Of Material | Database/Collection of data |
| Year Produced | 2016 |
| Provided To Others? | Yes |
| Title | Individual Household Method |
| Description | Evidence for Development's Individual Household Method (IHM) is a method for collecting quantitative monitoring data at a household level with detailed information on household assets, demography and specific income sources. |
| Type Of Material | Database/Collection of data |
| Provided To Others? | No |
| Impact | The IHM model is already available for use to those who contract EfD, however, this project is funding the development of changes to the software and the research model to enable better comparison of household level changes over time and comparison with qualitative data collected through the QUIP. |
| URL | http://www.efd.org/our-work/methods/the-individual-household-method-ihm/ |
| Title | QUIP Analysis |
| Description | The development of a bespoke qualitative data analysis technique is key to the Assessing Rural Transformations project, and this is closely tied in with the development of the bespoke software within Excel, referred to elsewhere in Outputs. The analysis technique has been described in detail in a published paper - 'Assessing rural transformations: piloting a qualitative impact protocol in Malawi and Ethiopia'. |
| Type Of Material | Data analysis technique |
| Year Produced | 2014 |
| Provided To Others? | Yes |
| Impact | We are engaging with interested practitioners to gain feedback on the model, and have offered to support further trials with any interested practitioners. The analysis methods is continuously being upgraded through interaction with F1F9 |
| URL | http://www.bath.ac.uk/cds/projects-activities/assessing-rural-transformations/documents/art-piloting... |
| Title | QUIP data collection |
| Description | The Qualitative Impact Protocol (the QUIP) is the main output of this funded project, and is now in its second round of testing. The questionnaire is now publicly available for use with the caveat that it is still in development. The bespoke Excel spreadsheet that the data is collected in is still in a Beta stage, ans can be made available to those who request it, alongside training to use it. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2014 |
| Provided To Others? | Yes |
| Impact | This has only just been made public (September 2014) and as yet has not been taken up by any other researchers. |
| URL | http://cdsblogs.wordpress.com/2014/09/20/the-state-of-the-art-project-update-2/ |
| Description | Aga Khan Development Network |
| Organisation | Aga Khan Foundation |
| Country | Switzerland |
| Sector | Charity/Non Profit |
| PI Contribution | BSDR has committed to providing training and consultancy services to AGDN to enable it to use and/or adapt the QuIP for internal purposes. A collaborative QuIP based study was conducted of midwifery graduates from the Aga Khan University in Uganda. More joint studies are planned for 2019. |
| Collaborator Contribution | AGDN financed a public lecture on the QuIP by James Copestake in Ottawa. They have also supported QuIP training undertaken by BSDR. The Uganda study led to a joint publication, submitted to the Journal of Midwifery. |
| Impact | Webcase of a public lecture: Cautionary tales of complex causation: Qualitative and mixed method impact assessment of climate change and livelihood transformations in Africa. |
| Start Year | 2016 |
| Description | Bath SDR Ltd |
| Organisation | Bath Social & Development Research Ltd |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | BSDR was set up in 2016 as an independent non-profit company to promote further development and use of the QuIP methodology. A legal agreement was signed between BSDR and the University of Bath in 2016 under which Bath SDR has rights to promote the QuIP under licence. During 2016 Bath SDR commissioned QuIP studies in Ethiopia, India, Kenya, Mexico and Uganda. Since then additional studies were conducted in a further six countries, and again . Actual and pipeline clients include Acumen, Aga Khan Development Network, Bristol City Council, the C&A foundation, Diageo Ltd, Gorta Self Help Africa, Habitat for Humanity, Oxfam GB, Save the Children Fund, Tearfund, Tree Aid. As of early 2021 the company employs two staff full-time and a growing network of part-timers, both in the UK and abroad. I am a member of the Board of Directors, and am also contracted to BSDR by the University to advise and support its activities. Bath SDR pays a licence fee to the University of Bath of 1% of its revenue for permission to utilise the intellectual property developed during the ART project. |
| Collaborator Contribution | In addition to mainstreaming use of the QuIP BSDR is building up an archive of experience with use of the QuIP to conduct qualitative impact evaluation of development projects that will inform further research under CDS. These form the core of the recently published book - Copestake et al. (2019). |
| Impact | BSDR has asssisted in producing various internal reports for other collaborators listed. |
| Start Year | 2016 |
| Description | Bristol City Council |
| Organisation | Bristol City Council |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | BSDR advised BCC on innovative ways to evaluate its support for community led activities across the city. |
| Collaborator Contribution | They commissioned a pilot study in 2016 and a fuller study in 2018, drawing on the QuIP methodology. |
| Impact | The pilot study is documented as a case study in Chapter 9 of Copestake et al (2019) |
| Start Year | 2016 |
| Description | C and A Foundation |
| Organisation | C and A Foundation |
| Country | Switzerland |
| Sector | Private |
| PI Contribution | C&A commissioned BSDR Ltd to conduct a QuIP study of a programme to improve labour standards in factories supplying C&A with ready made garments in Mexico. The final report was published on the web in 2017 - see http://www.candafoundation.org/impact/results-learning/evaluation-c-foundation-yo-quiero-yo-puedo-cuidarm/ |
| Collaborator Contribution | Commissioned QuIP study through BSDR. This is the basis of a case study in Chapter 4 of Copestake et al. (2019) |
| Impact | BSDR QuIP report https://d2bkwed3dzgijf.cloudfront.net/live/media/filer_public/4b/9e/4b9e20d0-5d41-4b7b-b567-a2df9d2a09f8/final_report_impact_evaluation_yqyp_programme_mexico_120617-vf_web_version.pdf |
| Start Year | 2015 |
| Description | Diageo Ltd (Ethiopia) |
| Organisation | Diageo |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | The research led to the Head of Corporate Social Responsibiity of Diageo commissioning QuIP studies of their smallholder malt barley procurement in Ethiopia from BSDR. This was completed in 2016, and a publication based on the findings (commissioned independently by Diageo) was published in 2017. Diageo saw the QuiP as a pilot "deep dive" to explore possible positive and negative effects of their smallholder barley procurement in Ethiopia at the household level and below. We found evidence of positive impact on household incomes and education. We also found some rising inequality, due to rising land rents and more seriously negative shocks to some farmers arising from credit being advanced against poor seed. This has prompted follow-up discussion about risk sharing mechanisms that Diageo might develop with their partners. Diageo commissioned a second QuIP study in Uganda in 2018, a third in Cameroon, and a fourth is planned (in Kenya). |
| Collaborator Contribution | The study will add to the database of QuIP applications from which we can learn. It is also a pioneering commission for BSDR - the spin-off social enterprise from the research. Diageo have contributed to funding a publication based on the report. This Ethiopia QuIP study also features as a case study chapter in Copestake et al, (2019). |
| Impact | Internal report submitted to Diageo by BSDR Ltd in 2016. Publication to follow. The study will also provide material for future CDS work on qualitative impact evaluation. |
| Start Year | 2016 |
| Description | Evidence for Development |
| Organisation | Evidence for Development |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | We paid Evidence for Development to undertake three rounds of quantitative monitoring in each of the four projects being assessed, using their Individual Household Method (IHM). We are also paid for further developments of the IHM software to enable better reporting of EfD results. |
| Collaborator Contribution | EfD worked with the University of Bath on ensuring that the QUIP works well with quantitative monitoring systems, and provided the data that we needed for triangulation, particularly in the final round of assessment. |
| Impact | All IHM data from the ART project has been archived along with the QuIP data in compliance with ESRC guidelines. |
| Start Year | 2012 |
| Description | F1F9 |
| Organisation | F1F9 India Private Limited |
| Country | India |
| Sector | Private |
| PI Contribution | We have worked with F1F9 consultants to design the outline of the spreadsheet required for analysis, based on the need for simple, open access software that any small NGO or consultant can use to analyse qualitative data. This has involved regular meetings, testing and feedback on each new design. |
| Collaborator Contribution | F1F9 have done all the coding and creation of the spreadsheet which we are now sharing with potential users, through BSDR Ltd. These include Acumen, Aga Khan Development Network, Tearfund and several self-employed consultants who have attended training. Much of the initial work undertaken by F1F9 was pro bono. |
| Impact | The main output is the QUIP spreadsheet, still undergoing further development so it is not publicly available yet. However, its development is charted int he blog post in the URL above. |
| Start Year | 2013 |
| Description | Farm Africa |
| Organisation | Farm Africa |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | Farm Africa was a named partner in the ART project (alongside Gorta Self Help Africa) and the QuIP was piloted on one of its programmes in Northern Ethiopia. |
| Collaborator Contribution | Farm Africa's senior evaluation officer provided field support and advice throughout the ART project. Farm Africa also convened the final project workshop in Ethiopia in August 2015. |
| Impact | The final reports on the Tigray project remained internal to Farm Africa |
| Start Year | 2012 |
| Description | Gorta Self Help Africa |
| Organisation | Gorta Self Help Africa |
| Country | Ireland |
| Sector | Charity/Non Profit |
| PI Contribution | As per the project proposal, the QuIP was designed and tested on three of Gorta Self Help Africa's projects in Ethiopia and Malawi (and one of Farm Africa's projects in Ethiopia - see separate entry). This involved us undertaking and paying for two rounds of impact assessment in each project, analysing the data and writing up a report for the project implementation team. The project also payed for three rounds of quantitative monitoring by Evidence for Development, using the Integrated Household Method (IHM). In 2016 (after completion of the ART project) GSHA commissioned BSDR to conduct an additional QuIP study of a cassava commercialisation project in Kenya. Further QuIP studies were commissioned in 2017 in Zambia and Burkina Faso. |
| Collaborator Contribution | Gorta Self Help Africaworked closely with BSDR on the development of the protocol, providing feedback and critical adjustments. They have since commissioned three follow-up QuIP evaluation studies in other countries. |
| Impact | The outputs are in the form of the reports, which are listed as outputs but which are at the moment confidential. The anonymised results, however, are available in the paper 'Assessing rural transformations: piloting a qualitative impact protocol in Malawi and Ethiopia.' |
| Start Year | 2012 |
| Description | Habitat for Humanity International |
| Organisation | Habitat for Humanity |
| Country | United States |
| Sector | Charity/Non Profit |
| PI Contribution | BSDR conducted a QuIP study for H4H relating to wholesale financing of microfinance for low cost housing improvement in South India. They also supported a further study incorporating QuIP in Peru in 2018. |
| Collaborator Contribution | They have funded two QuIP studies. The India study also features as a case study chapter in Copestake et al. (2019). |
| Impact | The India study contributed to HFHIs internal reflection on their mission and strategy both in India and more widely. This is documented in Chapter 5 of Copestake et al (2019). |
| Start Year | 2016 |
| Description | Oxfam GB |
| Organisation | Oxfam GB |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | Project evaluation in the Jimma area of Ethiopia, using the QuIP methodology developed under the ART project |
| Collaborator Contribution | Funding the study. |
| Impact | A final report was presented to Oxfam GB in 2016 and also formed the basis of a workshop with Oxfam staff in Ethiopia in November. Oxfam then produced a briefing paper based on this report - https://policy-practice.oxfam.org.uk/publications/cash-cropping-and-care-how-cash-crop-development-is-changing-gender-relations-a-620392 |
| Start Year | 2016 |
| Description | Save the Children UK |
| Organisation | Save the Children |
| Department | Humanitarian Affairs Team |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | Copestake has participated in various evaluation training activities organised by SC. BSDR have conducted two QuIP studies of Save the Children projects in Tanzania and in Ethiopia. |
| Collaborator Contribution | They have funded two QuIP studies. |
| Impact | The QuIP study in Tanzania is documented as a case study in Chapter 7 of Copestake et al. (2019). |
| Start Year | 2016 |
| Description | SetSquared Partnership |
| Organisation | SETsquared Partnership |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | Selected for an ICURe (Innovation to Commercialisation of University Research) grant |
| Collaborator Contribution | Financial, plus participation in training workshop |
| Impact | Post I-Cure we prepared a business plan for launch of a spin-off company to promote the QuIP and other applied social and development consultancy and applied research activities generated by the Centre for Development Studies. This was pitched unsuccessfully to Innovate UK early in 2016 for a start-up grant of £300k. However, feedback was useful and informed the subsequent launch of Bath Social and Development Research Ltd. |
| Start Year | 2015 |
| Description | Tearfund |
| Organisation | Tearfund |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | BSDR undertook to conduct a QuIP study of its "community and church mobilisation programme" in Uganda in 2016. Further studies were undertaken in Sierra Leone in 2017 and Bolivia in 2019. A fourth study is planned. The long-term goal is to support Tearfund's capacity to undergo QuIP studies with its own staff. |
| Collaborator Contribution | Tearfund has now funded three QuIP studies with a fourth planned in 2016. The Uganda study also features as a case study chapter in Copestake et al. (2019). |
| Impact | The first study has been published on the web |
| Start Year | 2016 |
| Title | Impact evaluation approach (Qualitative Impact Protocol) |
| Description | The QUIP is an approach to impact evaluation development through the ART Project research, an Excel based analysis tool. The University of Bath has licenced its use to Bath Social and Development Research Ltd - a social enterprise dedicated to promoting improved social impact evaluation through its use. |
| IP Reference | |
| Protection | Protection not required |
| Year Protection Granted | 2016 |
| Licensed | Yes |
| Impact | Under the licence agreement BSDR has undertaken more than twenty QuIP studies in more than ten countries. Under the licence agreement it pays the University 1% of revenue. A trademark for the QuIP is also pending. |
| Company Name | Bath Social & Development Research Limited |
| Description | |
| Year Established | 2015 |
| Impact | Shortlisted for start-up grant of cerca £300,000 from Innovate UK in January 2016 (unsuccessfully). During 2016 commissions were received for QuIP studies from Gorta Self Help Africa (Kenya), Oxfam GB (Ethiopia), Diageo Ltd (Ethiopia), C&A Foundation (Mexico), Habitat for Humanity (India), Acumen (India), Tearfund (Uganda). In 2017 commissions were received for QuIP studies from Tree Aid (Uganda) and Save the Children (Tanzania). Training was also commissioned by the Aga Khan Development Network. |
| Description | "Attributing Development Impact: scope for improving qualitative evaluations" University of Malawi, Zomba, Malawi |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Talk at the Centre for Socio-economic Research, plus two day training of field researchers |
| Year(s) Of Engagement Activity | 2018 |
| Description | "Attributing Development Impact: the QuIP quest for better impact evaluation" Lusaka, Zambia |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Public Lecture in the School of Public Health of the University of Zambia (UNZA). |
| Year(s) Of Engagement Activity | 2018 |
| Description | "Attributing development impact: scope for improving qualitative evaluation." IDS Sussex, Brighton, UK |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | QuIP presentation |
| Year(s) Of Engagement Activity | 2018 |
| Description | (European Evaluation Society Conference): "Building livelihood resilience in rural Malawi: constraints to credible impact evaluation and synthesis." Thessaloniki, Greece |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Conference presentation on evaluation practice in Malawi in relation to climate change |
| Year(s) Of Engagement Activity | 2018 |
| Description | (European Evaluation Society). Impact evaluation using the QuIP Thessaloniki, Greece |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | to add |
| Year(s) Of Engagement Activity | 2018 |
| Description | ART Launch Event |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Around 40 people attended the launch of the Assessing Rural Transformations project to find out more about our plans. Each partner in the project presented what role they planned to play, and what they hoped to gain from the project. The event was aimed at a targeted group of influential stakeholders and gave us a diverse constituency with whom to start communicating the outcomes of our research. Since then the PI has collaborated with a number of attendees on papers and workshops. |
| Year(s) Of Engagement Activity | 2013 |
| Description | ART Project dissemination workshop (Addis Ababa) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | Formal presentation of findings from Art project QuIP studies in Ethiopia, hosted by Farm Africa. Addis, 6-7 July. |
| Year(s) Of Engagement Activity | 2015 |
| Description | ART Project dissemination workshop (Lilongwe) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | Dissemination workshop to present findings from QuIP and IHM studies commissioned under the ART project in Malawi, hosted by Self Help Africa. Lilongwe, 8-10 July |
| Year(s) Of Engagement Activity | 2015 |
| Description | Conference presentation (Fourth Annual South West Qualitative Research Symposium) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Presented a paper entitled "positionality in qualitative analysis: who are we coding for?" at the fourth annual South West Qualitative Research Sympoisum at the University of Bath. The bulk of the audience were social science PhD students from across the region. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Conference presentation. Development impact assessed inclusively? The qualitative impact protocol case book Manchester, UK |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | to add |
| Year(s) Of Engagement Activity | 2018 |
| Description | Conference: Impact, Innovation and Learning: Towards a Research and Practice Agenda for the Future |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | New contacts forged with key stakeholders. This lead to collaboration on a journal paper as well as a book chapter. |
| Year(s) Of Engagement Activity | 2013 |
| URL | http://www.ids.ac.uk/events/impact-innovation-and-learning-towards-a-research-and-practice-agenda-fo... |
| Description | DFID: seminar for evaluation specialists (London) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited talk on the QuIP methodology to DFID evaluation staff in London, live transmitted to staff in DFID offices worldwide and recorded for video dissemination. DFID Head Office in Whitehall, 9 December. |
| Year(s) Of Engagement Activity | 2015 |
| Description | DSA Annual Conference (Bath) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Hosted a panel session jointly with participating NGOs and independent peer reviews (Hayman from Intrac, Hutchings from Oxfam GB) on the ART project. Bath, 7 September. |
| Year(s) Of Engagement Activity | 2015 |
| Description | DSA Conference 2013 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presented "Credible accounts of causation in complex contexts" to the DSA audience More people asked to be added to the mailing list and engage more closely with the project. |
| Year(s) Of Engagement Activity | 2013 |
| Description | European Evaluation Society Sep 2016 - Maastricht |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Part of a panel presentation on innovation in impact assessment. |
| Year(s) Of Engagement Activity | 2016 |
| Description | ISTR Conference presentation (Stockholm) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Third sector organisations |
| Results and Impact | I gave a presentation on the QuIP as part of a panel on innovative methods of qualitative impact assessment organised by INTRAC. Approximately 40 people were in the audience - from all over the world - and I received several e-mail enquiries requesting copies of my paper afterwards. The paper, entitled "Managing relationships in qualitative impact evaluation to improve development outcomes: QuIP choreography as a case study." is currently being revised for submission to a journal. |
| Year(s) Of Engagement Activity | 2016 |
| Description | Impact Evaluation Workshop (Manchester) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | I was invited to present at a small workshop at the Institute of Development Policy and Management, University of Manchester (a demonstration of the interest in ART in the field of impact assessment). Potential for future collaboration. |
| Year(s) Of Engagement Activity | 2014 |
| Description | Impact investment workshop (Bath) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This was a meeting over three days of international experts, mostly from the private sector, concerned with how impact investors assess the social impact of their programmes. It aimed to generalise from the experience of social performance assessment of microfinance for the impact investment field more widely. It also sought to build on the work developing QuIP to reflect more widely on methodological issues and options associated with impact assessment using qualitative methods and how to promote better practice in this field. In addition to the UK participants came from Cameroon, India, Kenya, Luxemburg, Poland, Switzerland and the USA. See page 3 of the workshop briefing at the URL below. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://qualitysocialimpact.org/wp-content/uploads/2016/05/QUIP-SII-Briefing-Paper-May-2016.pdf |
| Description | Innovations in Impact Evaluation: the case of the QuIP - DfID talk |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Talk at the Department of International Development (Residential for Evaluation and Statistics Teams) |
| Year(s) Of Engagement Activity | 2018 |
| Description | Internal webinar for staff working for Save the Children |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | We were invited to give a seminar to Save the Children staff about the QuIP. This comprised a presentation using Skype for Business, followed by Q&A. It was recorded (see below). |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://www.youtube.com/watch?v=uxoZZgGY_V4 |
| Description | International webinar to Evaluation specialists in the Aga Khan Development Network (9 November 2016) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This Webinar brought together evaluation specialists acros the global Aga Khan Development Network, for discussion of the QuIP following my Ottawa lecture a few weeks before |
| Year(s) Of Engagement Activity | 2016 |
| Description | Invited lecture organised by the Aga Khan Development Foundation Canada (20 October 2016) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | I was invited to give the fifth of a programme of invited lectures on the topic of "measuring development impact" organised by the Aga Khan Development Fundation in Ottawa, Canada. The title was "Cautionary tales of complex causation: qualitative and mixed method impact assessment of climate change and livelihood transformations in Africa" |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.akfc.ca/en/events/item/278-complex-causation |
| Description | Invited presentation (UK Department for International Development, Annual Professional Development Conference for Statistics and Evaluation specialists) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited to make a plenary presention on the Qualitative Impact Protocol (QuIP) and then engage in more detailed discussion through break-out groups. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Invited presentation on the QuIP hosted by London School of Hygiene and Tropical Medicine (23 November 2017) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation at a conference on "timely evaluation" hosted by Centre for Evaluation of LSHTM at the Wellcome Trust in London, and sponsored by the DFID Centre of Excellence in Development Impact and Learning. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://evaluation.lshtm.ac.uk/ |
| Description | Lunchtime seminar - Qualitative approaches to impact evaluation |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | Yes |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | Employees of Triple Line Consulting attended the lunchtime seminar and took part in a discussion. A KTP employee wrote an extended paper on a study related to this work, which was then presented at a 3ie conference in Manilla in 2014. |
| Year(s) Of Engagement Activity | 2014 |
| Description | Mixed Methods Research in Poverty and Vulnerability: sharing ideas and learning lessons |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Participants engaged with my presentation 'Credible accounts of causation in complex rural contexts' More people requested to be added to the mailing list to keep in touch with the ART project's progress, and these people continue to engage with the project to this day. |
| Year(s) Of Engagement Activity | 2013 |
| URL | https://www.uea.ac.uk/international-development/research/poverty-and-vulnerability |
| Description | Mixed methods in Poverty Research Book Launch (Brighton) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Launch of book edited by Roelen and Camfield at Institute for Development Studies. As co-author of the chapter on QuIP Remnant was invited to join the panel. Brighton, 14 October. |
| Year(s) Of Engagement Activity | 2015 |
| Description | Mokoro Seminar on "The story behind the numbers: making, analysing and using qualitative evidence" in Oxford |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Mokoro Ltd is an Oxford based international development consultancy. It runs an annual public seminar, to which I was invited as a speaker to talk about experience in mainstreaming use of the QuIP |
| Year(s) Of Engagement Activity | 2019 |
| Description | Participation in SDG2 International Technical Seminar (Rome) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This event was organised jointly by the UN institutions in Rome (IFAD, FAO, WFP and CGIAR) to reivew how to enhance the evaluability of the second Sustainable Development Goal (to end hunger...). I was an invited participant and discussant - one of very few academics present. It was a useful opportunity to share reflections on qualitative impact assessment and to promote the QuIP. This led to its inclusion in a working paper from the UN Evaluation Group in May 2016 on "Evaluation in the SDG era". See http://www.uneval.org/document/detail/1912 paragraph 125 on page 26. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://www.ifad.org/documents/10180/c56d0c85-1821-4bb7-9e2c-77c3afdfe745 |
| Description | Partner NGO presentation to BOND (London) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Third sector organisations |
| Results and Impact | Two NGO collaborators in the ART project (van Bekkum and Allen) were invited by the M&E working group of BOND (the British Overseas NGO Development network) to present on their experience under the project and development of the QuIP. London, 10 December. |
| Year(s) Of Engagement Activity | 2015 |
| Description | Presentation on the QUIP (Bordeaux) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited plenary presentation at a pan-European conference organised by the University of Bordeaux and Trinity College Dublin on methodologies for assessing the impact of microfinance initiatives. Bordeaux, 28 October. |
| Year(s) Of Engagement Activity | 2015 |
| Description | Professional Training (BSDR LtD) QuIP Lead Evaluator Course Bath,UK |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | to add |
| Year(s) Of Engagement Activity | 2019 |
| Description | Professional Training (BSDR Ltd) QuIP Lead Evaluator Course Bath, UK |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Two day formal training in QuIP for lead evaluators, organised jointly with BSDR Ltd |
| Year(s) Of Engagement Activity | 2018 |
| Description | Professional Training (BSDR Ltd) QuIP Lead Evaluator Course Bath, UK |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | to add |
| Year(s) Of Engagement Activity | 2018 |
| Description | Professional workshop (BBC Media Action): Impact Evaluation using the QuIP |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Introductory talk about the QuIP following collaboration between BBC Media Action and BSDR in evaluating a radio programme in Myanmar |
| Year(s) Of Engagement Activity | 2018 |
| Description | Professional workshop (WYG Engineering, Research Dept in London): Impact evaluation using the QuIP |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Introductory training in impact evaluation, drawing upon QuIP experience. |
| Year(s) Of Engagement Activity | 2018 |
| Description | QuIP Training in Ghana (January 2017) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | QuIP training for six members of staff of "Participatory Development Associates" - a development consultancy based in Accra, Ghana. This was part of their partnership with BSDR to conduct a QuIP evaluation of Tree Aid's work in Northern Ghana |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.bathsdr.org |
| Description | QuIP briefing to Irish Department of Foreign Affairs (Dublin) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited talk on QuIP and the ART project by Remnant to evaluation and policy staff in the Irish Ministry of Foreign Affairs. Dublin, 2 November. |
| Year(s) Of Engagement Activity | 2015 |
| Description | QuIP presentation at ESRC-DFID sponsored conference (Pretoria) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | I gave a presentation at a session on "Innovation in Research Methods" at the ESRC-DFID Joint Fund for Poverty Alleviation Research sponsored conference in Pretoria from 16-18 March entitled "Lessons from a decade of research innovation, engagement and impact." |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.theimpactinitiative.net/files/lessons-decades-research-poverty-innovations-engagement-and... |
| Description | QuIP talk to Aga Khan Development Network Technical Working Group (Geneva) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Formal presentation at the annual workshop of the AKDN Quality of Life Technical Working Group, attended by M&E and related specialists from Geneva, US, Kenya, France, Afghanistan, Tajikistan, Pakistan, Canada and India. |
| Year(s) Of Engagement Activity | 2016 |
| Description | QuIP talk to the National Audit Office (London) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Formal presentation to the qualitative research technical group of the National Audit Office in London, also relayed by weblink to regional offices elsewhere in the UK. |
| Year(s) Of Engagement Activity | 2016 |
| Description | QuIP training for Field Research Team, Kampala, Uganda (3 October 2016) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Training in QuIP field research ahead of Tearfund sponsored QuIP study. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://bathsdr.org |
| Description | QuIP training for Field Researchers in Amsterdam (20 June 2017) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Training in data collection for QuIP impact evaluations to PhD students in international development at the Free University of Amsterdam |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.bathsdr.org |
| Description | QuIP training for evaluation consultants (7 January 2017) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Two day training for development consultants in how to lead QuIP based impact evaluations |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.bathsdr.org |
| Description | QuIP training for the Aga Khan Foundation (7 April 2017) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | QuIP training for staff of the Aga Khan Development Network from UK, Switzerland, Canada and Kenya |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.bathsdr.org |
| Description | QuIP: lead evaluator training course |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | to add |
| Year(s) Of Engagement Activity | 2018 |
| Description | Qualitative Research Symposium, Bath. Presentation on positionality in qualitative analysis: Who are we coding for? |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | Talk on positionality in doing QuIP data anaysis. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Seminar on impact evaluation: Oxfam Ethiopia |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Briefing to Oxfam staff based on a QuIP evaluation of a fairtrade coffee programme in SW Ethiopia |
| Year(s) Of Engagement Activity | 2016 |
| Description | Seminar on the QuIP at Trinity College Dublin (15 November 2017) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Third sector organisations |
| Results and Impact | Invited seminar on the QuIP hosted by the Department of Economics at Trinity College Dublin. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Seminar on the QuIP at the Oxford Department for International Development (6 November 2017) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Invited seminar on the QuIP, convened by the Oxford Poverty and Human Development Initiative (OPHI) - part of the Oxford Department of International Development of the University of Oxford |
| Year(s) Of Engagement Activity | 2017 |
| Description | Seminar on the QuIP for Bath professional doctorate students (12 September 2017) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Overview of QuIP for first year participants on the Professional Doctorate in Policy Research and Planning at the University of Bath |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.bath.ac.uk/courses/postgraduate-2018/policy-research-and-practice/dprp-professional-docto... |
| Description | Seminar on the QuIP to Bath PhD students (2 November 2017) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Seminar on QuIP to PhD students in DSPS, University of Bath |
| Year(s) Of Engagement Activity | 2017 |
| Description | Seminar presentation for OPM Ltd (Oxford) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited presentation on the QuIP to professional staff at OPM Ltd for their qualitative and mixed methods working group. Oxford, 30 July. |
| Year(s) Of Engagement Activity | 2015 |
| Description | South West Qualitative Research Symposium: Third Annual Event, Speaker Feb 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Two semianr presentations about the QuIP, mostly to doctoral students. Publication in process. |
| Year(s) Of Engagement Activity | 2019 |
| Description | UK Evaluation Society Conference (London) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presentation on QuIP and the ART project at the annual conference of the UK Evaluation Society. London, 13 May. |
| Year(s) Of Engagement Activity | 2015 |