Qualitative impact assessment protocol for social impact investment (QUIPSII)

Under the ESRC-DFID funded 'ART' project (Assessing Rural Transformations) a robust methodology and protocol for qualitative impact evaluation (known as the 'QUIP') was designed and piloted for use primarily by NGOs promoting household level food security and poverty reduction through rural livelihood interventiions. This should help to strengthen learning and accountability feedback loops for their development interventions.

The QUIP incorporates many best practice characteristics of other qualitative impact evaluation methods, including contribution analysis and process tracing. It also emphasises complementarity with quantitative performance monitoring. In addition, it broke new ground in addressing problems of pro-project or confirmation bias, and in promoting transparency (hence auditability) in the analysis of text data. This follow-on proposal is to help us broaden and deepen adoption of these innovations, following positive feedback on the QUIP from both practitioners and evaluation specialists.

While believing that the QUIP has potential to be adapted for use by a wide range of organisations, the proposal is to promote its adoption particularly among impact investors in the field of socially responsible financial inclusion and enterprise development. Impact investment is a rapidly growing field, and its performance also hinges on being able to make credible and cost-effective assessment of impact at the household level in complex contexts that are often subject to rapid change. Financial inclusion and enterprise development are leading areas of impact investment. They are subject to ongoing debate about the potential for market-led development strategies for which better evidence of household level impact is needed. This is also a field in which the Principal Investigator has substantial expertise and an established international reputation.

The following steps are required to promote the QUIP in this field:
(1) Close engagement with existing networks and leading organisations promoting financial inclusion and enterprise development (e.g. CGAP, ACTION International, DCED, GIIN, SEEP, SoCAP, UNCDF), including analysis of current impact evaluation approaches and relevant publications. (2) Adaptation of the QUIP to meet the needs of organisations working in this sector, including further development of the open access QUIP analysis software.
(3) Preparation and web dissemination of QUIP materials specifically tailored to potential users in this sector - e.g. to accommodate different impact domains.
(4) Development and testing of tools for training and supporting those interested in (a) commissioning QUIP studies, (b) being employed as researchers to carry them out.
(5) Translation and testing of key materials into Spanish, French and Hindi. Further web based dissemination activities in these languages.
(6) Identification of - and collaboration with - organisations willing to commit to QUIP studies at their own expense.

To spearhead promotion of the QUIP we are also exploring setting up a specialist consultancy organisation dedicated to this task as a research spin-off company. We believe that doing so will enable us to scale-up dissemination and use of the QUIP more rapidly, effectively and sustainably. Separate funds have already been obtained in the form of an 'ICURe' grant to support this business development aspect of promotion of the QUIP from the SETsquared Partnership. Backed by Innovate UK and HEFCE, this is the leading enterprise collaboration between the universities of Bath, Bristol, Exeter, Southampton and Surrey. The purpose of the ICURe grant is restricted to helping a named early career researchera through a narrowly specified set of market validation and business planning activities. These are complementary to the impact and engagement activities requested in this proposal.

Who will benefit and how?

The immediate beneficiaries of the project will be public and private social impact investors who face the problem of how to address the attribution challenge in assessing the social impact of their work. Improved feedback on the contribution of activities to positive and negative social impact can support (a) short feedback loops via internal learning and operational adjustments (b) long feedback through improved wider accountability and resource allocation. The QUIP will be useful to any social impact investor serious about the need for strong social impact reality checks in evaluating their efforts to reduce problems of economic insecurity, poverty, injustice and social exclusion through improved access to financial services, employment and self-employment.

Benefits will be achieved directly by seeking out organisations willing to take up and use the QUIP. We will do this by building up and supporting networks of (a) skilled commissioners willing and able to finance QUIP studies themselves, and (b) professional researchers skilled in carrying out such studies. Our longer term goal is to set up an independent non-profit company to drive this work forward on a sustainable footing. Initial interest in the QUIP has come from organisations ranging from Diageo to IIED and the Stromm Foundation in Norway.

Second, benefits will be achieved indirectly by broadening the range and accessibility of material available about the QUIP (and hence the ideas underpinning it) in order to influence social impact evaluation methods and practices. We will do this by engaging in practitioner and academic debates over how to add to the evidence base for socially responsible impact investment in ways that are methodologically robust.