Measuring Cultural Value (Phase 2)

Essential context for the project is the view that, given the stringent economic climate, the cultural sector must lobby for public funding using the concepts and tools of economics or face dire consequences, a strategy that has worked for other areas of public policy, most notably health and the environment. However, this is a contested viewpoint because a) such economic reductionism grates with the ethos of the cultural sector; b) state of the art evaluation in health and the environment includes but also extends beyond economic indicators (Donovan, 2007; HM Treasury, 2007). So, following Oscar Wilde (1917) we are faced with a caricature of two cultures of evaluation: cynics who know 'the price of everything and the value of nothing' and sentimentalists who see 'an absurd value in everything, and [don't] know the market price of a single thing' (Donovan, under review). This project proposes an exercise that will engage key stakeholders to build consensus around a 'toolkit' for measuring cultural value, based on a similar project conducted with rival economics research groups (Donovan and Butler, 2007). The goal is to facilitate stakeholders' empirical assessment of a diverse range of indicators, evaluation methods, and analytical assessment frameworks, to produce a balanced 'cynical-sentimental' evaluation toolkit based on best practice in cultural economics and other disciplines.

An expert advisory group will be recruited, representing policymakers and practitioners within government, stakeholder institutions, arts and gallery curators, cultural economists, and arts and humanities academics, initially drawing from stakeholders from Phase 1 of 'Measuring Cultural Value'. Indicators, evaluation methods and analytical assessment frameworks will be identified from Phase 1 recommendations, a review of literature and best practice in the public sector, academia and consultancy, and where possible, raw data shall be obtained. Experimental quantitative and qualitative indicators will also be proposed, e.g. the equivalent of a QALY for cultural engagement as a possible indicator to compare disparate areas of culture, and hence to calculate the value (cost benefit analysis) of a policy intervention; b) indicators of supposedly 'intangible' broader cultural value, i.e. 'stimulating creativity within the community; contributing to cultural preservation and enrichment; and bringing new ideas and new modes of experience to the nation' (Donovan, 2008: 54). In Workshop 1, stakeholders will collectively select which indicators and approaches to test.

Following Donovan and Butler (2007: 232) the Fellow will test each indicator for validity, reliability, fairness, transparency, independence, cost effectiveness, behavioural impact, Green Book or Magenta Book compliance, and cultural sector approval. In Workshop 2, stakeholders will assess the indicators and measures, and through accepting and rejecting options will construct a 'toolkit' for measuring cultural value; recommend analytical assessment framework(s) for conducting policy evaluation ex ante, ex nunc, ex post, and at various levels of aggregation (programme, capital project, regional, national); identify the most promising areas in which to commission future valuation studies. Please refer to 'Case for Support' for project timelines, and see Donovan and Butler (2007) for methodological details.

My CV shows many years' experience leading projects on novel quantitative and qualitative indicators of research impact, a skill set transferable to evaluating public policy; and as an advocate for a better evidence-base for funding for the humanities and arts through my academic and consultancy work (e.g. Council for the Humanities, Arts and Social Sciences), as Chair of an Australian government committee that advised the Chief Scientist on the optimal methodology for assessing research impact across all fields, and in opinion pieces for the Australian Higher and Times Higher.

There three main groups of potential beneficiaries of this Fellowship are government, the cultural sector, and other research funding organisations.

More impact is likely to occur if key stakeholders are engaged with the research process. Therefore, stakeholder workshops will be the arena where the key project decisions are made. Involving key stakeholders in the research from the very beginning, and empowering them to decide which indicators to test and adopt, is a fundamental part of project's consensus-building role. This consensus-building between government and cultural sector interests will be an immediate benefit from the research.
The workshops will be held with an expert advisory group comprised of policymakers and practitioners within government (e.g. DCMS, Treasury), stakeholder institutions (e.g. Arts Council, British Museum), key arts and gallery curators, cultural economists, and arts and humanities academics. Recruitment will initially draw on stakeholders from Phase 1 of 'Measuring Cultural Value', thus helping to maximise the impact of the first placement Fellowship and to provide a sense of continuity.

This stakeholder involvement will also give more credibility to the final report's recommendations (including the evaluation toolkit) among the cultural sector and within government (DCMS, HM Treasury).
The most desirable potential medium- to long-term impact would be the adoption and development of the evaluation 'toolkit' by government and the cultural sector. This toolkit would help to build a strong evidence-based case for investment in the cultural sector, which is currently lacking. This will benefit the cultural sector as it can better make the case for financial support from public funds; and this will benefit government and citizens, as government can be confident that policy decisions and financial investments are based on solid ex ante, ex nunc, and ex post evidence.

As stated above, stakeholder engagement with the research process lies at the heart of this project, and is essential in making the report's recommendations credible. Dissemination of project results is also vital, and the previous placement Fellow for Phase 1 of this research has advised that funds for dissemination (especially for hard copies of the project report) are crucial. Flyers describing the evaluation toolkit can be distributed, providing links to an online version of the final report.

Other potential ideas for dissemination after the placement Fellowship has ended include a high profile report launch, and constructing an online evaluation toolkit for the sector (and evaluators within government) to use.

Other likely beneficiaries of this research are other cultural sector funding agencies, who are also struggling with how best to make the case for more funding, and how best to evaluate the impact of their own investments. Also, the toolkit may be of use for evaluating other areas of public policy, particularly with regard to social and cultural benefits that are felt to be 'intangible'.