The Role of Technology in Evaluating Cultural Value

Methods of evaluating cultural impact still need substantial development to support robust, long-term empirical research on cultural value. Meanwhile, technological innovations have raised new possibilities for evaluating cultural value that promise to increase efficiency, reach and validity beyond what is possible with conventional methods. This proposed project considers several technology-enhanced methods for evaluating cultural value, providing descriptions and critical assessments to elucidate their strengths and weakness. This project focuses on evaluation methodology per se, with the aim of supporting future empirical evaluation and research on cultural value. The project begins with existing research and methodological literature on this topic. This project will review current literature on the use of technologies to evaluate the development of cultural value through events, institutions and digital platforms such as Twitter, Facebook, individual webpages, etc.

The project team will identify the strengths and weakness of different technologies and technology-enhanced evaluation approaches to suggest an appropriate framework for making best use of digital technologies in cultural value evaluation. The results of this critical literature review will be able to broadly be applied to a framework that can advance the way in which the value of cultural engagement is evaluated and employed.

Methods
The methodology employed for this project will integrate a range of research approaches, embracing the partners' knowledge of arts/culture engagement impact evaluation, software design, production and implementation, mixed methods data analysis, visualisation of digital programmes and the requirements for integrating these new modes with more traditional cultural impact analysis. At its core, the project is focused on how we can capitalise on digital technologies to build a robust, evidence-based evaluation framework, while at the same time maintaining a critical perspective with respect to the limits of the technologies' contribution to evaluating cultural value.

Critical Review
This phase of the research will involve a review of the evidence identified in the scoping review, using transparent, clear protocols for critical appraisal of the studies, predefined relevance criteria, and any extraction and analysis of data, and thus fulfilling criteria of Campbell Collaboration guidelines. Summaries of evidence under appropriate headings will note where evidence gaps or weaknesses occur.

Critical Review Process
The critical review process will involve the following key stages:
1. Literature identified
2. Decision on relevance
3. Extract/obtain full papers
4. Summarise included papers' methods and applications of methods
5. Critically assess methods in existing literature
6. Identify additional technology-linked methodological options from beyond the existing literature on evaluating culture value
7. Critically assess additional methodological options
8. Prepare table(s) summarising methodological options, strengths and weaknesses and write up report

To enhance the project's impacts and to create a stronger research network amongst researchers, arts and culture practitioners and arts and culture funders, it is imperative to the project team that our learning is shared beyond the primary participating research institutions. The team is committed to both communicate our findings and seek further input within our individual fields and institutions, but also to engage researchers and practitioners across disciplines and institutions. All team members will participate in a number of events, aimed at reaching a wide and diverse audience of practitioners, funders, evaluators and academic researchers. These events include:

Two dissemination workshops, one at the University of Warwick and the other at Plymouth University, conducted with the i-Dat. These workshops will be targeted at arts and culture practitioners, evaluators, researchers, and arts and culture funders across the UK. This workshop will offer an introduction to the project and its outcomes, as well as an opportunity for feedback and further discussion about the possible future utility for the tools and frameworks we have summarised and critically assessed.

Informal meetings with researchers and practitioners at universities and institutions in the UK, Europe and the USA, aimed at increasing the global knowledge exchange relating to the theory and practice of evaluating arts and culture impacts. These informal meetings will be used both for dissemination purposes and for perception-checking to ensure that the project has not missed key technology-linked methodological options, as well as identifying future trends in this domain.

The team will disseminate our work extensively online through a project website and existing university, research and practitioner networks.

We will also disseminate the work completed during this project through academic and practitioner-oriented publications.
The aim of these impact generation activities is to ensure that other researchers in this field, as well as evaluators and arts and culture engagement practitioners, benefit from the work we do in the project.

This unique trans-disciplinary collaboration will create meaningful dialogues, through agile practice based research methodologies to harness the power of (among other technologies) data visualisation, sentiment analysis and social media platforms. Arts and culture events are unique in that they provide access to different constituencies and multiple layers of collaborators in a forum where the latest ideas and culture from around the world may be shared. This enables us to explore the similarities and differences of these constituencies and their interaction with the arts. This project explores the ways in which cultural value may be evaluated through innovative platforms and methods of digital audience evaluation. By bringing together research leaders working at this intersection between arts/culture, technology, research and evaluation (Dr Eric Jensen, University of Warwick, and Professor Mike Phillips, i-DAT, University of Plymouth), we will help to develop current methodological thinking about evaluating cultural value in arts and culture experiences. This research will benefit from i-DAT's prototyping methods in producing models of digital engagement that can be applied across the sector of arts and culture engagement, as well as previous and on-going externally funded (e.g. NESTA/AHRC/Arts Council; JISC) projects led by Phillips and Jensen on impact evaluation technology and methodology.