Developing Innovative Methods for Configurative Capture of the Cultural Value of Arts and Humanities Research

Lead Research Organisation: University of Oxford
Department Name: Education

Abstract

The project aims to contribute a refined conceptual model of what constitutes cultural value of arts and humanities research (and what impact evaluation methods are attempting to capture), empirical evidence of how interpretations of cultural value play out in practice (via 35 case studies, 70-100 interviews, and over 20 maps of value networks), and an innovative, in-depth methodological approach to capturing cultural value. The project will engage with scholars from 10 groups of arts and humanities disciplines and from over 20 settings, to ensure that the methods for capturing and articulating value are conceptualized in their own terms, rather than imposing simplifying dichotomies. In addition, the study will explore the experiences, understandings and practices of a further 15 value actors, including public cultural institutions, third sector organizations, cultural engagement initiatives, and other partners, users and beneficiaries of arts and humanities research.

The study will draw on social scientific approaches to make a distinctive contribution to enhancing methodological capability to capture cultural value in its richness and diversity. The development and testing of a robust configurative approach (i.e., an approach that explores and maps a field in its depth and richness) to recognising and capturing research value in the arts and humanities will be a core contribution of the study. The in-depth case studies to be carried out will combine attention to institutional and individual experiences, with careful mapping and weighting of the networks within which value is enacted and recognized.

The conceptual model and innovative methods for capturing the value of research that will be refined and tested in the project were originally developed in Oancea (2011), drawing on Oancea and Furlong (2005, 2007a, 2008) and Oancea (2007b). This body of work has critiqued excessively instrumental interpretations of the value of research, but sought to move beyond dualist arguments about, for example, intrinsic vs. instrumental value, value vs. impact, articulating vs. measuring value, or social accountability vs. economic accounting. On this basis, the project proposes a textured notion of cultural value. The conceptual model identifies aspects of cultural value and suggests that the availability of evidence and the straightforwardness of the measures we use are in inverse proportion to the breadth and complexity of value realisation.

The methodology of the study is driven by concerns for texture, diversity, nuance and ecology. The configurative approach to capturing cultural value will complement the focus on breadth and precision (i.e. aggregative approaches) that has been common in recent efforts to measure cultural value from socio-economic perspectives. A multiple case-study design will ensure both breadth (through 35 case studies of research units, research initiatives, value-oriented programmes, partner and user or beneficiary organisations) and depth (through 70-100 interviews and network mapping methods "drilling down" into each case).

The project will pursue primary impacts on individual and organizational understanding and practices, and secondary impacts on public discourses about the cultural value of arts and humanities research.

Planned Impact

The project is expected to have primary impacts on individual and organizational understanding and practices, and secondary impacts on public discourses about the cultural value of arts and humanities research.

First, an important area of primary impacts will be on the numerous higher-education (HE) and non-HE settings included in research. Prior work on research impact by the applicant, using similar methodologies, has shown the potential of these approaches to catalyze interest in the value of research and influence individual and organizational practices within the settings researched, with ripple effects in their immediate professional, disciplinary and institutional communities.

Second, the project will engage with arts and humanities scholars from a wide range of disciplines and settings, to ensure that methods for capturing and articulating value are conceptualized in their own terms, rather than being based on simplifying dichotomies. Participants in the project will be invited, for example, to engage in face-to-face (via 4-5 meetings) and written dialogue with the research team, leading to a publishable, co-owned output. By these means, the project will contribute (as a secondary impact) to the carving out of a discursive space, within which wider and more nuanced articulations of cultural value are made possible.

And third, the project will aim to inform the refinement of impact assessment practices by funding bodies, as well as the development of institutional impact strategies (primary impacts), in higher education and beyond, to ensure that they recognize the distinctive value of arts and humanities research and the diversity of ways in which it can be articulated (secondary impacts). Project briefings will be targeted at relevant constituencies and communicated via impact blogs, Twitter and other social media.
 
Description This study explored conceptually and empirically (drawing on over 500k words worth of interview data) the ways in which those engaged with university-based arts and humanities research (researchers, managers, partners, beneficiaries) construct and respond to the challenges of generating, interpreting, and demonstrating the cultural value of research.

Cultural value is a contested concept, beset by philosophical, practical and political tensions. the study argues that interpretations of value - cultural or otherwise - are part of complex ecologies of cultural life, creation and understanding, while at the same time underpinning economies of description, prescription, inscription and ascription. Meaning, expression, narrative and practice, combined and recombined in experience, are core themes in our participants' description of the arts and the humanities. However, more needs to be done across all levels of the research governance system so that meaningful engagement is sustained and narratives in cultural terms are not perceived as a risk in accountability contexts.

In addition, the study developed an innovative methodological approach for articulating and communicating the impact and cultural value of research: qualitative network analysis using collaborative configuration tracing and visualisation. The approach uses co-constructed, case-centred network diagrams to enable the structured elicitation and visualization of information from participants (researchers, administrators, facilitators, partners, users and beneficiaries) and the assessment of the relative strength of different flows and relationships relevant to creating, articulating and demonstrating value. Unlike quantitative network studies, the emphasis here is on the process of construction and interpretation of qualitative network maps by the participants. Subject to further testing and refinement, the approach and tools that we have developed are amenable to use in research, evaluation, communication and developmental work in higher education institution and funding organisations.
Exploitation Route Following publication of one peer-reviewed article, two blogs and magazine items, and several conference inputs in 2015, and forthcoming publication of another peer reviewed article in 2016, the substantive and conceptual findings will benefit scholars and research managers in the arts and humanities, particularly in their work on refined understandings of research value and on reporting the impact of research for research assessment purposes. Particularly important is the project's methodological contribution to the creation of robust configurative approaches to impact case study development, which can contribute to advances in the fields of research impact and evaluation, but also to practical developments in the creative sector, heritage, museums, and cultural policy. The findings have been used in research training and there is potential for further use in professional development. The methodological tools generated can lead to digital developments and we have started exploring this possibility.

Other communication about the project to date includes an extended report to the Cultural Value project, a blog entry, posters, talks and presentations, invited talks, publicly-available Prezi, and Twitter.

The project is expected to have primary impacts on individual and organizational understanding and practices, and secondary impacts on public discourses about the cultural value of arts and humanities research. Feedback from some of the 25 participant settings has pointed to the potential of this work to catalyze interest in the value of research and influence individual and organizational practices. Further application came from collaboration with two of the initiatives studied (a KE partnership in the heritage sector and a research enterprise unit in the arts) after completing the fieldwork.

The project also aimed to inform the refinement of impact assessment practices by funding bodies. Examples of engagement include participation in an invited HEFCE roundtable on novel valuation methods (spring 2015), a keynote to the European Foundation for Management Development (June 2015); contribution to the Cultural Value's project workshop on novel methodologies for capturing value configurations (July 2014); and an invited contribution to the Independent Metrics Review (2015).

An online resource developed with subsequent funding (HEIF) is available at http://www.education.ox.ac.uk/impact-and-knowledge-exchange/. A short paper with recommendations for the future of the research assessment system is in preparation.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections,Other

URL http://www.tandfonline.com/doi/pdf/10.1080/10286632.2015.1128418
 
Description An online resource developed with subsequent funding (HEIF) is available at http://www.education.ox.ac.uk/impact-and-knowledge-exchange/. Further policy impacts are likely to mature over the next two years through engagement with research policy bodies at European and national level. For example, Prof Oancea is currently a member of a EC Expert Group on indicators for researchers' engagement with open science and scholarship (2017-19) and had also been appointed to the Advisory Group on the Future of Research Assessment convened by Research England (2018-19). Feedback from some of the 25 participant settings has pointed to the potential of this work to catalyze interest in the value of research and influence individual and organizational practices. More direct application came from collaboration with two of the initiatives studied (a KE partnership in the heritage sector and a research enterprise unit in the arts) after completing the fieldwork. In addition, the findings have been used in researcher training. The project also aimed to inform the refinement of impact assessment practices by funding bodies. Examples of engagement include participation in an invited HEFCE roundtable on novel valuation methods (spring 2015), a keynote to the European Foundation for Management Development (June 2015); contribution to the Cultural Value's project workshop on novel methodologies for capturing value configurations (July 2014); and an invited contribution to the Independent Metrics Review (2015). Further potential collaborations are being explored. In February 2018, Prof Oancea gave an invited talk on cultural value from research to kick-start a series of KE meetings with senior professionals in the museum and heritage sectors (convened by Dr K Eccles) focused on understanding and measuring impact and outcomes in the cultural and heritage sector (https://www.oii.ox.ac.uk/research/projects/measuring-what-matters/).
First Year Of Impact 2014
Sector Creative Economy,Education,Culture, Heritage, Museums and Collections,Other
Impact Types Cultural,Policy & public services

 
Description "What is Cultural Value?" Contribution to 'Measuring what Matters' knowledge exchange workshop with museum and heritage sectors
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
URL http://www.oii.ox.ac.uk/research/projects/measuring-what-matters/
 
Description HEFCE Metrics Review 2015
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_m...
 
Description HEFCE Metrics Review 2015
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_m...
 
Description HEFCE Metrics Review 2015
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://www.hefce.ac.uk/news/Events/2015/Name,101073,en.html
 
Description Invitation to HEFCE expert round table on novel valuation methods
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Title Impact and Knowledge Exchange Resource 
Description This is a web-based resource intended for doctoral students, early career researchers, and other researchers who want to develop their understanding of knowledge exchange and research impact, as support in learning about KE paths and impact outcomes at different stages of the research process. It also aims to support research trainers and facilitators, as a source of teaching materials and ideas to be used flexibly in any courses they provide on KE and research impact in the social sciences. The main aim of this site is to support career development in ways that are both respectful of the values of those involved in research and knowledge exchange, and open to innovation and diversity. 
Type Of Material Improvements to research infrastructure 
Year Produced 2015 
Provided To Others? Yes  
Impact The resource is being used in researcher training at Oxford. There is interest in developing it further from both Social Sciences and Digital Humanities. 
URL http://www.education.ox.ac.uk/impact-and-knowledge-exchange/
 
Title Methodology for qualitative network analysis of impact and cultural value published 
Description The qualitative network analysis using collaborative configuration tracing and visualization approach was piloted in Oancea (Interpretations and Practices of Research Impact across the Range of Disciplines Report, Oxford, Oxford University, 2011) and uses co-constructed qualitative network diagrams to enable the systematic elicitation and visualization of information from participants (such as researchers, administrators, facilitators, partners, users, and beneficiaries of research) about the different flows and relationships that they see as relevant to creating, articulating, and demonstrating impact and value from research. Unlike quantitative network studies, the emphasis is on the process of construction and interpretation of qualitative network maps by the participants. Subject to further testing and refinement and to critical understanding of the conceptual, technical, practical, and political limitations of measurement in this area, the approach can be adapted for use in research, evaluation, communication, engagement, knowledge exchange, and developmental work. 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact Follow-up projects 
 
Title Qualitative Network Analysis methodology 
Description A novel approach to collecting and analysing qualitative network data, for use in particular to explore and articulate impact, KE and cultural value. 
Type Of Material Data analysis technique 
Year Produced 2014 
Provided To Others? Yes  
Impact The methodology was presented to an AHRC methodological workshop (2014) and informed participation in a HEFCE roundtable on novel valuation methods (2015). Another researcher is already considering it for use in their projects. The PI applied versions of the method institutionally and in researcher training. A peer reviewed article detailing the methodology is has been published by Research Evaluation - https://doi.org/10.1093/reseval/rvx014. 
URL https://doi.org/10.1093/reseval/rvx014
 
Description Advisory Board, 'Humanities in practice' 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Study participants or study members
Results and Impact Invited to join the Advisory Board for the 'Humanities in practice: Expanding the interdisciplinary understanding of the practice and evaluation of quality in the humanities' application (University of Bergen, School of Social Anthropology) to the Norwegian Research Council
Year(s) Of Engagement Activity 2018
 
Description An alternative education for the heritage decision makers of the future 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Study participants or study members
Results and Impact 'An alternative education for the heritage decision makers of the future' conference paper given by Jeanette Atkinson (With S Atkinson) at The Ends of the Humanities, University of Luxembourg, 10-13 September 2017
Year(s) Of Engagement Activity 2017
URL http://endsofthehumanities.com/wp/
 
Description Member of European Commission expert group on New Indicators for Researchers' Engagement with Open Science and its Impacts 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact The group, comprising of six international experts, is tasked with making recommendations to the European Commission for future policies concerning responsible indicators for researchers' engagement with open science. The group met for the first time in November 2017 and will produce its final report and recommendations for European policy by Spring 2019. ((https://ec.europa.eu/research/openscience/index.cfm?pg=altmetrics_eg)
Year(s) Of Engagement Activity 2017,2018,2019
URL https://ec.europa.eu/research/openscience/index.cfm?pg=altmetrics_eg
 
Description Scientific Advisory Board, Diversity Approach to Research Evaluation 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Invited to join the Scientific Advisory Board for Diversity Approach to Research Evaluation (DARE) (M. Hopkins and I Rafols, Sussex), on the basis of the methodological work on cultural value networks.
Year(s) Of Engagement Activity 2019