The impact of impact: How research value is created and assessed in the UK, US and Australia

Lead Research Organisation: King's College London
Department Name: The Policy Institute

Abstract

Context and aims

As universities become bound up in a public role as drivers of economic prosperity, there has been a global policy shift towards measuring the value and benefit of publicly funded research. Governments have invested substantial resources in assessing research impact despite contestation around definitions, methodologies and processes. The rapid channelling of resources into impact has resulted in an under-theorised field that is largely dominated by pragmatic research policy concerns (i.e. focused on procedures and mechanisms). What is missing is a more nuanced conceptualisation of the societal value of research impact, which is essential in ensuring that research investment leads to meaningful societal outcomes. To capture this value, theoretically informed empirical insights into impact and the effects of its assessment on research cultures are needed. This, in turn, can inform the responsible evaluation of publicly funded research.

This will be the first large-scale comparative study examining research impact across countries. The project will compare academic cultures in three countries involved in the international push for the assessment of research impact; the UK, US and Australia, which utilise substantially different assessment models. The project will develop a theoretical understanding of research impact and its measurement. It will identify the effects of national research impact assessment policies and practices, and ascertain how they shape the ways research is done and their consequences for knowledge.

Potential applications and benefits

There is a pressing need for in-depth studies of the institutional and individual effects of research evaluation to interrogate frequently uncritical acceptance of dominant forms of valuation. This study will be novel in its examination of the powerful rhetoric of value that dominates discussions of research impact, and in its articulation of the challenges involved in the audit and commodification of academic research. This research will advance sociological and political understandings of research value, and will contribute theoretically informed knowledge to the applied fields of research evaluation and public policy. It will examine the emerging cultural patterns that surround research impact, and shed light on the political processes of decision-making and authority around impact policies. The study will shape the academic debate on the impact cultures of the UK, US and Australia, and contribute to a larger debate on the value of publicly funded research.

The study has strong potential to influence the policies that determine how impact is measured and how publicly funded research is governed in the UK, US, Australia and globally. By working directly with stakeholders including science policymakers, research funders and researchers, it will inform debate on the role of universities and research in society and facilitate more robust, reflexive and ethical research systems. It will provide recommendations on the structures and resources that policymakers, funders and higher education leaders could make available to researchers to support them in achieving high-quality research that makes a meaningful contribution to society. Understanding research impact and its assessment is of critical importance, not only strategically for UK, US and Australian governments, but also for countries with similar or emerging evaluation and funding models. It will allow countries to be intentional about creating responsive, equitable research systems that promote core values, such as a dual focus on research excellence and societal benefit.

Outputs from the study will include dissemination to policymakers, practitioners and the wider public through events for stakeholders, a working paper, a policy brief, blogs and media articles and a project website. Academic dissemination will occur through conference participation and open access articles in top journals.

Publications

10 25 50
 
Description Cited in University of Melbourne Central Research Strategy
Geographic Reach Australia 
Policy Influence Type Citation in other policy documents
Impact Academic work was the basis for the design of University of Melbourne's Chancellery strategy, which informs how the University invests in and manages research impact.
 
Description Participation in evaluation of the University Grants Committee (Hong Kong) Research Assessment Exercise
Geographic Reach Asia 
Policy Influence Type Membership of a guideline committee
 
Description Participation in policy research project for the Australia Academy of Social Sciences, 'Modernising research assessment in Australia'
Geographic Reach Australia 
Policy Influence Type Contribution to a national consultation/review
 
Description Participation in policy research project for the Royal Society, 'Taxonomy of Artificial Intelligence related technologies for scientific research'
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description National research impact policies: Uncovering the 'value' in evaluation'
Amount $164,764 (AUD)
Organisation Australian Research Council 
Sector Public
Country Australia
Start 01/2024 
End 12/2027
 
Description Wider societal value of research and consequences of its assessment: A multi-country and multi-method study (MultiSocVal)
Amount € 1,597,400 (EUR)
Organisation Volkswagen Foundation 
Sector Charity/Non Profit
Country Germany
Start 08/2024 
End 08/2028
 
Description Expert workshop on the theory and practice of research impact in international perspective 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact An invited group of 12 guests, involving both other scholars of research impact as well as impact/evaluation professional practitioners, attended a two-day workshop to discuss the project's interim findings and share related ongoing work/research. This resulted in the attendees successfully submitting a special issue proposal at the journal Research Evaluation (expected publication 2025), which has a reputation of reaching both research and practitioner audiences.
Year(s) Of Engagement Activity 2023
 
Description Invited to UKRI-DSIT roundtable to discuss effective ways of supporting and funding R&D. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Supporters
Results and Impact Invited to roundtable to contribute to discussion on how to best support and fund R&D, partly based on knowledge/expertise gained during this project.
Year(s) Of Engagement Activity 2024
 
Description Presentation at Impactful Career Pathways Short Course, University of Melbourne 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Delivered talk to Impactful Career Pathways, drawing on Bourdieu's social theory as a framework to explore different forms of capital underpinning the five different facets of impact from scholarly and creative, to policy/government/law, to public debates, to practices and processes, and enterprise and commercialisation.
Year(s) Of Engagement Activity 2023
 
Description Revisiting the role of research in society' public talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Convened a public panel discussion on research impact called 'Revisiting the role of research in society' at the University of Melbourne, on 9 September 2021. The live virtual event drew over 80 attendees, with many more watching the recording.
Year(s) Of Engagement Activity 2022
 
Description Talk at Westminster Higher Education Forum 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Talk at Westminster Higher Education Forum on research impact.
Year(s) Of Engagement Activity 2022
 
Description Talk at the Innovation to Research Impact Assessment, Virtual International Convention for Research Administrators 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Talk given at the Innovation to Research Impact Assessment, Virtual International Convention for Research Administrators, 1-3 March 2022, a virtual international event run across four continents.
Year(s) Of Engagement Activity 2022