Accountability, Efficiency, Improvement and Change in UK Local Public Services: The Role of Benchmarking and External Performance Assessment

Lead Research Organisation: CARDIFF UNIVERSITY
Department Name: Cardiff Business School

Abstract

The knowledge exchange will facilitate the development of a network of policy makers and practitioners who will work with social science researchers to analyse and respond to the challenges of performance assessment in public services.

A series of studies of performance assessment in public services over the last decade have identified the hitherto largely untapped potential for policy learning between countries (Nutley et al. 2011). Our knowledge exchange activities will aim to facilitate policy learning between the UK, Canada and Australia. It will use theoretical frameworks developed in the course of previous research - in particular, regime theory and the concept of theories of improvement which underpin assessment frameworks (Downe et al. 2010), the alternative co-operative versus competitive approaches (Fenna 2010), and the UK experience of 'voluntary' comparative benchmarking (Grace 2010) - to apply to current challenges including reductions in public expenditure and the issue of how to assess innovation and the capacity for continuous change and improvement.

The proposed programme will apply these concepts to assist policy makers and practitioners in analysing and addressing current challenges in the UK and internationally. It will offer five inter related sets of activities:

1. International events - we will convene three international conferences, one each in the UK, Canada and Australia. The conferences will bring together policy makers and practitioners from the UK as well as OECD-type federal countries and the EU to learn from each other. The events will:
- Raise awareness of the knowledge exchange at senior levels.
- Encourage policy transfer and debate between countries.
- Generate feedback into the wider programme of international learning and comparison.
- Help design the UK seminars.
- Help to identify and motivate key stakeholders to be involved in future events.
- Inform the policy briefing on best practice.

After each event the applicants will write a summary of the main emerging issues and lessons learnt which will be fed back both to individual participants and to their organisations, to help encourage knowledge exchange.

2. Seminars - we will use our excellent networks built from previous research, to work with a small group of policy makers/practitioners in each country of the UK to co-produce the design of the seminars and the participant list - three in England and one each in Scotland, Wales and Northern Ireland. Each seminar will involve 15 to 20 participants including senior policy makers, public services practitioners and academics. The seminars will provide opportunities for facilitated analyses of existing approaches to performance assessment and the challenges currently facing policy makers and practitioners. As with the conferences, the applicants will write up a summary of the seminars for distribution to all participants.

3. Network - the conferences and seminars will create a network of policy makers, practitioners and researchers who will be invited to inform the design and co-produce subsequent research on the effectiveness of emerging performance assessment frameworks.

4. Policy Briefing - this will provide a review of international performance assessment written specifically for a policy and practitioner audience and distributed widely via the web, professional associations and other media. It will draw on existing research knowledge and will draw together lessons from the conferences and seminars to highlight best practice from across the world.

5. Dissemination exchange workshops - these will draw upon early versions of the policy briefing involving one of the research team delivering presentations at pre-existing events of professional, local government, and consumer bodies. We will deliver ten workshops across the four countries (five in England, two each in Scotland and Wales and one in Northern Ireland).

Planned Impact

This project aims for societal and economic impact by helping to develop better ways of assessing and comparing local public service performance, helping policy makers and practitioners to know about and apply academic knowledge to that task, and helping a much wider range of organisations and people understand about the best practices for carrying out such assessment.

The high cost of failure of local public services, and the benefits which accrue to service users and to wider communities if they are successful, makes it critical that they can be assessed and compared by all stakeholders with confidence and reliability. This topic has real importance to economic and social life in the UK because it bears directly on the quality of life of individuals and communities, and also has a major relevance to the public expenditure and public services issues created by the economic and financial crisis. Consequently it is of major contemporary interest and concern to practitioners and policy makers, and is in a state of flux as a result of the policies of the new Coalition Government and the new mandates granted in Wales and Scotland through the recent elections. Many of them are open to learning about and applying new and different methods and approaches - hence the strong and broad support from potential users for this proposal.

Thus we aim for impact in:
- Evidence based policy making and influencing public services
- Enhancing the effectiveness and sustainability of public services organisations, and helping to change organisational practices
- Enhancing the quality of life, health and well being, and social welfare, because local public services strive directly to support such outcomes

In terms of specifically who will benefit, and how, amongst non-academic users:

- UK policy makers (in the Department for Communities and Local Government, Treasury, Scottish and Welsh Governments and Northern Ireland Executive) will benefit in two ways. They will gain insights into: (a) how to develop effective performance assessment frameworks based on localist principles of voluntary and co-operative comparisons, and (b) which approaches work best in increasing the efficiency and effectiveness of local public services

- Policy makers in Australia and Canada will learn from the experiences of the different parts of the UK about designing effective benchmarking and external performance assessment frameworks in devolved contexts.

- Senior practitioners responsible for assessing local public services in the National Audit Office, Audit Commission, Audit Scotland, the Wales Audit Office, and the Northern Ireland Audit Office will learn about which methods of assessments work best in different settings

- Senior local government managers will learn about how to implement effective performance assessment from the 'bottom up'.

- Service users and intermediate bodies who help them assess the performance of public services will benefit from more effective approaches to performance assessment which will result in more efficient and effective public services and clearer accountability for performance.

The specific legacy arrangements we propose will extend the engagement of beneficiaries and increase the likelihood of impact having lasting value. These are:

- Creation of a network of researchers, policy makers and practitioners to form a 'community of practice' connected to the mainstream activities of the Local Government Group and its sister associations, and those of the principal public audit bodies across the UK;
- Publication of the Policy Briefing and of further academic and professional publications (see 'Academic Beneficiaries'); and,
- Development of further research building on this project and engaging the proposed network in collaborative design of the key questions and hypotheses.

Publications

10 25 50
 
Description The project found that benchmarking and external performance assessment of local public services is a transnational phenomenon which has become a central instrument for improving the performance of the public sector. While the UK has been at the forefront of developments in this area, the project revealed that there was much to learn through knowledge exchange with other countries, both within the UK and internationally. We also found that there was a large amount of untapped potential for countries to learn from each other, and for practitioners, policy makers and academics from each other.

We found that little is known about how different benchmarking and external performance assessment systems are developed, their theory of improvement, their use as an instrument of governance (particularly in federal systems) or how they fit with other improvement mechanisms. We feel, therefore, that the significant time invested in writing up reports from each of the international conferences and UK seminars and sharing them with participants and others on an on-line forum (LGA 'Knowledge Hub') was time well spent so that knowledge across countries can be exchanged. Practitioners clearly can make use of and apply academically generated conceptual frameworks if the communication and engagement is tuned to their needs and interests.

While benchmarking and external performance assessment have the potential to be useful tools for government in determining whether services are being delivered effectively and efficiently, we found that all benchmarking systems face considerable challenges in creating and capturing robust and comparable indicator data. There are a number of important points to bear in mind in trying to manage these challenges. First, robustness can only be judged in relation to the purpose for which the data is to be used (fitness for purpose). Second, it is difficult to standardise models of data collection because of the variety of interests (e.g. government and citizens). Third, most benchmarking systems capture historical rather than real-time data. Fourth, benchmarking requires open collaboration and co-operation between organisations and sectors. Fifth, producing good comparative data is only one step in the benchmarking process; an equally important step is to provide analysis and interpretation of this data as this narrative is likely to influence the impact of benchmarking. Finally, what is the role of citizens and service users in benchmarking? The public are rarely involved in discussions about what the indicators should be, what they mean, or what should change due to the results of benchmarking. The design of a new benchmarking or external performance assessment framework needs to take all of these issues into account.

Meeting the grant objectives: The project aimed to draw upon knowledge from research on external performance assessment and benchmarking to foster two-way dialogue between policy-makers, practitioners and academics to help improve the assessment of public services. It had six main objectives as set out below with a commentary on each:

1. Enhance the impact of existing social science research on performance assessment in public services through an exchange of experiences and practices within the UK and internationally: We co-designed an ambitious programme of initiatives which included international conferences, UK seminars, dissemination exchange workshops and setting up a network of policy-makers, practitioners and researchers. We have managed to deliver all of these initiatives and the events were well attended by all the key stakeholders.

2. Develop and strengthen the networks of social science researchers and non-academic stakeholders working on performance assessment. We did this through systematic engagement and co-production of the events. The involvement of users at the design stage was a key success factor for the UK seminars on external performance assessment and benchmarking. We held a number of face-to-face meetings with policy makers and practitioners in each country to scope the agenda and discuss potential invitees. Participants appreciated being part of the planning of these events which resulted in each seminar being tailored to the particular interests of each country.

3. Promote the application of conceptual frameworks and practical insight from research to inform the design and implementation of performance assessment: Each event was co-designed with potential users of the research so that the content was bespoke to each country. For example, the Australian conference was co-designed with representatives from the Council of Australian Governments (COAG). They were interested in hearing about international case studies and discussing how these could provide insight for their own policy-making. A member of the project team gave a presentation at each conference outlining the aims of the project, summarising learning to date and sharing future plans. A report from each conference was produced and shared with all participants.

4. Provide user communities with evidence of good practice to guide the future design and implementation of performance assessment frameworks: Feedback from the various knowledge exchange seminars has been overwhelmingly positive. For example, participants in Northern Ireland welcomed the opportunity provided by the seminar to hear about approaches elsewhere in the UK as they were 'behind the curve' and been 'in our own wee bubble'. This seminar triggered the commissioning of an additional piece of work which has applied and extended our previous analysis of performance assessment and benchmarking in the UK.

5. Prepare the way for co-production of future research on performance assessment by involving user communities in defining research questions, identifying data sources and securing access to key organisations and interviewees: (This is dealt with under Q.2.)

6. Extend the impacts of the knowledge exchange activities to a wider audience through the production and active dissemination of a policy briefing on best practice and lessons learnt: We produced and distributed a policy briefing which included a review of the lessons learned and key themes of the project, together with 15 pieces on benchmarking from a wide range of jurisdictions and covering a wide range of services and activities. It was published through a national newspaper web site, and also distributed through key channels of local government bodies.
Exploitation Route Our research outcomes have been and will be taken forward through both academic and non-academic routes. It will be important to do so for two specific reasons:

1. The strengthening trend in the UK towards a more devolved and even federal governance arrangement will increase the relevance not only of benchmarking generally, but also the relevance of experience of benchmarking public services in federal environments.
2. The sustained and continued need for public expenditure reduction, coupled with the opportunities for service improvement and innovation which flow from the increasing application of digital technologies to both public administration back office and client facing services, make benchmarking as a method very significant for both cost comparison and for learning. In this context the conceptual development through the project of differentiating modes of benchmarking in relation to its varying purposes is likely to be of enduring value.

In terms of specific actions:

a) Our co-funder (Forum of Federations) continues its' work in the area of public services benchmarking and uses the policy briefing. The policy briefing drew on 'best practice' examples from a number of countries including the USA, Canada, Australia, China, England, Scotland and Wales and has the potential for further impact on policy and practice given its' range of international experiences and evidence base.
b) We continue to liaise periodically with the network of collaborators we have built. We are considering how best to explore opportunities for further research in the area ourselves.
c) We continue to apply the findings and themes of the work in related projects and consultancy.
Sectors Government, Democracy and Justice

 
Description One of the findings from our previous ESRC-funded project (166-25-0034) was that there was insufficient attention paid to opportunities for policy learning. This knowledge exchange project aimed to debate the strengths and weaknesses of different approaches of external performance assessment and encourage policy transfer between countries. We have achieved this through a series of events which have been well-attended by all the key stakeholders including government departments, inspectorates and improvement agencies across the UK. We set up an on-line network of policy-makers, academics and practitioners to exchange knowledge on benchmarking and external performance assessment. All presentations and reports from the project have been loaded onto this site. It has taken considerable effort to set up and manage this network, but it has been a good investment of time and has the potential to act as a spring-board for a future research proposal and for practitioners to draw on. Thus, for example, the materials have recently been introduced at a senior level into the Department for Business, Innovation and Science as a contribution to their interest in establishing benchmarking in the shared services area. The research team have continued to conduct research projects in this area and to publish findings in academic journals. Members of the team conducted research on the intervention process in the Isle of Anglesey County Council and evaluated two approaches to sector-led improvement for the Local Government Association - the corporate peer challenge in local government and the fire peer challenge. There is also direct application of the outputs of the project into a three year project in Kosovo to improve a benchmarking based system of performance of public services at municipal level. These impacts are contributions to the two major themes affecting both the UK and many other advanced and developing jurisdictions. Benchmarking is a key methodology for helping to identify better economy in public services through input and process benchmarks. It is also a tool for improved effectiveness and innovation though output and outcome benchmarks, and through the learning by comparison which is at the heart of the benchmarking methodology. Further, in devolved, decentralised and quasi-federal jurisdictions - which is the growing trend globally in terms of national governance - it has special relevance as a comparative methodology to support both equitable resource distribution and also cross learning and improvement. Our findings have been used by the UK government, the Scottish Government and the Welsh Government. This can be evidenced through citations to our research and the commissioning of additional research.
First Year Of Impact 2013
Sector Government, Democracy and Justice
Impact Types Policy & public services

 
Description Benchmarking : What is it? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Presentation to the Local Government and Regeneration Committee in relation to Strand 2 of its Inquiry into Public Services Reform

Our research was cited by the Scottish Government.
Year(s) Of Engagement Activity 2012
 
Description Benchmarking Public Services for Excellent Performance 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Article for Cardiff Business School's Synergy magazine

We received contacts from the Welsh Government to find out more about our research.
Year(s) Of Engagement Activity 2013
 
Description House of Lords seminar 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The seminar was chaired by Lord Bichard at the House of Lords introduced by Clive Grace, who edited a special issue of the journal Public Money & Management. An eminent panel drawn from the authors made the case for benchmarking as an approach as well as exploring its limitations.

It helped to raise the profile of our ESRC research as we also issued a press release.
Year(s) Of Engagement Activity 2013
 
Description Notes from the Northern Ireland Roundtable on Performance Assessment 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This note provides a summary of the discussions held at the seminar held at Belfast City Hall on 29th November 2012.

Our presentation provoked an invitation to work with Belfast City Council on performance management issues which had reach into national policy debates.
Year(s) Of Engagement Activity 2012
 
Description Notes from the UK Roundtable on Performance Assessment 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact This event, the concluding part of a knowledge exchange programme on sharing experiences from across the UK and internationally, was organised by an international team of researchers from Cardiff and St. Andrews Universities. The seminar was hosted at the Local Government Association.



The event provided an important opportunity to exchange knowledge and experiences between policy makers, practitioners and academics from England, Scotland, Wales and Northern Ireland, drawing also on experiences from other countries and from previous events in the knowledge exchange programme.

Our research fed into a UK Government White Paper
Year(s) Of Engagement Activity 2013