Centre for the Evaluation of Complexity Across the Nexus (CECAN)

Lead Research Organisation: University of Surrey
Department Name: Sociology

Abstract

Responding to the increasing recognition of complexity of policy and policy implementation, CECAN produced innovation in policy evaluation for policies relating to energy, food, water and the environment (the 'Nexus'). CECAN developed and extended methods for policy evaluation, established case study based co-production between researchers, evaluation practitioners, policy makers and analysts and evolved an agile, problem driven way of running a multi-institutional, multi-disciplinary centre. While having produced high impact work (see Part 3: Key outputs and impacts to date), the kind of transformation CECAN set out to bring about is not achievable in 30 months. CECAN 2 is a consolidation of previous achievements, and a response to insights gained in the first phase and to new challenges faced by the UK. It will allow the Centre to develop and implement its innovations, knowledge exchange and co-production, with the goal of operating sustainably in the long term.

CECAN 2 is particularly timely as a number of drivers are coinciding to make complexity appropriate approaches to appraisal and evaluation ever more vital. Brexit is the most complex policy and regulatory change in the UK for decades, with associated impacts on the potentially increasingly divergent relationship between the devolved nations. The issue of "big data", both its opportunities and risks, is rarely far from the news, so a key future challenge is data linkage across Government departments and sectors, particularly in the context of public-private partnerships and hybrid models. Indirect and unintended effects are increasingly being demonstrated across the Nexus: for example, declines in rural infrastructure have adverse consequences for both the environment (more car transport) and health/wellbeing (more isolation and loneliness). Larger-scale and longer-term approaches are urgently needed to account for the cumulative impact of policies and "knock-on" effects across sectors. Finally, decision-making in both the public and private sector will be severely impacted by increasing automation, with currently unknown consequences for governance. Current Government management processes are not well-equipped to deal with these multiple challenges and urgently need to have evaluation and complexity thinking more embedded within them. CECAN 2 aims to ensure that the practitioners and policy-makers we engage with are equipped with the best methods to negotiate this future complexity .

The work envisaged for CECAN 2 is organised into five streams:
1: Exploring the contribution that complexity-appropriate evaluation methods can make across the full policy cycle.
2: CECAN Methods Innovation Lab: delivering innovation in complexity-appropriate evaluation methods by exploration of the full range of new and emerging methods.
3: Real world testing and refining of complexity-appropriate evaluation methods to co-produce solutions to the evaluation challenges posed by complexity in policy making and analysis.
4: Building individual and institutional capacity to deliver complexity-appropriate policy analysis
5: Securing CECAN's future, using ESRC's Centre Transition Funding to deepen existing partnerships and build new ones to secure alternative and long-term funding for CECAN's work.

Planned Impact

Following in the footsteps of CECAN 1, impact will be achieved through developing innovative methods and translating them through co-produced case studies, knowledge exchange and capacity building. CECAN has produced high impact research and has set the ball of change rolling. CECAN 2 will allow this nascent change to gather momentum, to push the ball "over the hill" to engender change across the system. It will penetrate new applications, new areas and new audiences to ensure change is rolled out and sustained in the long term.

1. New Areas
Branching out into new policy areas will facilitate the development of more joined up governance and better integration across government departments and associated agencies and organisations. CECAN 2 will help people take advantage of the opportunities complexity offers to manage intractable issues across the Nexus and wider policy areas. Taking the learning from the Nexus, CECAN 2 will work on the application of CECAN methods and concepts to the current long term goals in industrial and environmental policy areas. The complexity appropriate, systems-based thinking will be particularly important during the Brexit process: CECAN 2 will take on the challenges that Brexit brings to policy making so impacting on the complex regulatory disentanglement of the UK from the EU.

2. New Applications
CECAN 2 will widen the impact from the evaluation part of policy making to develop complexity appropriate policy appraisal and monitoring. It will go beyond the policy programme level and think strategically about the policy cycle as a whole to produce an integrated complexity appropriate policy framework. CECAN 2 will also go down into the policy delivery level, bridging the divide between policy researchers, analysts and decision makers.

Departmental and everyday work constraints can make it difficult to look at problems and their solutions across different scales. CECAN 2 will provide a learning space to allow people to break out of organisational and disciplinary silos. Through the integration of a wider set of policy areas and the whole policy cycle, CECAN will improve efficiencies and save money to deliver better outcomes through complexity responsive, agile policy making.

3. New Audiences
Defra and BEIS will remain at the centre of CECAN. However, the work on the Nexus has shown that a wider set of policy areas needs to be incorporated, by going beyond the government departments currently involved. This expansion has already begun with bespoke training courses delivered to the Department for Education, a CECAN fellow working with the Department of Health and Social Care and collaborations with the Department for Transport and the Ministry of Housing, Communities and Local Government.

CECAN 2 will also go beyond the public sector. We have already begun this process of extending our network and it will be accelerated during CECAN 2. We have begun work with Anglian Water and for CECAN 2 we envisage working with NESTA, the National Trust, the Scottish and Welsh Governments, and management consultancies such as Deloitte, KPMG and PwC, often as expert partners within consortia bidding for evaluation contracts. We also intend to work with multinational bodies such as the EU, the World Bank and the UN in relation to the SDGs.

To summarise, building on impact from CECAN 1, the proposal for CECAN 2 is driven by the recognition that the long term, radical change needed for complexity appropriate policy making needs to integrate the whole of the policy cycle and new policy areas; this automatically leads to the involvement of new audiences. CECAN 2 sees complexity as an opportunity for more joined up work across government departments, agencies and organisations, helping to reduce replication and waste and allowing for more responsive, agile and efficient policy making.

Publications

10 25 50
 
Description The main conclusions we have drawn from the work of CECAN so far are: Linear methods such as Randomised Control Trials are simply not suitable for the effective evaluation of the sorts of complex policies that many Government Departments are responsible for. All government interventions in social and environmental domains are acting on complex systems. The best policy teams, analysts and evaluators in government acknowledge this, and are taking steps to address it.

There are some occasions where evaluations may not need to be complexity-appropriate, though these are rare, and the emphasis should be on being sure that this is the case, rather than assuming simplicity. Government departments crave the time and space to develop capacity for complexity-appropriate evaluation. Without a clear picture of available solutions, they may struggle to articulate clearly their complexity-related needs; evaluators must give a strong and creative lead to government for successful complexity-appropriate evaluation. There is not enough of a focus on cross-departmental evaluation to deal with unintended consequences and Nexus interactions beyond one policy area. Nexus-related objectives and outcomes must be considered. Acknowledging the complexity of an area highlights the importance of decisions around the scope of an evaluation; artificially separating questions about the intervention, knock-on effects, and its context, may be counter-productive.

From the start of the second phase of CECAN, we have taken this approach to evaluation and developed it in a number of outputs. Most importantly, we have laid out the approach in a Supplementary Guide to HM Treasury's Magenta Book (see Narrative Impact for details). This joint endeavour, involving most members of the CECAN team as well as advice from the Cross-Government Evaluation Group and others, gave us the opportunity to consider in more detail what complexity-appropriate evaluation is and how it may be carried out. These ideas are being further developed and refined in writing a set of academic articles that will appear towards the end of 2020 as a special issue of the journal 'Evaluation'.

To support Participatory System Mapping, the Centre has written a web-based app that allows stakeholders to collectively create a system map, collaborating in real time. This will shortly be trialled with BEIS policy analysts and stakeholders in the logistics sector.
Exploitation Route The methods we have developed and refined, such a participatory system modelling, can be used by others (and, indeed, are already being used by by a variety of projects, such as Local Institutions, Productivity, Sustainability and Inclusivity Trade-offs (LIPSIT) [ES/T002468/1].
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Energy,Environment,Healthcare,Government, Democracy and Justice

URL https://www.cecan.ac.uk
 
Description The most impactful of our outcomes is the Supplementary Guide to the Magenta Book. The Magenta Book, published by HM Treasury, is the official government guide about policy evaluation. The Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation is based on three years' research and development of evaluation methods by CECAN. It will be published in early 2020 by HM Treasury to accompany the 2020 edition of the Magenta Book. It will serve as a guide to complexity-appropriate evaluation for the whole of government, and will also be influential in the design of policy evaluation in other sectors (e.g. by charitable funders, Research Councils and businesses). We have also produced the Complexity Evaluation Framework under contract to Defra. This provides a tailored guide relevant to socio-environmental evaluation for Defra social researchers. Discussions are ongoing about adapting the Framework for other government departments. As noted in the Outcomes section, we continue to organise a series of very popular webinars on evaluation, we have now published 16 Evaluation Policy and Practice Notes, and we continue to run training courses for continuous personal development (CPD). In the near future some of these courses will be run in association with the UK Evaluation Society to increase their reach.
First Year Of Impact 2019
Sector Communities and Social Services/Policy,Environment
Impact Types Policy & public services

 
Description Bespoke course on participatory systems mapping (SIPHER project), 30th Sept 2019
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact Alex Penn and Pete Barbrook-Johnson (Senior Research Fellows in CECAN) recently delivered a bespoke 1-day course on participatory systems mapping for researchers and policy makers in the new SIPHER project. SIPHER is an ambitious systems science project focussed on public health issues at different geographical scales in the UK, it includes policy makers, embedded policy researchers, public health experts, and academic modelling experts. They were keen to learn about CECAN's approach to systems mapping, understand its strengths and weaknesses, and how they might apply it.
URL https://sipher.ac.uk/systems-mapping-workshop-delivered-by-alex-and-pete-from-cecan/
 
Description CECAN Ltd workshop (CPD): Participatory Systems Mapping, Wednesday 26th June 2019
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact "CECAN Ltd workshop (CPD): Participatory Systems Mapping, Wednesday 26th June 2019, 09:30 - 17:00, Regent's University London NW1 4NS. This course will introduce the innovative participatory systems mapping methodology being developed by researchers in CECAN, which allows us to bring together diverse knowledge and stakeholders in order to construct shared causal maps of an issue or system. The course will introduce the theory behind and analytical approaches to systems maps, as well as providing practical training in map construction with significant time spent in hands on map construction and interrogation. Particular focus will be given to how these methods will be used in participatory approaches and how they contribute to policy appraisal and evaluation. This course is part of the CECAN Module for future policy analysts and evaluators. The syllabus was developed to support capacity building nationally and internationally in the evaluation of complex policy and programmes. Intended Audience: Policy analysts, evaluation professionals, consultants and academic researchers. Learning outcomes: By the end of this session, participants will have gained: • An understanding of how participatory systems mapping contributes to understanding causality and interaction in complex systems; • An understanding of the contexts in which it can be used and how it fits in relation to other social research methods and modelling approaches • An understanding of how to construct, or facilitate the construction of, appropriate systems maps with a variety of stakeholders • Be aware of a range of analysis methods which can be deployed on maps as appropriate • An understanding of how to use this approach for informing policy development and evaluation design.
URL https://mailchi.mp/43d8e7411337/cecan-seminar-invitation-we-ask-both-too-much-and-too-little-of-prog...
 
Description CECAN Ltd: Complex Evaluation Workshops for Public Health England, Jan 2020
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact On 9th Jan 2020, Pete Barbrook Johnson and Dione Hills ran the first in a set of bespoke workshops for Public Health England as part of the Complex Evaluation Workshops tender that we submitted in November 2019. 20 participants.
 
Description Nigel Gilbert appointed Research Expert and Academic Advisory Panel of National Leadership Centre (NLC), Oct 2019
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
URL https://www.nationalleadership.gov.uk/research
 
Description Reference to CECAN in Defra ITT (M&E for ELM), Mar 2019
Geographic Reach National 
Policy Influence Type Citation in other policy documents
 
Description Reference to CECAN in NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation, Mar 2019
Geographic Reach National 
Policy Influence Type Citation in other policy documents
 
Description Reference to CECAN in the CEDIL Evaluation Agenda, July 2019
Geographic Reach National 
Policy Influence Type Citation in other policy documents
 
Description Surrey Sociology Short Courses: Agent-Based Modelling for the Social Scientist, 10 - 12 February 2020 (Corinna Elsenbroich)
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact "Surrey Sociology Short Courses: Agent-Based Modelling for the Social Scientist, 10 - 12 February 2020 (Corinna Elsenbroich) Simulating social interactions in virtual research labs using agent-based modelling is increasingly allowing researchers to gain new insights into the complex ways that individuals and societies function. In this course, you will be introduced to foundation theoretical and practical aspects, the process of agent-based modelling, from conceptualising a research question, where to obtain data, operationalisation and formalisation of data, model implementation, and model analysis and interpretation. In addition to the theoretical content, you will learn NetLogo as a programming language for agent-based models. "
URL https://www.cecan.ac.uk/events/university-surrey-social-research-methods-short-courses
 
Description Defra Invitation to Tender: Invitation to Tender for Complexity Evaluation Framework, Tender Reference: ITT_4883 (bid via CECAN Ltd)
Amount £58,304 (GBP)
Organisation Department For Environment, Food And Rural Affairs (DEFRA) 
Sector Public
Country United Kingdom
Start 01/2019 
End 05/2019
 
Description GCRF pump priming project: Peri-Urban Resilient Ecosystems (PURE), (Adam Hejnowicz and Richard Gunton) (£unknown amount)
Amount £0 (GBP)
Organisation Stockholm Environment Institute (SEI) 
Sector Academic/University
Country Sweden
Start 01/2020 
 
Description Local Institutions, Productivity, Sustainability and Inclusivity Trade-offs (LIPSIT)
Amount £715,318 (GBP)
Funding ID ES/T002468/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 10/2019 
End 03/2021
 
Description PHE Invitation to Tender: Building capacity to evaluate complex system interventions to tackle health inequalities (bid via CECAN Ltd)
Amount £10,000 (GBP)
Organisation Public Health England 
Sector Public
Country United Kingdom
Start 01/2020 
 
Description Surrey County Council Invitation to Tender: Forces Connect South East - Veterans' Hubs Project External Evaluation (bid via CECAN Ltd)
Amount £8,000 (GBP)
Organisation Surrey County Council 
Sector Public
Country United Kingdom
Start 01/2020 
End 05/2020
 
Description University of Surrey Faculty of Arts and Social Sciences (FASS) pump-priming fund (bid from Pete Barbrook-Johnson)
Amount £2,000 (GBP)
Organisation University of Surrey 
Sector Academic/University
Country United Kingdom
Start 07/2019 
 
Description University of Surrey's QR Strategic Priority Funds (SPF) from Research England
Amount £16,100 (GBP)
Organisation University of Surrey 
Sector Academic/University
Country United Kingdom
Start 01/2020 
End 03/2020
 
Description CECAN Ltd and UKES Masterclasses trial 
Organisation UK Evaluation Society
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution CECAN Ltd is working closely with the UK Evaluation Society (UKES) to jointly provide three 'Method Masterclasses' in 2020, covering Qualitative Comparative Analysis (QCA), Process Tracing and Bayesian Updating. The first Masterclass, to be held in London, will be piloted in Q1 2020, with the two further classes then planned for Q2.
Collaborator Contribution As above
Impact TBC
Start Year 2020
 
Description CECAN Ltd: DFID Framework: International Multi-Disciplinary Programme Lot 20, Research (High Value) - Consortium led by LTS - June 2019 
Organisation LTS International Ltd
Country United Kingdom 
Sector Private 
PI Contribution CECAN Ltd is a member of a consortium led by LTS Consultancy, which submitted a successful tender to secure a place on the DFID Framework: International Multi-Disciplinary Programme Lot 20, Research (High Value). At a launch event for the successful Framework suppliers held on 7th June 2019, DFID confirmed that they foresee 80% of routine procurements to be conducted through the IMDP Framework with an estimated value of £1 - 3 million over a 4 year period (2 + 1 + 1 years). The frameworks are also open for other UK government departments to use for their procurements.
Collaborator Contribution As above
Impact TBC
Start Year 2019
 
Description CECAN Ltd: Water Systems Mapping project at Defra led by Mott Macdonald, Oct 2019 
Organisation Mott Macdonald UK Ltd
Country United Kingdom 
Sector Private 
PI Contribution CECAN Ltd submitted a successful bid with Mott Macdonald and Risk Solutions to Defra to work on a Water Systems Mapping project (Invitation to Tender for: Systems Analysis of Water Resources WT15121 (Project 27436)). The project will run from approx. Nov 2019 - June 2020. Income to CECAN Ltd (as sub-contractor, hence not recorded under Further Funding) of approx. £37.5k. Alex Penn and Pete Barbrook-Johnson are leading on this work. Alex will utilise her knowledge and experience of systems analysis methods and their application in the design of the workshops and delivery of the maps and training. Alex also has significant experience in stakeholder engagement and workshop facilitation. Peter is currently working with Anglian Water and the South Lincolnshire Water Partnership on governance issues. His experience and knowledge from this and previous systems mapping projects for Defra and other regulators will be invaluable to the delivery of this project.
Collaborator Contribution Mott MacDonald will provide administration support staff to assist with workshop logistics. This will allow the technical project team to concentrate on workshop design and engaging with stakeholders. MM staff will also act as Project Manager and Institutional Advisor, bringing existing connections with key stakeholders.
Impact TBC
Start Year 2019
 
Description Adam Hejnowicz is showcasing the work of CECAN at the YESI Citizen Food Assembly event, 8th Nov 2019 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Adam Hejnowicz is showcasing the work of CECAN at the YESI Citizen Food Assembly event (part of ESRC Festival of Science)
Year(s) Of Engagement Activity 2019
 
Description CECAN Blogs (active and ongoing - regularly updated) 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Members of the CECAN Team, CECAN Fellows, CECAN Partners and others regularly contribute to the CECAN Blog, which can be found on the CECAN website. Posts to date include: CECAN contributes to Norway Anniversary Evaluation Conference (08/10/2019 - 09:38), Complexity and Evaluation Failure (07/08/2019 - 10:04), Value, Values and Valuing Nature (25/06/2019 - 20:58), Slipping into simplicity (24/06/2019 - 09:47), How to evaluate complex research impact (26/04/2019 - 08:31). NOTE: The following blog posts are still present on the CECAN website but were created before 1/3/19 so have been logged under CECAN1 award. FOR REFERENCE ONLY: Complexity and Evaluative Thinking (21/01/2019 - 09:44) What has economic pluralism got to do with understanding complexity? (17/12/2018 - 09:40) The Story of the Pluralistic Evaluation Framework (19/11/2018 - 10:55) Power and Hope: Using Evaluation in Volatile Contexts (24/10/2018 - 14:27) The Nexus: A New Approach to Sustainability Transformations - What, Why and How (12/09/2018 - 11:38) CECAN Fellowship Blog: Evaluative methodologies for the transition towards an institutional ecosystem approach 'system-cultures' within Natural Resources Wales and CECAN (27/07/2018 - 13:22) A Pluralistic Evaluation Framework (11/06/2018 - 21:06) The Defiance of Global Commitment (15/05/2018 - 21:36) Looking Through a Glass Darkly: Public Goods and Agricultural Policy(09/03/2018 - 14:38) Humility and courage: strategy and evidence in our complex world (27/02/2018 - 20:25) Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes (22/02/2018 - 22:20) Finding The Common Ground (21/02/2018 - 17:46) Carillion may have collapsed, but public-private partnerships can be so much more... (08/02/2018 - 18:07) Teaching Evaluation of Complex Policy and Programmes (12/01/2018 - 16:29) Putting Values Back in Evaluation. (03/01/2018 - 15:30) Building Collaborative Narratives and Developing Trust in Policy and its Evaluation. (13/12/2017 - 20:13) An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!(15/11/2017 - 12:26) Travels in Evaluation - Systematising the Complex, Enjoying the Journey.(15/11/2017 - 12:20) CECAN's 'The Visual Representation of Complex Systems' at Environment, Economy, Democracy: Flourishing Together #RSD6.(13/11/2017 - 15:51) A Whole New World - What Next? (12/09/2017 - 16:38) Complexity Settings to the Rescue: A New Lease of Life for Evidence-Based Policy?(28/08/2017 - 21:14) Models as 'Interested Amateurs' (09/08/2017 - 10:33) Taking The Temperature of Trust - Henry Leveson Gower (03/07/2017 - 20:44) One Researcher's Anecdote is Another Researcher's Data (09/03/2017 - 11:44) That Way Lies Prosperity: Sustainability and the Nexus(07/02/2017 - 17:24) Agriculture Fit For A Complex World (16/11/2016 - 11:28) Why We Need Network Analysis to Understand the Future of Economics(08/11/2016 - 13:59) CECAN Complexity in Evaluation Workshop: What we Did and What we Learned (01/11/2016 - 13:35) Why Carry Out Economic Evaluation? (01/11/2016 - 09:49) Access to Data is Crucial(25/10/2016 - 15:42) Aligning Policy and Evidence for the Age of Complexity(24/10/2016 - 15:43) Complexity High on the Agenda at the EES 2016 Biannual Conference(11/10/2016 - 14:20) Likelihoods (06/10/2016 - 14:02) Is Policy Evaluation Fit For Purpose? (26/09/2016 - 13:51) After the Brexit vote: What Next for the UK's Environment? (29/06/2016 - 13:51) How Should Academics Interact with Policy Makers? Lessons on Building a Long-term Advocacy Strategy.(22/06/2016 - 12:09) Reflections on Language and Complexity (22/06/2016 - 11:59) Should Academics be Expected to Change Policy? Six Reasons Why it is Unrealistic for Research to Drive Policy Change.(26/05/2016 - 10:03) The Science of Using Research... (24/05/2016 - 21:52) Clearing The Fog(04/05/2016 - 14:44) Global Challenges Require Cross-Cutting Solutions(04/04/2016 - 16:35) Tools, Tools Everywhere and not a Hammer in Sight!(24/03/2016 - 18:48)
Year(s) Of Engagement Activity 2019,2020
URL https://www.cecan.ac.uk/blog
 
Description CECAN Intra Project Meeting, 20 - 21 November 2019, Guildford 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact "CECAN Intra Project Meeting, 20 - 21 November 2019, Guildford

Workshop with invited speakers: Jess Phoenix, Defra; Corey Schimpf (Complex-IT team); Kathryn Oliver, London School of Hygiene and Tropical Medicine. Circa 25 participants including CECAN team members"
Year(s) Of Engagement Activity 2019
 
Description CECAN Ltd took part in a University of Surrey hosted "Pitch@Palace" event, 23 Sept 19 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact CECAN Ltd took part in a University of Surrey hosted "Pitch@Palace" event in an effort to publicise university spin-outs and engage with new organisations. HRH the Duke of York - founder of Pitch@Palace - attended the event.
Year(s) Of Engagement Activity 2019
 
Description CECAN Ltd: launch event for Defra CEF, June 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact "CEF launch event, 5th June 2019, London

CECAN Ltd presented a first look at the new Complexity Evaluation Framework to an audience of policy-makers, analysts and commissioners in the Department for Environment, Food and Rural Affairs (Defra). The workshop included a discussion with Roisin O'Brien (Principal Social Researcher - Resources and Waste Evidence) and Tom Pye (Senior Policy Advisor - Resources, Waste and Plastics Strategy), hosted by CECAN Deputy Director Ben Shaw."
Year(s) Of Engagement Activity 2019
URL https://www.cecan.co.uk/projects/
 
Description CECAN Newsletter - Dec 2019 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact CECAN Newsletter Dec 2019 https://mailchi.mp/d58409fb1855/46i5hbo49y-4710021
Year(s) Of Engagement Activity 2019
URL https://mailchi.mp/d58409fb1855/46i5hbo49y-4710021
 
Description CECAN Newsletter - June 2019 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact CECAN Newsletter June 2019 https://mailchi.mp/f518f8346b87/46i5hbo49y-2048137
Year(s) Of Engagement Activity 2019
URL https://mailchi.mp/f518f8346b87/46i5hbo49y-2048137
 
Description CECAN Newsletter - March 2019 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact CECAN Newsletter, March 2019 https://mailchi.mp/bd4dde53ae9a/46i5hbo49y-1961397
Year(s) Of Engagement Activity 2019
URL https://mailchi.mp/bd4dde53ae9a/46i5hbo49y-1961397
 
Description CECAN Newsletter - Sept 2019 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact CECAN Newsletter Sept 2019 https://mailchi.mp/e542b558be3b/46i5hbo49y-2130917
Year(s) Of Engagement Activity 2019
URL https://mailchi.mp/e542b558be3b/46i5hbo49y-2130917
 
Description CECAN Twitter 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact CECAN has a Twitter account @cecanexus which is regularly used to post about CECAN events, seminars, blogs, papers, policy practice notes, other achievements (eg. syllabus, toolkits). As of 11/3/20, the account has 1,226 followers (last count: 914 on 13/3/19), is following 199 people (last count: 185 on 13/3/19) and has made 763 Tweets (last count: 637 on 13/3/19). Almost all Tweets indicate some level of engagement with Followers, eg. usually 10 re-tweets / likes / comments, if not more. NOTE that this Twitter account started under CECAN1 in 2016 and has run continuously ever since. As advised by ESRC, all impacts from 1 March 2019 are recorded under CECAN's continuation funding, ES/S007024/1 hence this new entry.
Year(s) Of Engagement Activity 2019,2020
URL https://twitter.com/cecanexus?lang=en
 
Description CECAN Webinar - Citizens' juries: how public policy should be made?, Tuesday 9th July 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar - Citizens' juries: how public policy should be made?, Tuesday 9th July 2019, 13:00 - 14:00 BST

Presenter: Dr Malcolm Oswald"
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-citizens-juries-how-public-policy-should-be-made
 
Description CECAN Webinar: Co-creation of innovation: Group Concept Mapping (GCM) to value and engage More Knowledgeable Others in authoring and valuing complex systems, Tuesday 24th September 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: Co-creation of innovation: Group Concept Mapping (GCM) to value and engage More Knowledgeable Others in authoring and valuing complex systems, Tuesday 24th September 2019, 14:00 - 15:00 BST / 09:00 - 10:00 EDT

Presenter: Mary Kane, Concept Systems, Inc. (US)"
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-co-creation-innovation-group-concept-mapping-value-and-...
 
Description CECAN Webinar: Demi-Regs and the Characteristics of Realist Evaluation, Thursday 5th December 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: Demi-Regs and the Characteristics of Realist Evaluation, Thursday 5th December 2019, 16:00 - 17:00 GMT

Presenter: Justin Jagosh, Director for the Centre for Advancement in Realist Evaluation and Synthesis (CARES)

"
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-demi-regs-and-characteristics-realist-evaluation
 
Description CECAN Webinar: Mapping the Cognitive Landscape of Productivity in Northern Ireland: A Systems Approach to Understanding Productivity Policy, Wednesday 12th February 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: Mapping the Cognitive Landscape of Productivity in Northern Ireland: A Systems Approach to Understanding Productivity Policy, Wednesday 12th February 2020, 13:30 - 14:30 GMT

Presenter: Jen Nelles, Centre for Regional Economic and Enterprise Development (CREED), University of Sheffield Management School, Productivity Insights Network (PIN)"
Year(s) Of Engagement Activity 2020
URL https://www.cecan.ac.uk/events/cecan-webinar-mapping-cognitive-landscape-productivity-northern-irela...
 
Description CECAN Webinar: The Human, Learning, Systems approach to managing in complexity, Tuesday 1st October 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: The Human, Learning, Systems approach to managing in complexity, Tuesday 1st October 2019, 13:00 - 14:00 BST

Presenter: Dr Toby Lowe, Senior Lecturer in Public Leadership and Management, Newcastle Business School "
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-human-learning-systems-approach-managing-complexity
 
Description CECAN Webinar: Theory of Change - Getting the most out of it - Insights from a Practitioner, Tuesday 2nd July 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: Theory of Change - Getting the most out of it - Insights from a Practitioner, Tuesday 2nd July, 13:00 - 14:00 BST

Presenter: Bridget Dillon, Head of Evaluation Profession, Department for International Development, UK Government (DFID)"
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-theory-change-getting-most-out-it-insights-practitioner
 
Description CECAN Webinar: Using the System Effects methodology to understand the user experience of complex systems, Tuesday 16th July 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact "CECAN Webinar: Using the System Effects methodology to understand the user experience of complex systems, Tuesday 16th July 2019, 09:00 - 10:00 BST (18:00 - 19:00 ACT)

Presenter: Dr Luke K Craven, Research Fellow (Public Service Research Group), University of New South Wales, Canberra "
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-webinar-using-system-effects-methodology-understand-user-experi...
 
Description CECAN Workshop - Commissioning Complex Evaluations, Monday 29th April 2019, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact "CECAN Workshop - Commissioning Complex Evaluations, Monday 29th April 2019, 10:00 - 16:00, Prince Philip House, 3 Carlton House Terrace London SW1Y 5DG
In this workshop, we will:

Share experience on procuring complexity-appropriate methods, and how to get round some of the rigidities in tender processes

Pinpoint barriers to complexity appropriate methods in competitive tendering processes

Develop immediately applicable suggestions and solutions; and have an opportunity to debate these with procurement staff (outside the context of an actual tendering exercise) to suggest mutually workable solutions.

The workshop will bring together policy analysts from government departments who have to commission the evaluation of complex policies and contractors who have experience of bidding against ITTs for policy evaluation."
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/events/cecan-workshop-commissioning-complex-evaluations
 
Description CECAN Workshop for BEIS: Dealing with complexity in policy evaluation, BEIS Conference Centre, 4th October 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact "Dealing with complexity in policy evaluation, BEIS/CECAN Briefing and Workshop, BEIS Conference Centre, 4th October 2019

CECAN held a briefing and workshop session with BEIS on dealing with complexity in policy evaluation. Held at the BEIS conference centre in central London the session was well attended by over 20 BEIS policy-makers and analysts. The event was chaired by CECAN Deputy Director, Ben Shaw, and included presentations from Dione Hills and Helen Wilkinson (providing an overview of the principles of complexity plus a taster of some of the issues included in the forthcoming CECAN Annex on complexity to the revised Magenta Book - the Government's guidance on evaluation); Simon Shrimpton (from Defra) and Martha Bicket (re: the development of Defra's Complexity Evaluation Framework which CECAN produced); Alex Penn and Pete Barbrook-Johnson (a practical example of complexity appropriate methods being used in government, focused on Participatory systems mapping).
Year(s) Of Engagement Activity 2019
URL https://www.cecan.ac.uk/news/dealing-complexity-policy-evaluation-beiscecan-briefing-and-workshop
 
Description CECAN Workshop: "Participatory Systems Mapping", 18th - 20th March 2019, University of Surrey, Guildford 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Alex Penn and Pete Barbrook-Johnson delivered a ""Participatory Systems Mapping"" workshop 18th - 20th March 2019, University of Surrey, Guildford, UK

This course will introduce participants to the innovative participatory systems mapping methodology being developed by researchers in CECAN, which allows researchers to bring together diverse knowledge and stakeholders in order to construct shared causal maps of an issue or system. The course will introduce participants to the theory behind, and analytical approaches to, systems maps, as well as giving participants significant time spent in hands-on practical workshops building a map of their own. Time will also be given to considering the wider value and use of participatory modelling approaches such as systems mapping.

Indicative content includes:
Theory
• Background on fuzzy cognitive mapping, causal mapping, and dependency modelling approaches
• Analysis approaches (including simple dynamic modelling, network analysis, map interpretation for scenario exploration and intervention design combining quantitative nalysis and stakeholder beliefs) and their strengths and weaknesses.
• Broader consideration of the use and value of participatory modelling
Model construction
• Practical skills and tools to build, verify and refine a systems map alone or in a group.
Practical hands-on workshops will provide participants with experience of
• Building, verifying and refining a map of an area of participants' interest, alone or in a group
• Reflection on the faciltiation process
• Observation of, and reflection on, group dynamics during model construction
• Preliminary observation and discussion of map structure and potential implications

Learning Outcomes:
On successful completion of this course, participants will be able to:
• Demonstrate a critical understanding of the ideas behind systems mapping (C,K)
• Build a map on their own (C,P)
• Facilitate a group to build a map (C,P,T)
• Understand when a map might be used, and how it fits in with other methodologies. (C,K)
• Be able to critically engage with existing research using this and similar methods (C,T)
• Be able to analyse and interpret maps (C,K,P)
Key: C-Cognitive/Analytical; K-Subject Knowledge; T-Transferable Skills; P- Professional/ Practical skills
Year(s) Of Engagement Activity 2019
 
Description CECAN members delivered a systems mapping presentation as part of the Sunbelt Conference in Montreal, 20 June 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Former CECAN Fellow, Verena Knerich, and CECAN Senior Research Fellows Pete Barbrook-Johnson and Alex Penn delivered a presentation as part of the Sunbelt Conference in Montreal. The presentation was called "Structure-based Identification of Threats and Opportunities from Participatory Systems Mapping".
Year(s) Of Engagement Activity 2019
URL https://sunbeltconference2019.sched.com/event/Qudq/innovations-in-social-network-based-interventions...
 
Description Magenta Book Annex Roadshow Event 1 - Handling Complexity in Policy Evaluation: Introducing the Magenta Book 2020 Supplementary Guide on Complexity, Tuesday 3rd March 2020, London 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Magenta Book Annex Roadshow Event 1 - Handling Complexity in Policy Evaluation: Introducing the Magenta Book 2020 Supplementary Guide on Complexity, Tuesday 3rd March 2020, 13:30 - 16:30, Church House, Deans Yard, Westminster, London SW1P 3NZ. Circa 90 people registered with c. 70 in attendance. Policy analysts, consultants, evaluators etc. Invited speaker: Steven Finch from DfT and Government Cross Evaluation Group. Event was professionally filmed and will be posted on the CECAN website after the second of the two Roadshow events has taken place in Birmingham (16th March 2020).
Year(s) Of Engagement Activity 2020
 
Description Nigel Gilbert and colleagues invited to give the Keynote presentation at Surrey's Dept of Sociology Research Afternoon event, 22 May 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Nigel Gilbert, Ben Shaw and Pete Barbrook-Johnson were invited to give the Keynote presentation at Surrey's Dept of Sociology Research Afternoon event. 'What is policy evaluation and does it support evidence-based policy?' Event summary: This event considers the 'real world' challenges of the policy/research nexus. Drawing on the perspectives and experiences of academics in the Department of Sociology we will consider the role of policy evaluation and examine what evidence-based practice in the 'real world' might look like. Professor Nigel Gilbert, Dr Ben Shaw, Dr Pete Barbrook-Johnson will examine 'What is policy evaluation and does it support evidence-based policy?' and Professor Nigel Fielding will consider 'evidence-based practice in the real world'. Our panel will then broaden this out and discuss: 'what policy needs from research and what research needs from policy.' The panel comprises academics working in diverse policy environments. Dr Sara Dicerto is a Senior Commissioning Officer at Surrey County Council. Ms Megan Georgiou is a Programme Manager, Royal College of Psychiatrists. Dr Paul Quinton is an Evidence and Evaluation Advisor at College of Policing.
Year(s) Of Engagement Activity 2019
URL https://www.eventbrite.com/e/evidence-based-policy-and-the-social-sciences-tickets-59853840448?utm_s...
 
Description Nigel Gilbert delivered a bespoke presentation ("Evaluating Complexity") for the Department of Health and Social Care, 11 Sept 19, London 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Nigel Gilbert delivered a bespoke presentation ("Evaluating Complexity") for the Department of Health and Social Care. The presentation took place at 39 Victoria Street, London, SW1H 0EU and was live-streamed to DHSC offices in Leeds. The presentation was delivered to DHSC Ops Research (OR) analysts covering complex evaluation and how it links to soft OR methods. It also included signposting to useful further reading/learning resources etc available via the CECAN website. c. 40 participants.
Year(s) Of Engagement Activity 2019
 
Description Nigel Gilbert presented CECAN's work at University of Surrey's Faculty of Arts and Social Sciences' Festival of Research, 23 Jan 2020 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Nigel Gilbert gave a presentation and presented a poster about CECAN's work at the University of Surrey's Faculty of Arts and Social Sciences' Festival of Research
Year(s) Of Engagement Activity 2020
URL https://www.surrey.ac.uk/news/celebrating-years-fass-festival-research
 
Description Nigel Gilbert was invited to talk at the Cross Government Evaluation Group meeting, 23 May 2019 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Nigel Gilbert was invited by Steven Finch, Head of Evaluation, Evaluation Centre of Excellence, Department for Transport, to talk at the Cross Government Evaluation Group meeting on Thurs 23 May at HMT. Nigel gave an hour-long presentation, which covered: 1. Update on CECAN's programme if work, learnings to date, current plans following the Centre's extension from ESRC; 2. Reflections from the Commissioning Complex Evaluations workshop. Other attendees at the meeting included Marianne Law (BEIS), Bridget Dillon (DFiD), Tarran Macmillan (Defra), Mike Daly (DWP) and Steven Finch (DfT).
Year(s) Of Engagement Activity 2019
 
Description Pete Barbrook-Johnson delivered an informal seminar "Exploring complex data from a case-based perspective: the COMPLEX-IT software", 19 Nov 19, Guildford 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact At University of Surrey, Dr. Pete Barbrook-Johnson (Senior Research Fellow, Surrey) and Dr. Corey Schimpf (Learning Analytics Scientist, Concord Consortium) delivered an informal seminar entitled "Exploring complex data from a case-based perspective: the COMPLEX-IT software". In this seminar, we will present our recent work which brings together, and makes accessible, traditional and machine learning cluster analysis in an easy-to-use software interface called COMPLEX-IT. We will describe how you can use COMPLEXIT to explore your data and uncover complex patterns. In addition, we will outline current plans to extend the software to include 'quick-and-dirty' agent-based modelling of your topic. Corey's visit was paid for as part of a Surrey Faculty pump-priming award and the work on COMPLEX-IT is part of the wider research of CECAN.
Year(s) Of Engagement Activity 2019
 
Description Presentation of Pluralistic Evaluation Framework methodology at a British Ecological Society conference, Bristol, 3 March 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Adam Hejnowicz and and Richard Gunton presented the Pluralistic Evaluation Framework methodology (developed during Richard's CECAN Fellowship) at a British Ecological Society and Valuing Nature conference in Bristol
Year(s) Of Engagement Activity 2020
URL https://www.britishecologicalsociety.org/wp-content/uploads/2020/02/BES-symposium-brochure-feb-2020-...