Centre for the Evaluation of Complexity Across the Nexus (CECAN)
Lead Research Organisation:
University of Surrey
Department Name: Sociology
Abstract
Responding to the increasing recognition of complexity of policy and policy implementation, CECAN produced innovation in policy evaluation for policies relating to energy, food, water and the environment (the 'Nexus'). CECAN developed and extended methods for policy evaluation, established case study based co-production between researchers, evaluation practitioners, policy makers and analysts and evolved an agile, problem driven way of running a multi-institutional, multi-disciplinary centre. While having produced high impact work (see Part 3: Key outputs and impacts to date), the kind of transformation CECAN set out to bring about is not achievable in 30 months. CECAN 2 is a consolidation of previous achievements, and a response to insights gained in the first phase and to new challenges faced by the UK. It will allow the Centre to develop and implement its innovations, knowledge exchange and co-production, with the goal of operating sustainably in the long term.
CECAN 2 is particularly timely as a number of drivers are coinciding to make complexity appropriate approaches to appraisal and evaluation ever more vital. Brexit is the most complex policy and regulatory change in the UK for decades, with associated impacts on the potentially increasingly divergent relationship between the devolved nations. The issue of "big data", both its opportunities and risks, is rarely far from the news, so a key future challenge is data linkage across Government departments and sectors, particularly in the context of public-private partnerships and hybrid models. Indirect and unintended effects are increasingly being demonstrated across the Nexus: for example, declines in rural infrastructure have adverse consequences for both the environment (more car transport) and health/wellbeing (more isolation and loneliness). Larger-scale and longer-term approaches are urgently needed to account for the cumulative impact of policies and "knock-on" effects across sectors. Finally, decision-making in both the public and private sector will be severely impacted by increasing automation, with currently unknown consequences for governance. Current Government management processes are not well-equipped to deal with these multiple challenges and urgently need to have evaluation and complexity thinking more embedded within them. CECAN 2 aims to ensure that the practitioners and policy-makers we engage with are equipped with the best methods to negotiate this future complexity .
The work envisaged for CECAN 2 is organised into five streams:
1: Exploring the contribution that complexity-appropriate evaluation methods can make across the full policy cycle.
2: CECAN Methods Innovation Lab: delivering innovation in complexity-appropriate evaluation methods by exploration of the full range of new and emerging methods.
3: Real world testing and refining of complexity-appropriate evaluation methods to co-produce solutions to the evaluation challenges posed by complexity in policy making and analysis.
4: Building individual and institutional capacity to deliver complexity-appropriate policy analysis
5: Securing CECAN's future, using ESRC's Centre Transition Funding to deepen existing partnerships and build new ones to secure alternative and long-term funding for CECAN's work.
CECAN 2 is particularly timely as a number of drivers are coinciding to make complexity appropriate approaches to appraisal and evaluation ever more vital. Brexit is the most complex policy and regulatory change in the UK for decades, with associated impacts on the potentially increasingly divergent relationship between the devolved nations. The issue of "big data", both its opportunities and risks, is rarely far from the news, so a key future challenge is data linkage across Government departments and sectors, particularly in the context of public-private partnerships and hybrid models. Indirect and unintended effects are increasingly being demonstrated across the Nexus: for example, declines in rural infrastructure have adverse consequences for both the environment (more car transport) and health/wellbeing (more isolation and loneliness). Larger-scale and longer-term approaches are urgently needed to account for the cumulative impact of policies and "knock-on" effects across sectors. Finally, decision-making in both the public and private sector will be severely impacted by increasing automation, with currently unknown consequences for governance. Current Government management processes are not well-equipped to deal with these multiple challenges and urgently need to have evaluation and complexity thinking more embedded within them. CECAN 2 aims to ensure that the practitioners and policy-makers we engage with are equipped with the best methods to negotiate this future complexity .
The work envisaged for CECAN 2 is organised into five streams:
1: Exploring the contribution that complexity-appropriate evaluation methods can make across the full policy cycle.
2: CECAN Methods Innovation Lab: delivering innovation in complexity-appropriate evaluation methods by exploration of the full range of new and emerging methods.
3: Real world testing and refining of complexity-appropriate evaluation methods to co-produce solutions to the evaluation challenges posed by complexity in policy making and analysis.
4: Building individual and institutional capacity to deliver complexity-appropriate policy analysis
5: Securing CECAN's future, using ESRC's Centre Transition Funding to deepen existing partnerships and build new ones to secure alternative and long-term funding for CECAN's work.
Planned Impact
Following in the footsteps of CECAN 1, impact will be achieved through developing innovative methods and translating them through co-produced case studies, knowledge exchange and capacity building. CECAN has produced high impact research and has set the ball of change rolling. CECAN 2 will allow this nascent change to gather momentum, to push the ball "over the hill" to engender change across the system. It will penetrate new applications, new areas and new audiences to ensure change is rolled out and sustained in the long term.
1. New Areas
Branching out into new policy areas will facilitate the development of more joined up governance and better integration across government departments and associated agencies and organisations. CECAN 2 will help people take advantage of the opportunities complexity offers to manage intractable issues across the Nexus and wider policy areas. Taking the learning from the Nexus, CECAN 2 will work on the application of CECAN methods and concepts to the current long term goals in industrial and environmental policy areas. The complexity appropriate, systems-based thinking will be particularly important during the Brexit process: CECAN 2 will take on the challenges that Brexit brings to policy making so impacting on the complex regulatory disentanglement of the UK from the EU.
2. New Applications
CECAN 2 will widen the impact from the evaluation part of policy making to develop complexity appropriate policy appraisal and monitoring. It will go beyond the policy programme level and think strategically about the policy cycle as a whole to produce an integrated complexity appropriate policy framework. CECAN 2 will also go down into the policy delivery level, bridging the divide between policy researchers, analysts and decision makers.
Departmental and everyday work constraints can make it difficult to look at problems and their solutions across different scales. CECAN 2 will provide a learning space to allow people to break out of organisational and disciplinary silos. Through the integration of a wider set of policy areas and the whole policy cycle, CECAN will improve efficiencies and save money to deliver better outcomes through complexity responsive, agile policy making.
3. New Audiences
Defra and BEIS will remain at the centre of CECAN. However, the work on the Nexus has shown that a wider set of policy areas needs to be incorporated, by going beyond the government departments currently involved. This expansion has already begun with bespoke training courses delivered to the Department for Education, a CECAN fellow working with the Department of Health and Social Care and collaborations with the Department for Transport and the Ministry of Housing, Communities and Local Government.
CECAN 2 will also go beyond the public sector. We have already begun this process of extending our network and it will be accelerated during CECAN 2. We have begun work with Anglian Water and for CECAN 2 we envisage working with NESTA, the National Trust, the Scottish and Welsh Governments, and management consultancies such as Deloitte, KPMG and PwC, often as expert partners within consortia bidding for evaluation contracts. We also intend to work with multinational bodies such as the EU, the World Bank and the UN in relation to the SDGs.
To summarise, building on impact from CECAN 1, the proposal for CECAN 2 is driven by the recognition that the long term, radical change needed for complexity appropriate policy making needs to integrate the whole of the policy cycle and new policy areas; this automatically leads to the involvement of new audiences. CECAN 2 sees complexity as an opportunity for more joined up work across government departments, agencies and organisations, helping to reduce replication and waste and allowing for more responsive, agile and efficient policy making.
1. New Areas
Branching out into new policy areas will facilitate the development of more joined up governance and better integration across government departments and associated agencies and organisations. CECAN 2 will help people take advantage of the opportunities complexity offers to manage intractable issues across the Nexus and wider policy areas. Taking the learning from the Nexus, CECAN 2 will work on the application of CECAN methods and concepts to the current long term goals in industrial and environmental policy areas. The complexity appropriate, systems-based thinking will be particularly important during the Brexit process: CECAN 2 will take on the challenges that Brexit brings to policy making so impacting on the complex regulatory disentanglement of the UK from the EU.
2. New Applications
CECAN 2 will widen the impact from the evaluation part of policy making to develop complexity appropriate policy appraisal and monitoring. It will go beyond the policy programme level and think strategically about the policy cycle as a whole to produce an integrated complexity appropriate policy framework. CECAN 2 will also go down into the policy delivery level, bridging the divide between policy researchers, analysts and decision makers.
Departmental and everyday work constraints can make it difficult to look at problems and their solutions across different scales. CECAN 2 will provide a learning space to allow people to break out of organisational and disciplinary silos. Through the integration of a wider set of policy areas and the whole policy cycle, CECAN will improve efficiencies and save money to deliver better outcomes through complexity responsive, agile policy making.
3. New Audiences
Defra and BEIS will remain at the centre of CECAN. However, the work on the Nexus has shown that a wider set of policy areas needs to be incorporated, by going beyond the government departments currently involved. This expansion has already begun with bespoke training courses delivered to the Department for Education, a CECAN fellow working with the Department of Health and Social Care and collaborations with the Department for Transport and the Ministry of Housing, Communities and Local Government.
CECAN 2 will also go beyond the public sector. We have already begun this process of extending our network and it will be accelerated during CECAN 2. We have begun work with Anglian Water and for CECAN 2 we envisage working with NESTA, the National Trust, the Scottish and Welsh Governments, and management consultancies such as Deloitte, KPMG and PwC, often as expert partners within consortia bidding for evaluation contracts. We also intend to work with multinational bodies such as the EU, the World Bank and the UN in relation to the SDGs.
To summarise, building on impact from CECAN 1, the proposal for CECAN 2 is driven by the recognition that the long term, radical change needed for complexity appropriate policy making needs to integrate the whole of the policy cycle and new policy areas; this automatically leads to the involvement of new audiences. CECAN 2 sees complexity as an opportunity for more joined up work across government departments, agencies and organisations, helping to reduce replication and waste and allowing for more responsive, agile and efficient policy making.
Organisations
- University of Surrey (Lead Research Organisation)
- LTS International Ltd (Collaboration)
- DURHAM UNIVERSITY (Collaboration)
- UK Evaluation Society (Collaboration)
- Mott Macdonald UK Ltd (Collaboration)
- ICF International, Inc. (Collaboration)
- Collingwood Environmental Planning (United Kingdom) (Collaboration)
- Technopolis Group (Collaboration)
Publications
Yao R
(2023)
Applications of Agent-Based Methods in Multi-Energy Systems-A Systematic Literature Review
in Energies
Wilkinson H
(2021)
Building a system-based Theory of Change using Participatory Systems Mapping
in Evaluation
Technopolis And CECAN Ltd
(2021)
Defra Theory of Change (ToC) Tool - Final Report
Suno Wu J
(2021)
Participatory complexity in tourism policy: Understanding sustainability programmes with participatory systems mapping
in Annals of Tourism Research
Stewart B
(2019)
Making Brexit work for the environment and livelihoods: Delivering a stakeholder informed vision for agriculture and fisheries
in People and Nature
Sheate W
(2020)
Learning Lessons for Evaluating Complexity Across the Nexus: A Meta-Evaluation of Environmental Projects
in Journal of MultiDisciplinary Evaluation
Sedighi T
(2021)
Evaluating the Bovine Tuberculosis Eradication Mechanism and Its Risk Factors in England's Cattle Farms.
in International journal of environmental research and public health
| Description | The main conclusions we have drawn from the work of CECAN so far are: Linear methods such as Randomised Control Trials are simply not suitable for the effective evaluation of the sorts of complex policies that many Government Departments are responsible for. All government interventions in social and environmental domains are acting on complex systems. The best policy teams, analysts and evaluators in government acknowledge this, and are taking steps to address it. There are some occasions where evaluations may not need to be complexity-appropriate, though these are rare, and the emphasis should be on being sure that this is the case, rather than assuming simplicity. Government departments crave the time and space to develop capacity for complexity-appropriate evaluation. Without a clear picture of available solutions, they may struggle to articulate clearly their complexity-related needs; evaluators must give a strong and creative lead to the government for successful complexity-appropriate evaluation. There is not enough of a focus on cross-departmental evaluation to deal with unintended consequences and Nexus interactions beyond one policy area. Nexus-related objectives and outcomes must be considered. Acknowledging the complexity of an area highlights the importance of decisions around the scope of an evaluation; artificially separating questions about the intervention, knock-on effects, and its context, may be counter-productive. From the start of the second phase of CECAN, we have taken this approach to evaluation and developed it in a number of outputs. Most importantly, we have laid out the approach in a Supplementary Guide to HM Treasury's Magenta Book (see Narrative Impact for details). This joint endeavour, involving most members of the CECAN team as well as advice from the Cross-Government Evaluation Group and others, gave us the opportunity to consider in more detail what complexity-appropriate evaluation is and how it may be carried out. These ideas have been further developed and refined in a set of academic articles that were published at the beginning of 2021 as a special issue of the journal 'Evaluation'. To support Participatory System Mapping, the Centre has written a web-based app, PRSM, that allows stakeholders to collectively create a system map, collaborating in real time. This is being trialled in a number of virtual workshops with policy analysts and stakeholders in the logistics sector. |
| Exploitation Route | The methods we have developed and refined, such a participatory system modelling, can be used by others (and, indeed, have already been used by by a variety of projects, such as Local Institutions, Productivity, Sustainability and Inclusivity Trade-offs (LIPSIT) [ES/T002468/1] and ANTICIPATE [NE/T001852/1]. |
| Sectors | Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Energy,Environment,Healthcare,Government, Democracy and Justice |
| URL | https://www.cecan.ac.uk |
| Description | The most impactful of our outcomes is the Supplementary Guide to the Magenta Book. The Magenta Book, published by HM Treasury, is the official government guide about policy evaluation. The Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation is based on three years' research and development of evaluation methods by CECAN. It will be published in early 2020 by HM Treasury to accompany the 2020 edition of the Magenta Book. It will serve as a guide to complexity-appropriate evaluation for the whole of government, and will also be influential in the design of policy evaluation in other sectors (e.g. by charitable funders, Research Councils and businesses). We have also produced the Complexity Evaluation Framework under contract to Defra. This provides a tailored guide relevant to socio-environmental evaluation for Defra social researchers. This has been supplemented by the Complexity Evaluation Toolkit, which provides more generally applicable advice. As noted in the Outcomes section, we continue to organise a series of very popular webinars on evaluation. We have now published 17 Evaluation Policy and Practice Notes, and we continue to run training courses for continuous personal development (CPD). Some of these courses are being run in association with the UK Evaluation Society to increase their reach. With additional funding from ESRC Impact Acceleration Account funding, we have also produced training modules on Handling Complexity in Policy Evaluation, which have proved to be very popular, and in addition to being provided publicly, bespoke versions are being provided to several government departments and agencies (e.g MoJ, Natural England). CECAN has also set up a spin out company, CECAN Ltd, to help with the mobilisation of its knowledge into government and the private sector. This spin out is engaging with a range of government departments through contracts and sub-contracts, and with evaluation consultancies (by participating in consortia). |
| First Year Of Impact | 2019 |
| Sector | Communities and Social Services/Policy,Environment |
| Impact Types | Policy & public services |
| Description | Bespoke course on participatory systems mapping (SIPHER project), 30th Sept 2019 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Alex Penn and Pete Barbrook-Johnson (Senior Research Fellows in CECAN) recently delivered a bespoke 1-day course on participatory systems mapping for researchers and policy makers in the new SIPHER project. SIPHER is an ambitious systems science project focussed on public health issues at different geographical scales in the UK, it includes policy makers, embedded policy researchers, public health experts, and academic modelling experts. They were keen to learn about CECAN's approach to systems mapping, understand its strengths and weaknesses, and how they might apply it. |
| URL | https://sipher.ac.uk/systems-mapping-workshop-delivered-by-alex-and-pete-from-cecan/ |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module One - Complexity and its Challenges for Policy and Evaluation Online, 8 Sep 2021 Martha Bicket and Helen Wilkinson |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Be aware of the main properties of complex systems and be able to recognise these in policy-making processes and their own work, Have a shared language for discussing and exploring complexity, Understand the implications of complex systems for policy-making, analysis and evaluation - the importance of context, the limitations in complex settings of 'traditional' standardising and controlling approaches and the need for flexibility in managing and delivering evaluations. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-one-c... |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module One - Complexity and its Challenges for Policy and Evaluation, Online, 23 Feb 2022, Martha Bicket and Helen Wilkinson |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Be aware of the main properties of complex systems and be able to recognise these in policy-making processes and their own work, Have a shared language for discussing and exploring complexity, Understand the implications of complex systems for policy-making, analysis and evaluation - the importance of context, the limitations in complex settings of 'traditional' standardising and controlling approaches and the need for flexibility in managing and delivering evaluations. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/cpd-courses/handling-complexity-in-policy-evaluation-online-workshop-... |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module Three - Selecting an Appropriate Evaluation Approach, Online, 23 Sep 2021, Helen Wilkinson and Martha Bicket |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand the importance of using a complexity-appropriate approach to evaluation Be aware of the breadth and diversity of approaches that can be used in complex settings Be equipped with a set of tools that help identify an appropriate approach or approaches based on the evaluation purpose, the complexity attributes present, and the feasibility of approach for intended use. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-three... |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module Three - Selecting an Appropriate Evaluation Approach. Online, 16 Mar 2022. Helen Wilkinson and Martha Bicket |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand the importance of using a complexity-appropriate approach to evaluation Be aware of the breadth and diversity of approaches that can be used in complex settings Be equipped with a set of tools that help identify an appropriate approach or approaches based on the evaluation purpose, the complexity attributes present, and the feasibility of approach for intended use. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-three... |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module Two - Commissioning and Managing a Complex Evaluation Online, 14 Sep 2021 Richard Allen, Georgie Parry-Crooke, Helen Wilkinson and Martha Bicket |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand what it means to manage in complexity and how this applies to commissioning and managing a complex evaluation. Have a shared framework for commissioning and managing complex evaluations. Be able to specify the project/programme complexity to contractors, clarify what is expected from them. Understand how the contracting relationship can accommodate the complex nature of the evaluation (e.g. non- linear, adaptive and dealing with the unexpected). Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-two-c... |
| Description | CECAN CPD Course: Handling Complexity in Policy Evaluation Online Workshop: Module Two - Commissioning and Managing a Complex Evaluation, Online, 9 Mar 2022, Richard Allen and Georgie Parry-Crooke |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand what it means to manage in complexity and how this applies to commissioning and managing a complex evaluation. Have a shared framework for commissioning and managing complex evaluations. Be able to specify the policy/project/programme complexity to contractors and clarify what is expected from them. Understand how the contracting relationship can accommodate the complex nature of the evaluation (e.g. non- linear, adaptive and dealing with the unexpected). Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-two-c... |
| Description | CECAN Ltd / UKES Masterclass - Handling Complexity in Policy Evaluation Online Workshop: Module One - Complexity and its Challenges for Policy and Evaluation Online, 7 Feb 2023 Martha Bicket and Stuart Astill |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Be aware of the main properties of complex systems and be able to recognise these in policy-making processes and their own work, Have a shared language for discussing and exploring complexity, Understand the implications of complex systems for policy-making, analysis and evaluation - the importance of context, the limitations in complex settings of 'traditional' standardising and controlling approaches and the need for flexibility in managing and delivering evaluations. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/cpd-courses/handling-complexity-in-policy-evaluation-online-workshop-... |
| Description | CECAN Ltd / UKES Masterclass - Handling Complexity in Policy Evaluation Online Workshop: Module Three - Commissioning and Managing a Complex Evaluation Online, 8 Mar 2023 Georgie Parry-Crooke and Clare Twigger-Ross |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand what it means to manage in complexity and how this applies to commissioning and managing a complex evaluation. Have a shared framework for commissioning and managing complex evaluations. Be able to specify the policy/project/programme complexity to contractors and clarify what is expected from them. Understand how the contracting relationship can accommodate the complex nature of the evaluation (e.g. non- linear, adaptive and dealing with the unexpected). |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-three... |
| Description | CECAN Ltd / UKES Masterclass - Handling Complexity in Policy Evaluation Online Workshop: Module Two - Selecting an Appropriate Evaluation Approach Online, 21 Feb 2023 Helen Wilkinson and Richard Allen |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand the importance of using a complexity-appropriate approach to evaluation Be aware of the breadth and diversity of approaches that can be used in complex settings Be equipped with a set of tools that help identify an appropriate approach or approaches based on the evaluation purpose, the complexity attributes present, and the feasibility of approach for intended use. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-two-s... |
| Description | CECAN Ltd Bespoke coaching series for Defra NI (from Aug 2022), Martha Bicket and Helen Wilkinson |
| Geographic Reach | Local/Municipal/Regional |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | commercial sensitive |
| Description | CECAN Ltd Bespoke training and coaching series for Natural England, March 2023 |
| Geographic Reach | Local/Municipal/Regional |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | commercial sensitive |
| Description | CECAN Ltd CPD Course: Agent-Based Modelling for Policy Analysis, Online, 2 Mar 2021, Corinna Elsenbroich and Jennifer Badham |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Tuesday 2nd March 2021, 09:30 - 17:00, Live Online Training (via Zoom) Tutor: Dr Corinna Elsenbroich, Senior Lecturer (University of Surrey) and CECAN Research Fellow, and Dr Jennifer Badham, Research Associate (Durham University) Course Details: This course is part of the CECAN Module for policy analysts and evaluators. The syllabus was developed to enhance skills nationally and internationally in the evaluation of complex policy and programmes. Agent-based modelling is a computational simulation method for understanding complex systems. An agent-based model is a computer model simulating the interrelationships and interactions of components of a system over time. These models are particularly good at modelling the heterogeneity and interaction of a population, tackling emergence and feedback loops and dealing with non-linear dynamic relationships. The course will cover aspects of model purpose and design, the use of data for model calibration and validation, and consider how to interpret and communicate modelling results in the social sciences and policy research. Intended Audience: Policy analysts, commissioners of evaluation, professional evaluators, social science postgraduate students and researchers. Learning Outcomes: By the end of this session, participants will have gained: An understanding of the basics of agent-based modelling; An understanding of how agent-based modelling can be used to model complex adaptive systems; An understanding of how agent-based modelling can be used in policy analysis and evaluation. Tutor Biographies: Dr Corinna Elsenbroich, is Senior Lecturer in the Department of Sociology (University of Surrey) and part of the CECAN team. Her research interests are in complexity social science, in particular methods to tackle complex causality. She teaches Evaluation Research and courses on Agent-based Modelling, Process Tracing and Qualitative Comparative Analysis. Dr Jennifer Badham originally trained as a mathematician and developed an interest in applying mathematical modelling methods to social policy while working for government and nongovernment health organisations in Australia. Her main research interest concerns the way that social structures affect transmission - of disease, information, beliefs and behaviour. She is currently working at Durham University developing models for COVID-19 policy planning in NE England. |
| URL | https://www.cecan.ac.uk/events/cecan-ltd-cpd-course-agent-based-modelling-for-policy-analysis/ |
| Description | CECAN Ltd CPD Course: An Introduction to Influence Mapping in Evaluation, Online, 10 - 11 Feb 2021, Stuart Astill |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | This course was sold out. // An Introduction to Influence Mapping in Evaluation: Methods, Practice and Use-Cases Wednesday 10th February, 13:00 - 16:00 (Part 1) & Thursday 11th February 2021, 09:30 - 13:00 (Part 2), Live Online Training (via Zoom) Tutor: Stuart Astill Course Details: This 1-day course (split across two consecutive half-days) introduces participants to influence mapping, within the context of evaluation or as a general introduction. Following this course, we are aiming to provide further workshops, which will provide an in-depth look at the different elements of theory and practice of the approach and how they are implemented. The course is built around the OLCA approach (Outcome Likelihood and Causal Analysis) developed by Stuart Astill and Simon Henderson (see their website). The material over the two half-days introduces participants to an innovative set of tools: Structured influence mapping of expert beliefs about the causal relationships that underpin a programme strategy, and Innovative quantitative analysis to compile this evidence to provide insight. This 'mapping plus analytics' offers new possibilities for detailed interrogation to enhance evaluative work and other evidence/learning and decision processes. Intended Audience: Innovative evaluators, evaluation thought leaders, methods experts in evaluation, experienced professional evaluators, those applying flexible and adaptive programming, social science postgraduate students. Learning Outcomes: By the end of this session, participants will: Know what influence mapping and subsequent analysis consists of; Appreciate some use cases and be able to assess their own cases as to whether influence mapping is an appropriate approach; Appreciate the benefits of this approach and how it can enhance their work; Have an understanding of the skills, tools and resources required to use the approach. Tutor Biography: Stuart Astill has over 25 years of experience working at the heart of analytical, strategic, policy and operational thinking in Government. This has taken place in a variety of settings working with public officials, private-sector, third-sector and academic experts, elected representatives and ministers. His qualitative and quantitative skills span performance analysis, statistical, economic and operational research functions across a breadth of experience identifying and tackling emerging high-profile issues, carrying out impact assessments and establishing innovative projects. Stuart's skills have been applied to create insights improving decision making, operational efficiency and VfM. He has worked with, or within, international governments/NGOs, UK government and the European Commission. He has also worked on a wide range of consultancy projects with private sector and third-sector clients in the UK and around the world. As well as his practitioner and academic work Stuart is an experienced speaker, trainer and coach on technical subjects, personal development and management/leadership skills. |
| URL | https://www.cecan.ac.uk/events/cecan-ltd-cpd-course-an-introduction-to-influence-mapping-in-evaluati... |
| Description | CECAN Ltd CPD Course: An Introduction to Influence Mapping in Evaluation, Online, 29 Jun 2021, Stuart Astill |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Intended Audience: Innovative evaluators, evaluation thought leaders, methods experts in evaluation, experienced professional evaluators, those applying flexible and adaptive programming, social science postgraduate students. Learning Outcomes: By the end of this session, participants will: Know what influence mapping and subsequent analysis consists of; Appreciate some use cases and be able to assess their own cases as to whether influence mapping is an appropriate approach; Appreciate the benefits of this approach and how it can enhance their work; Have an understanding of the skills, tools and resources required to use the approach. |
| URL | https://www.cecan.ac.uk/events/cecan-ltd-cpd-course-an-introduction-to-influence-mapping-in-evaluati... |
| Description | CECAN Ltd workshop (CPD): Participatory Systems Mapping, Wednesday 26th June 2019 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | "CECAN Ltd workshop (CPD): Participatory Systems Mapping, Wednesday 26th June 2019, 09:30 - 17:00, Regent's University London NW1 4NS. This course will introduce the innovative participatory systems mapping methodology being developed by researchers in CECAN, which allows us to bring together diverse knowledge and stakeholders in order to construct shared causal maps of an issue or system. The course will introduce the theory behind and analytical approaches to systems maps, as well as providing practical training in map construction with significant time spent in hands on map construction and interrogation. Particular focus will be given to how these methods will be used in participatory approaches and how they contribute to policy appraisal and evaluation. This course is part of the CECAN Module for future policy analysts and evaluators. The syllabus was developed to support capacity building nationally and internationally in the evaluation of complex policy and programmes. Intended Audience: Policy analysts, evaluation professionals, consultants and academic researchers. Learning outcomes: By the end of this session, participants will have gained: • An understanding of how participatory systems mapping contributes to understanding causality and interaction in complex systems; • An understanding of the contexts in which it can be used and how it fits in relation to other social research methods and modelling approaches • An understanding of how to construct, or facilitate the construction of, appropriate systems maps with a variety of stakeholders • Be aware of a range of analysis methods which can be deployed on maps as appropriate • An understanding of how to use this approach for informing policy development and evaluation design. |
| URL | https://mailchi.mp/43d8e7411337/cecan-seminar-invitation-we-ask-both-too-much-and-too-little-of-prog... |
| Description | CECAN Ltd: Complex Evaluation Workshops for Public Health England, Jan 2020 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | On 9th Jan 2020, Pete Barbrook Johnson and Dione Hills ran the first in a set of bespoke workshops for Public Health England as part of the Complex Evaluation Workshops tender that we submitted in November 2019. 20 participants. |
| Description | CECAN team at Centre for Rural Economy (Newcastle) submitted written evidence to the House of Lords Select Committee on rural economies, Sept 2018 |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| URL | https://www.ncl.ac.uk/media/wwwnclacuk/centreforruraleconomy/files/HoL%20Select%20Committee%20Rural%... |
| Description | CECAN's work cited in Government blog "Why we should embrace complexity, not reduce it." Posted by:James Hostford, Posted on:7 May 2020 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| URL | https://systemsthinking.blog.gov.uk/2020/05/07/why-we-should-embrace-complexity-not-reduce-it/ |
| Description | CECAN's work cited in Government website 'News' pages (blog) "Five things you didn't know about evaluation" Written by Steven Finch, Department for Transport, Chair of Cross-Government Evaluation Group. May 2021 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| URL | https://www.gov.uk/government/news/part-three-five-things-you-didnt-know-about#evaluation |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module One - Complexity and its Challenges for Policy and Evaluation Online, 5 Oct 2022 Martha Bicket and Stuart Astill |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Be aware of the main properties of complex systems and be able to recognise these in policy-making processes and their own work, Have a shared language for discussing and exploring complexity, Understand the implications of complex systems for policy-making, analysis and evaluation - the importance of context, the limitations in complex settings of 'traditional' standardising and controlling approaches and the need for flexibility in managing and delivering evaluations. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-one-c... |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module One - Complexity and its Challenges for Policy and Evaluation, Online, 8 Jun 2022 Martha Bicket and Stuart Astill |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Be aware of the main properties of complex systems and be able to recognise these in policy-making processes and their own work, Have a shared language for discussing and exploring complexity, Understand the implications of complex systems for policy-making, analysis and evaluation - the importance of context, the limitations in complex settings of 'traditional' standardising and controlling approaches and the need for flexibility in managing and delivering evaluations. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-one-c... |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module Three - Commissioning and Managing a Complex Evaluation Online, 2 Nov 2022 Georgie Parry-Crooke and Clare Twigger-Ross |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand what it means to manage in complexity and how this applies to commissioning and managing a complex evaluation. Have a shared framework for commissioning and managing complex evaluations. Be able to specify the policy/project/programme complexity to contractors and clarify what is expected from them. Understand how the contracting relationship can accommodate the complex nature of the evaluation (e.g. non- linear, adaptive and dealing with the unexpected). Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-three... |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module Three - Commissioning and Managing a Complex Evaluation Online, 30 Jun 2022 Georgie Parry-Crooke and Clare Twigger-Ross |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand what it means to manage in complexity and how this applies to commissioning and managing a complex evaluation. Have a shared framework for commissioning and managing complex evaluations. Be able to specify the policy/project/programme complexity to contractors and clarify what is expected from them. Understand how the contracting relationship can accommodate the complex nature of the evaluation (e.g. non- linear, adaptive and dealing with the unexpected). Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-three... |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module Two - Selecting an Appropriate Evaluation Approach Online, 19 Oct 2022 Helen Wilkinson and Richard Allen |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand the importance of using a complexity-appropriate approach to evaluation Be aware of the breadth and diversity of approaches that can be used in complex settings Be equipped with a set of tools that help identify an appropriate approach or approaches based on the evaluation purpose, the complexity attributes present, and the feasibility of approach for intended use. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-two-s... |
| Description | Handling Complexity in Policy Evaluation Online Workshop: Module Two - Selecting an Appropriate Evaluation Approach Online, 22 Jun 2022 Helen Wilkinson and Richard Allen |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this workshop, participants will: Understand the importance of using a complexity-appropriate approach to evaluation Be aware of the breadth and diversity of approaches that can be used in complex settings Be equipped with a set of tools that help identify an appropriate approach or approaches based on the evaluation purpose, the complexity attributes present, and the feasibility of approach for intended use. Intended Audience: These workshops are designed to support policy makers, analysts and evaluators at all levels facing complexity challenges in their current work. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-online-workshop-module-two-s... |
| Description | Handling Complexity in Policy Evaluation: Online Training and Support - Complexity Challenge 'Drop In' Surgeries; One to One Coaching; Action Learning Sets. Multiple Dates from Sept 2021, Martha Bicket and other members of the CECAN Team |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | This suite of support activities is designed to support policy makers, analysts and evaluators facing complexity challenges in their work, as well as offering organisations resources to help build evaluation capacity among their staff. |
| URL | https://www.cecan.ac.uk/events/handling-complexity-in-policy-evaluation-training-and-support-2/ |
| Description | Ian Christie was part of a review group on draft chapters for the Dasgupta interim report (April 2020) |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| URL | https://www.gov.uk/government/publications/interim-report-the-dasgupta-review-independent-review-on-... |
| Description | Liz Varga has contributed a small piece on self-organisation to the Dasgupta review on biodiversity |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| URL | https://www.gov.uk/government/collections/the-economics-of-biodiversity-the-dasgupta-review |
| Description | Nigel Gilbert appointed Research Expert and Academic Advisory Panel of National Leadership Centre (NLC), Oct 2019 |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| URL | https://www.nationalleadership.gov.uk/research |
| Description | Reference to CECAN in Defra ITT (M&E for ELM), Mar 2019 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| Description | Reference to CECAN in NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation, Mar 2019 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| Description | Reference to CECAN in the CEDIL Evaluation Agenda, July 2019 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| Description | Surrey Sociology Short Courses: Agent-Based Modelling for the Social Scientist, 10 - 12 February 2020 (Corinna Elsenbroich) |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | "Surrey Sociology Short Courses: Agent-Based Modelling for the Social Scientist, 10 - 12 February 2020 (Corinna Elsenbroich) Simulating social interactions in virtual research labs using agent-based modelling is increasingly allowing researchers to gain new insights into the complex ways that individuals and societies function. In this course, you will be introduced to foundation theoretical and practical aspects, the process of agent-based modelling, from conceptualising a research question, where to obtain data, operationalisation and formalisation of data, model implementation, and model analysis and interpretation. In addition to the theoretical content, you will learn NetLogo as a programming language for agent-based models. " |
| URL | https://www.cecan.ac.uk/events/university-surrey-social-research-methods-short-courses |
| Description | UKES / CECAN Online Masterclass - Bayesian Updating (Diagnostic Theory-Based Evaluation) Online, 10 - 11 Nov 2021 Barbara Befani |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | CECAN Ltd are pleased to be working with the UK Evaluation Society on their Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Learning Outcomes: By the end of this session, participants will: Gain a new perspective on Theory-Based Evaluation and what it takes to improve its credibility and reliability Understand the connection between theory development and empirical observations Learn to use the Bayes formula to update confidence in theories and claims Gain an in-depth understanding of Process Tracing Intended Audience: Policy analysts, evaluation professionals, doctoral students, and academic researchers. |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-bayesian-updating-diagnostic-theory-bas... |
| Description | UKES / CECAN Online Masterclass - Bayesian Updating (Diagnostic Theory-Based Evaluation) Online, 19 Nov 2020, Barbara Befani |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Thursday 19th November 2020, 09:30 - 17:00 GMT, Live Online Training (via Zoom) Tutor: Dr Barbara Befani, CECAN Research Fellow CECAN Ltd are pleased to be working with the UK Evaluation Society on their Autumn 2020 Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Course Details: This course covers the theory and practice of diagnostic evaluation and Bayesian Updating. It addresses its epistemological and mathematical basis, its appropriateness and strengths/weaknesses in comparison with other methods, and its application steps. The method is focused on the empirical testing of propositions, including theories, causal statements and explanations, and its application steps will be addressed in three different modules, each coming with specific group work/exercises: The formulation of theories/propositions and their implications for empirical testing; The design of data collection and the ranking of observations in terms of probative value and updating direction; The estimation of probabilities for use in the Bayes formula and the updating of confidence in the existence of non-existence of the theories/statements. The course includes presentations and case illustrations with prepared examples as well as more interactive discussion of evaluations participants are familiar with. Plan of the Day: 09:30 - 11:00 Recap of Theory-Based Evaluation, process Tracing and other methods based on generative causality. Theoretical Introduction to BU/DE and its application steps. 11:00 - 11:20 Break 11:20 - 12:45 Step One : Developing Testable Theories (with group work) 12:45 - 13:45 Lunch 13:45 - 15:10 Step Two : Designing data collection seeking conclusive tests (with group work) 15:10 - 15:30 Break 15:30 - 17:00 Step Three : Assessing the evidence strength and updating confidence in theory (with group work) Learning Outcomes: By the end of this session, participants will: Gain a new perspective on Theory-Based Evaluation and what it takes to improve its credibility and reliability Understand the connection between theory development and empirical observations Learn to use the Bayes formula to update confidence in theories and claims Gain understanding of Bayesian Process Tracing Tutor Biography: Dr Barbara Befani, CECAN Research Fellow, has been developing evaluation methods for 15 years. Her interests include 1) evaluation quality; 2) methodological appropriateness and comparative advantages and weaknesses of different evaluation methods; 3) causal inference frameworks for impact evaluation; and 4) specific hybrid, quali-quanti methodologies, like Qualitative Comparative Analysis (QCA) and Process Tracing (in particular its Bayesian formalisation, which she is extending to all forms of Theory-Based Evaluation). |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-bayesian-updating-diagnostic-theory-bas... |
| Description | UKES / CECAN Online Masterclass - Discourse Analysis: Exploring the Potential for Use in Evaluation Online, 24 Nov 2021, 09:30 - 17:00 Maria Xenitidou |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | CECAN Ltd are pleased to be working with the UK Evaluation Society on their Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Who will benefit from this Masterclass? Those with some experience of evaluation, those interested in text analysis. Those with a social research background. After this Masterclass you will know : - What discourse analysis is - The essentials of how to apply it - What information it will generate And you will have : - had some initial practice - considered its use in evaluation |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-discourse-analysis-exploring-the-potent... |
| Description | UKES / CECAN Online Masterclass - Process Tracing Introductory Course Online, 13 Oct 2021, 09:30-17:00 Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Learning Outcomes: By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-process-tracing-introductory-course-5/ |
| Description | UKES / CECAN Online Masterclass - Process Tracing Introductory Course Online, 28 Sep 2020, 09:30-17:00 Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Monday 28th September 2020, 09:30 - 17:00 BST, Live Online Training (via Zoom) Tutor: Dr Corinna Elsenbroich, Senior Lecturer (University of Surrey) and CECAN Research Fellow CECAN Ltd are pleased to be working with the UK Evaluation Society on their Autumn 2020 Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Course Details: Process Tracing is a method used to understand causal chains in single case analysis settings. It is the systematic analysis of contribution claims through explicit investigation of how pieces of evidence support or undermine a particular hypothesis or claim. Process Tracing is particularly good for the analysis of single scenarios where no comparison to other cases is possible. It focuses on the causal contribution of particular events to final outcomes by carefully classifying the contribution of identified causes by necessity and sufficiency. Plan of the Day: 09:30 - 11:00 Complex Causality in Evaluation 11:00 - 11:15 Break 11:15 - 13:00 What is Process Tracing? 13:00 - 14:00 Lunch 14:00 - 14:45 How and when to use Process Tracing 14:45 - 15:00 Break 15:00 - 16:30 Process Tracing Case Study 16:30 - 17:00 Reflections on Process Tracing Learning Outcomes: By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study Tutor Biography: Dr Corinna Elsenbroich, Senior Lecturer in the Department of Sociology (University of Surrey) and part of the CECAN team. Her research interests are in complexity social science, in particular methods to tackle complex causality. She has been teaching Evaluation Research and courses on Agent-based Modelling, Process Tracing and Qualitative Comparative Analysis. |
| URL | https://www.cecan.ac.uk/events/cpd-courses/ukes-cecan-online-masterclass-process-tracing-introductor... |
| Description | UKES / CECAN Online Masterclass - Process Tracing Introductory Course, Online, 19 Jan 2021, Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Tuesday 19th January 2021, 09:30 - 17:00 GMT, Live Online Training (via Zoom) Tutor: Dr Corinna Elsenbroich, Senior Lecturer (University of Surrey) and CECAN Research Fellow CECAN Ltd are pleased to be working with the UK Evaluation Society on their Autumn 2020 Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Course Details: Process Tracing is a method used to understand causal chains in single case analysis settings. It is the systematic analysis of contribution claims through explicit investigation of how pieces of evidence support or undermine a particular hypothesis or claim. Process Tracing is particularly good for the analysis of single scenarios where no comparison to other cases is possible. It focuses on the causal contribution of particular events to final outcomes by carefully classifying the contribution of identified causes by necessity and sufficiency. Plan of the Day: 09:30 - 11:00 Complex Causality in Evaluation 11:00 - 11:15 Break 11:15 - 13:00 What is Process Tracing? 13:00 - 14:00 Lunch 14:00 - 14:45 How and when to use Process Tracing 14:45 - 15:00 Break 15:00 - 16:30 Process Tracing Case Study 16:30 - 17:00 Reflections on Process Tracing Learning Outcomes: By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study Tutor Biography: Dr Corinna Elsenbroich, Senior Lecturer in the Department of Sociology (University of Surrey) and part of the CECAN team. Her research interests are in complexity social science, in particular methods to tackle complex causality. She has been teaching Evaluation Research and courses on Agent-based Modelling, Process Tracing and Qualitative Comparative Analysis. |
| URL | https://www.cecan.ac.uk/events/cpd-courses/ukes-cecan-online-masterclass-process-tracing-introductor... |
| Description | UKES / CECAN Online Masterclass - Process Tracing Introductory Course, Online, 29 Apr 2021, Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-process-tracing-introductory-course-3/ |
| Description | UKES / CECAN Online Masterclass - Process Tracing Online, 11 Oct 2022, 09:30 - 17:00 Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-process-tracing-2/ |
| Description | UKES / CECAN Online Masterclass - Process Tracing Online, 29 Mar 2022, 09:30 - 17:00 BST Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | By the end of this session, participants will: Appreciate the problems of complex evaluation Acknowledge the problem of complex causality for contribution claims Understand how and when Process Tracing can be used Work through a Process Tracing Case Study |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-process-tracing/ |
| Description | UKES / CECAN Online Masterclass - Qualitative Comparative Analysis (QCA) Online, 22 Oct 2020, Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Thursday 22nd October 2020, 09:30 - 17:00 BST, Live Online Training (via Zoom) Tutor: Dr Corinna Elsenbroich, Senior Lecturer (University of Surrey) and CECAN Research Fellow CECAN Ltd are pleased to be working with the UK Evaluation Society on their Autumn 2020 Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Course Details: This course is an introduction to Qualitative Comparative Analysis (QCA). QCA is a method for the comparative study of cases and is used to analyse qualitative social data systematically. The analysis compares cases by assigning set memberships across a set of attributes. QCA is particularly good at understanding multiple causation because it is focused on causal recipes (combination of causes) rather than individual variables. This helps with understanding (different) causal pathways in policy evaluation settings. The course will look at the theoretical and practical aspects of QCA, working through a detailed example using the fsQCA software package to understand crisp and fuzzy set QCA. Plan of the Day: 09:30 - 11:00 Configurational Causality and QCA 11:00 - 11:15 Break 11:15 - 13:00 QCA Walkthrough 13:00 - 14:00 Lunch 14:00 - 15.15 QCA Case Study - csQCA 15:15 - 15:30 Break 15:30 - 16:30 QCA Case Study - fsQCA 16:30 - 17:00 Q&A Wrap up Learning Outcomes: By the end of this session, participants will: Understand why and where QCA is a useful method Be able to prepare data for use in a QCA Have basic understanding of the fsQCA software for application for QCA Tutor Biography: Dr Corinna Elsenbroich, Senior Lecturer in the Department of Sociology (University of Surrey) and part of the CECAN team. Her research interests are in complexity social science, in particular methods to tackle complex causality. She has been teaching Evaluation Research and courses on Agent-based Modelling Process Tracing and Qualitative Comparative Analysis. |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-qualitative-comparative-analysis-qca/ |
| Description | UKES / CECAN Online Masterclass - Qualitative Comparative Analysis (QCA) Online, 25 Nov 2021, 09:30-17:00 Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | CECAN Ltd are pleased to be working with the UK Evaluation Society on their Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Learning Outcomes: By the end of this session, participants will: Understand why and where QCA is a useful method Be able to prepare data for use in a QCA Have basic understanding of the fsQCA software for application for QCA |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-qualitative-comparative-analysis-qca-4/ |
| Description | UKES / CECAN Online Masterclass - Qualitative Comparative Analysis (QCA) Online, 5 Jul 2022, 09:30 - 17:00 Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | By the end of this session, participants will: Understand why and where QCA is a useful method Be able to prepare data for use in a QCA Have basic understanding of the fsQCA software for application for QCA |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-qualitative-comparative-analysis-qca-6/ |
| Description | UKES / CECAN Online Masterclass - Qualitative Comparative Analysis (QCA), Online, 18 May 2021, Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | CECAN Ltd are pleased to be working with the UK Evaluation Society on their Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Learning Outcomes: By the end of this session, participants will: Understand why and where QCA is a useful method Be able to prepare data for use in a QCA Have basic understanding of the fsQCA software for application for QCA |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-process-tracing-introductory-course-4/ |
| Description | UKES / CECAN Online Masterclass - Qualitative Comparative Analysis (QCA), Online, 25 Mar 2021, Corinna Elsenbroich |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Thursday 25th March 2021, 09:30 - 17:00 GMT, Live Online Training (via Zoom) Tutor: Dr Corinna Elsenbroich, Senior Lecturer (University of Surrey) and CECAN Research Fellow CECAN Ltd are pleased to be working with the UK Evaluation Society on their Autumn 2020 Masterclass series for those working in consultancy, NGOs, Government, academia and other evaluation settings Course Details: This course is an introduction to Qualitative Comparative Analysis (QCA). QCA is a method for the comparative study of cases and is used to analyse qualitative social data systematically. The analysis compares cases by assigning set memberships across a set of attributes. QCA is particularly good at understanding multiple causation because it is focused on causal recipes (combination of causes) rather than individual variables. This helps with understanding (different) causal pathways in policy evaluation settings. The course will look at the theoretical and practical aspects of QCA, working through a detailed example using the fsQCA software package to understand crisp and fuzzy set QCA. Plan of the Day: 09:30 - 11:00 Configurational Causality and QCA 11:00 - 11:15 Break 11:15 - 13:00 QCA Walkthrough 13:00 - 14:00 Lunch 14:00 - 15.15 QCA Case Study - csQCA 15:15 - 15:30 Break 15:30 - 16:30 QCA Case Study - fsQCA 16:30 - 17:00 Q&A Wrap up Learning Outcomes: By the end of this session, participants will: Understand why and where QCA is a useful method Be able to prepare data for use in a QCA Have basic understanding of the fsQCA software for application for QCA Tutor Biography: Dr Corinna Elsenbroich, Senior Lecturer in the Department of Sociology (University of Surrey) and part of the CECAN team. Her research interests are in complexity social science, in particular methods to tackle complex causality. She has been teaching Evaluation Research and courses on Agent-based Modelling Process Tracing and Qualitative Comparative Analysis. |
| URL | https://www.cecan.ac.uk/events/ukes-cecan-online-masterclass-qualitative-comparative-analysis-qca-3/ |
| Description | AI FORA (University of Mainz), Volkswagen Foundation, led by Petra Ahweiler |
| Amount | € 350,000 (EUR) |
| Organisation | Volkswagen Foundation |
| Sector | Charity/Non Profit |
| Country | Germany |
| Start | 04/2021 |
| End | 04/2024 |
| Description | Advancing Capacity for Climate and Environment Social Science (ACCESS) |
| Amount | £5,118,884 (GBP) |
| Funding ID | ES/W00805X/1 |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 02/2022 |
| End | 01/2027 |
| Description | Cabinet Office Invitation to Tender: CCZZ21A26 - Provision of National Disability Strategy Evaluation Framework for the Cabinet Office, via CECAN Ltd |
| Amount | £84,900 (GBP) |
| Organisation | Cabinet Office |
| Sector | Public |
| Country | United Kingdom |
| Start | 10/2021 |
| End | 07/2022 |
| Description | Defra Invitation to Tender: Invitation to Tender for Complexity Evaluation Framework, Tender Reference: ITT_4883 (bid via CECAN Ltd) |
| Amount | £58,304 (GBP) |
| Organisation | Department For Environment, Food And Rural Affairs (DEFRA) |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2019 |
| End | 05/2019 |
| Description | Digital Catapult Evaluation Framework - Invitation to Tender via CECAN Ltd |
| Amount | £24,500 (GBP) |
| Organisation | Digital Catapult |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 08/2021 |
| End | 10/2021 |
| Description | EMPOWER: Empowering citizen and community adaptation to systemic risks from climate change (NERC COP26 Adaptation and Resilience) |
| Amount | £177,581 (GBP) |
| Organisation | Natural Environment Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 10/2021 |
| End | 03/2022 |
| Description | ESRC Impact Acceleration Account: Embedding CECAN guidance into government practice through development and delivery of support and capacity building activities (Gilbert) |
| Amount | £16,150 (GBP) |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 12/2020 |
| End | 12/2021 |
| Description | ESRC Impact Acceleration Account: Implementing innovative legal and organisational structures for a just transition: a catchment management beacon project (Barbrook Johnson) |
| Amount | £20,000 (GBP) |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 12/2019 |
| Description | Facebook Inc feasibility study using CECAN's approach to Participatory Systems Mapping as a co-design tool in connectivity and sustainability projects (Alex Penn) |
| Amount | £33,000 (GBP) |
| Organisation | |
| Sector | Private |
| Country | United States |
| Start | 11/2019 |
| End | 06/2020 |
| Description | GCRF pump priming project: Peri-Urban Resilient Ecosystems (PURE), (Adam Hejnowicz and Richard Gunton) (£44k) |
| Amount | £44,000 (GBP) |
| Organisation | Stockholm Environment Institute (SEI) |
| Sector | Academic/University |
| Country | Sweden |
| Start | 01/2020 |
| End | 07/2020 |
| Description | InSPIRE: Innovating UK clean air policies to prevent cognitive disorders across the lifespan, particularly for vulnerable urban populations |
| Amount | £41,099 (GBP) |
| Funding ID | MR/T045221/1 |
| Organisation | Medical Research Council (MRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 05/2020 |
| End | 12/2020 |
| Description | Local Institutions, Productivity, Sustainability and Inclusivity Trade-offs (LIPSIT) |
| Amount | £715,318 (GBP) |
| Funding ID | ES/T002468/1 |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 10/2019 |
| End | 03/2021 |
| Description | Natural England Request for Quotation (RfQ): Systems Thinking Options Appraisal and Evidence Review, via CECAN Ltd |
| Amount | £40,000 (GBP) |
| Organisation | Natural England |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 11/2021 |
| End | 04/2022 |
| Description | Optimising air quality and health benefits associated with a low-emission transport and mobility revolution in the UK |
| Amount | £508,012 (GBP) |
| Funding ID | NE/V002449/1 |
| Organisation | Natural Environment Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2020 |
| End | 08/2023 |
| Description | PHE Invitation to Tender: Building capacity to evaluate complex system interventions to tackle health inequalities (bid via CECAN Ltd) |
| Amount | £10,000 (GBP) |
| Organisation | Public Health England |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2020 |
| Description | PHE Invitation to Tender: Heat Health and Cold Weather Alerting systems - end user engagement, user journey mapping and actions taken on the ground (ITT_4616), via CECAN Ltd |
| Amount | £17,990 (GBP) |
| Organisation | Public Health England |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2021 |
| End | 05/2021 |
| Description | Surrey County Council Invitation to Tender: Forces Connect South East - Veterans' Hubs Project External Evaluation (bid via CECAN Ltd) |
| Amount | £8,000 (GBP) |
| Organisation | Surrey County Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2020 |
| End | 05/2020 |
| Description | Systemic environmental risk analysis for threats to UK recovery from COVID-19 |
| Amount | £402,043 (GBP) |
| Funding ID | NE/V018159/1 |
| Organisation | Natural Environment Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2021 |
| End | 01/2022 |
| Description | University of Surrey Faculty of Arts and Social Sciences (FASS) pump-priming fund (bid from Pete Barbrook-Johnson) |
| Amount | £2,000 (GBP) |
| Organisation | University of Surrey |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 07/2019 |
| Description | University of Surrey's QR Strategic Priority Funds (SPF) from Research England |
| Amount | £16,100 (GBP) |
| Organisation | University of Surrey |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 01/2020 |
| End | 03/2020 |
| Description | CECAN Ltd and UKES Masterclasses trial |
| Organisation | UK Evaluation Society |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | CECAN Ltd is working closely with the UK Evaluation Society (UKES) to jointly provide three 'Method Masterclasses' in 2020, covering Qualitative Comparative Analysis (QCA), Process Tracing and Bayesian Updating. The first Masterclass, to be held in London, will be piloted in Q1 2020, with the two further classes then planned for Q2. |
| Collaborator Contribution | As above |
| Impact | TBC |
| Start Year | 2020 |
| Description | CECAN Ltd: DFID Framework: International Multi-Disciplinary Programme Lot 20, Research (High Value) - Consortium led by LTS - June 2019 |
| Organisation | LTS International Ltd |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | CECAN Ltd is a member of a consortium led by LTS Consultancy, which submitted a successful tender to secure a place on the DFID Framework: International Multi-Disciplinary Programme Lot 20, Research (High Value). At a launch event for the successful Framework suppliers held on 7th June 2019, DFID confirmed that they foresee 80% of routine procurements to be conducted through the IMDP Framework with an estimated value of £1 - 3 million over a 4 year period (2 + 1 + 1 years). The frameworks are also open for other UK government departments to use for their procurements. |
| Collaborator Contribution | As above |
| Impact | TBC |
| Start Year | 2019 |
| Description | CECAN Ltd: Water Systems Mapping project at Defra led by Mott Macdonald, Oct 2019 |
| Organisation | Mott Macdonald UK Ltd |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | CECAN Ltd submitted a successful bid with Mott Macdonald and Risk Solutions to Defra to work on a Water Systems Mapping project (Invitation to Tender for: Systems Analysis of Water Resources WT15121 (Project 27436)). The project will run from approx. Nov 2019 - June 2020. Income to CECAN Ltd (as sub-contractor, hence not recorded under Further Funding) of approx. £37.5k. Alex Penn and Pete Barbrook-Johnson are leading on this work. Alex will utilise her knowledge and experience of systems analysis methods and their application in the design of the workshops and delivery of the maps and training. Alex also has significant experience in stakeholder engagement and workshop facilitation. Peter is currently working with Anglian Water and the South Lincolnshire Water Partnership on governance issues. His experience and knowledge from this and previous systems mapping projects for Defra and other regulators will be invaluable to the delivery of this project. |
| Collaborator Contribution | Mott MacDonald will provide administration support staff to assist with workshop logistics. This will allow the technical project team to concentrate on workshop design and engaging with stakeholders. MM staff will also act as Project Manager and Institutional Advisor, bringing existing connections with key stakeholders. |
| Impact | Consultancy Report to Defra Anticipate academic publications co-authored with Mott and Defra |
| Start Year | 2019 |
| Description | CECAN consultancy partners (via CECAN Ltd) |
| Organisation | Collingwood Environmental Planning |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Through CECAN Ltd we have contributed expertise in complexity appropriate evaluation methods, especially participatory systems mapping. Projects have included evaluating Defra's Environmental Land Management (ELM) test and trials; evaluating the UKRI's Strategic Priorities Fund; creating a Theory of Change tool for Defra; providing systems mapping expertise for evaluations of water systems (at Defra and with Water Resources South East); developing a Health Wellbeing Nature and Sustainability tool for Natural England. CECAN Ltd has also been able to join the Research Development Framework at Defra. |
| Collaborator Contribution | This partnerships have provided CECAN an opportunity to strengthen relationships with key policy stakeholders and evaluation practitioners and given us access to test our methods in real world settings. |
| Impact | Where appropriate, CECAN derives academic publications from its involvement in these consultancy projects (often co-authored with consultancy professionals and/or policy clients). Income generated through CECAN Ltd consultancy is used to support the wider activities of CECAN the research group, in particular it is used to fund extensions to staff contracts. |
| Start Year | 2020 |
| Description | CECAN consultancy partners (via CECAN Ltd) |
| Organisation | ICF International, Inc. |
| Department | ICF International, Europe |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Through CECAN Ltd we have contributed expertise in complexity appropriate evaluation methods, especially participatory systems mapping. Projects have included evaluating Defra's Environmental Land Management (ELM) test and trials; evaluating the UKRI's Strategic Priorities Fund; creating a Theory of Change tool for Defra; providing systems mapping expertise for evaluations of water systems (at Defra and with Water Resources South East); developing a Health Wellbeing Nature and Sustainability tool for Natural England. CECAN Ltd has also been able to join the Research Development Framework at Defra. |
| Collaborator Contribution | This partnerships have provided CECAN an opportunity to strengthen relationships with key policy stakeholders and evaluation practitioners and given us access to test our methods in real world settings. |
| Impact | Where appropriate, CECAN derives academic publications from its involvement in these consultancy projects (often co-authored with consultancy professionals and/or policy clients). Income generated through CECAN Ltd consultancy is used to support the wider activities of CECAN the research group, in particular it is used to fund extensions to staff contracts. |
| Start Year | 2020 |
| Description | CECAN consultancy partners (via CECAN Ltd) |
| Organisation | Mott Macdonald UK Ltd |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Through CECAN Ltd we have contributed expertise in complexity appropriate evaluation methods, especially participatory systems mapping. Projects have included evaluating Defra's Environmental Land Management (ELM) test and trials; evaluating the UKRI's Strategic Priorities Fund; creating a Theory of Change tool for Defra; providing systems mapping expertise for evaluations of water systems (at Defra and with Water Resources South East); developing a Health Wellbeing Nature and Sustainability tool for Natural England. CECAN Ltd has also been able to join the Research Development Framework at Defra. |
| Collaborator Contribution | This partnerships have provided CECAN an opportunity to strengthen relationships with key policy stakeholders and evaluation practitioners and given us access to test our methods in real world settings. |
| Impact | Where appropriate, CECAN derives academic publications from its involvement in these consultancy projects (often co-authored with consultancy professionals and/or policy clients). Income generated through CECAN Ltd consultancy is used to support the wider activities of CECAN the research group, in particular it is used to fund extensions to staff contracts. |
| Start Year | 2020 |
| Description | CECAN consultancy partners (via CECAN Ltd) |
| Organisation | Technopolis Group |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Through CECAN Ltd we have contributed expertise in complexity appropriate evaluation methods, especially participatory systems mapping. Projects have included evaluating Defra's Environmental Land Management (ELM) test and trials; evaluating the UKRI's Strategic Priorities Fund; creating a Theory of Change tool for Defra; providing systems mapping expertise for evaluations of water systems (at Defra and with Water Resources South East); developing a Health Wellbeing Nature and Sustainability tool for Natural England. CECAN Ltd has also been able to join the Research Development Framework at Defra. |
| Collaborator Contribution | This partnerships have provided CECAN an opportunity to strengthen relationships with key policy stakeholders and evaluation practitioners and given us access to test our methods in real world settings. |
| Impact | Where appropriate, CECAN derives academic publications from its involvement in these consultancy projects (often co-authored with consultancy professionals and/or policy clients). Income generated through CECAN Ltd consultancy is used to support the wider activities of CECAN the research group, in particular it is used to fund extensions to staff contracts. |
| Start Year | 2020 |
| Description | COVID-19 Community Health and Social Care Modelling Team (Durham University) |
| Organisation | Durham University |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Between March and July 2020 Brian Castellani and Pete Barbrook-Johnson became part of the COVID-19 Community Health and Social Care Modelling Team at Durham University. The team was led by Dr Camila Caiado and Professor Brian Castellani, with the purpose of creating a series of modelling tools that Trusts and Councils in the North East of England could use to help support decisions and planning accordingly. Castellani and Barbrook-Johnson were particularly involved in developing two models. The first model used COMPLEX-IT, a case-based, mixed-methods platform that Castellani, Barbrook-Jonnson and Corey Schimpf developed for applied social inquiry into complex data/systems. Using this first model, they provided weekly reports on COVID-19 for the local authorities in the North East of England, including short-term predictions on the case-trends for each of these 27 authorities based on similar trends in Italy's provinces, given that the spread of COVID-19 was roughly two weeks ahead of the UK. For more on this first model and how COMPLEX-IT works, see the recent CECAN webinar by Castellani, Barbrook-Johnson and Schimpf. The second model was created by Jennifer Badham, working with Barbrook-Johnson, Caiado and Castellani, called the COVID-19 Social interventions agent-based model. The novelty of this model is that it was created to help local stakeholders in the North East of England (councils, trusts, etc) simulate and explore the impact various governmental social planning scenarios will have on local COVID-19 transmission and disease progression (exit-strategies, social distancing measures, etc). The model is freely available on Git-Hub to anyone interested in using it. Presently the team is writing articles on both models, which will be published in 2021. The team was under the guidance of the University's Executive Deans, Jacqui Ramagge (Science) and Charlotte Clarke (Social Sciences and Health) and was supported, in part, through the Wolfson Research Institute for Health and Wellbeing, the Institute of Data Sciences to the University's Health@Durham strategy, and CECAN (Centre for the study of Complexity Across the Nexus). |
| Collaborator Contribution | This provided an opportunity for CECAN to widen its remit and also test our COMPLEX-IT app in the real world. |
| Impact | This work resulted in a number of related publications, which have been listed in the relevant sections of our 2020-21 submission. |
| Start Year | 2020 |
| Title | COMPLEX-IT - Exploring complex data from a case based perspective (Castellani, Barbrook-Johnson) |
| Description | In November 2019, we launched our COMPLEX-IT software. Developed by CECAN researchers at Durham and Surrey, COMPLEX-IT is designed to increase access to the tools of computational social science (e.g. artificial intelligence, micro-simulation, predictive analytics) through its visually intuitive learning environment. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2019 |
| Open Source License? | Yes |
| Impact | COMPLEX-IT has been successfully used in real-world situations, for example the Covid-19 Community Health and Social Care Modelling Team at Durham University were able to use the COMPLEX-IT software to create a model that provided weekly reports on Covid-19 for the local authorities in the North East of England to support decisions and planning accordingly (Between March and July 2020) |
| URL | https://www.art-sciencefactory.com/complexit.html |
| Title | Choosing Appropriate Evaluation Methods tool v2 (Befani) |
| Description | Dr Barbara Befani, CECAN Research Fellow, has expanded and partially redeveloped the Choosing Appropriate Evaluation Methods tool, originally published by BOND, to include, amongst else, more complexity-appropriate methodologies. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/ |
| Type Of Technology | Webtool/Application |
| Year Produced | 2020 |
| Open Source License? | Yes |
| Impact | This software has been used and proved popular within CECAN's suite of case studies, as described on our website. |
| URL | https://www.cecan.ac.uk/news/choosing-appropriate-evaluation-methods-a-tool-for-assessment-and-selec... |
| Title | Participatory System Mapper (Gilbert) |
| Description | The Participatory System Mapper (PRSM) is an app that makes it easy for a group of people working together to draw networks (or 'maps') of systems. It runs in a web browser on a desktop PC or a tablet. The network or map can be anything that has items (or 'factors' or 'nodes') connected by links (or 'edges'). Here are some examples: People (the nodes) connected by knowing each other Factors or variables causing (the links) changes in other factors Switches connected by wires Computers connected by network links Theories expressed as variables and relationships between them Company boards of directors (the nodes) and the directors that sit on more than one board (the links) Twitter hashtags (the nodes) included together on posts (the links) Scientists (the nodes) co-authoring papers (the links) and so on. PRSM enables groups of people, each using their own computer (or tablet) to collaborate in the drawing of a map. They may be sitting around a table, discussing the map as it is created face to face, or working remotely, using video conferencing or the chat feature that is built into the app. Everyone can participate because every edit (creating nodes and links, arranging them and so on) is broadcast to all the other participants as the changes are made (just as Google Docs does for text, for example). When you start PRSM in your browser, a 'room' is created for you in which to draw your network. You can add other users to this room to share the work. Only those with access to the room can see what is being created. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2020 |
| Open Source License? | Yes |
| Impact | The app has been used in CECAN's work with the Health and Safety Executive and in the NERC project Systemic environmental risk analysis for threats to UK recovery from COVID-19 that Prof Gilbert is associated with. |
| URL | https://cress.soc.surrey.ac.uk/prsm/ |
| Title | Participatory System Mapper (PRSM) v2.0 |
| Description | Feb 2023: The Participatory System Mapper (PRSM) is a free, open-source and secure tool for mind-mapping and system visualisation. PRSM continues to develop, aided by feedback from its growing number of users. Enhancements include: • The User Manual has been completely re-implemented and is now much easier to use. • The drawing layer behind the map now uses 'objects' which can be moved, resized, modified and deleted. • Factors can be voted on. • Data from Excel spreadsheets can be imported and the data in maps can be exported to Excel. • Notes attached to factors and links can now be edited in a separate window, which offers a wider range of editing commands. • It is possible to run PRSM within an intranet, with no access needed to the wider internet when you require the highest level of security. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2023 |
| Open Source License? | Yes |
| Impact | This software is consistently used in a wide range of CECAN Ltd consultancy projects with clients including Defra, BEIS and Natural England |
| URL | https://prsm.uk/ |
| Description | "Systems in Evaluation" presented to the American Evaluation Association, Systems in Evaluation Topical Interest Group by Alex Penn, Pete Barbrook-Johnson and Helen Wilkinson, May 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Alex Penn, Pete Barbrook-Johnson and Helen Wilkinson gave a presentation "Systems in Evaluation" to the American Evaluation Association, Systems in Evaluation Topical Interest Group in May 2021 |
| Year(s) Of Engagement Activity | 2021 |
| Description | "What can agent-based modelling do for you?" seminar presented by Nigel Gilbert to Defra Social Research Group, 26 April 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel Gilbert gave a seminar for Defra Social Research Group methodology series. Title: What can agent-based modelling do for you? Audience: 20 members of the SRG (all Defra policy analysts) Date: 26 April 2021 |
| Year(s) Of Engagement Activity | 2021 |
| Description | Adam Hejnowicz is showcasing the work of CECAN at the YESI Citizen Food Assembly event, 8th Nov 2019 |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | Adam Hejnowicz is showcasing the work of CECAN at the YESI Citizen Food Assembly event (part of ESRC Festival of Science) |
| Year(s) Of Engagement Activity | 2019 |
| Description | Adam Hejnowicz presented CECAN's case study work at the YESI Summer Social Programme on 5th June 2020 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Other audiences |
| Results and Impact | Adam gave an invited presentation describing CECAN's case study work at the York Environmental Sustainability Institute (YESI)'s Summer Social Programme: "Opportunities for transformation: Bridging divides at the sciencepolicy interface". |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.york.ac.uk/media/yesi/yesinew2018/YESI%20Summer%20Social%20Programme.pdf |
| Description | Ben Shaw was invited to present about CECAN to the Charity Evaluation Working Group (CHEW) on 5 Oct 2022 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Ben Shaw was invited to present about CECAN to the Charity Evaluation Working Group (CHEW) on 5 Oct 2022 |
| Year(s) Of Engagement Activity | 2022 |
| Description | Ben Shaw was invited to present to Transforming Evidence online panel, 'How do we map the evidence system?' 24 Jan 2023 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Ben Shaw was invited to present to Transforming Evidence online panel, 'How do we map the evidence system?' 24 Jan 2023. //// I'm writing to invite you to be a speaker on a 90 minute panel for Transforming Evidence on Zoom 12.30 to 14.00 on Tuesday 24 January 2023 entitled 'How do we map the evidence system?' We would be particularly interested to hear your thoughts from work on CECAN, including your participatory mapping. It would be great if you could speak for 10 minutes - and be open to a range of questions from others. I would be very happy to speak in advance to discuss what you might cover. Timetable: 12:30 Intro, Chair: Jonathan Breckon (Senior Associate, Transforming Evidence) 12:35 Kathryn Oliver (Co-director, Transforming Evidence) 12.45 Harsha Dayal (Director, Research & Knowledge Management, Gov of South Africa) 12:55 Kristian Krieger (Policy Officer, European Commission - Joint Research Centre) 13:05 Ben Shaw (Deputy Director, UK Centre for the Evaluation of Complexity Across the Nexus) 13:15 Q&A + MIRO comments on maps (35mins) 13:50 Summing up + final words 14:00 Ends Background and context: To help navigate research engagement with policy and practice, many organisations have mapped, visualised, or conceptualised the landscape. For example, the EU's Joint Research Centre conceptual mapping of the science-policy system, or an examination of the Science Advice System for the UK Government's Science and Innovation Network, or a description of the South African Evidence Ecosystem. Much of this mapping working is based on a view that we can see an evidence and policy system - or a metaphorical 'ecosystem'. But if there is a system - how best can it really be mapped? Although there are many schools of thought on systems or systems change, one original conception was of dynamic, complex, and non-linear process. But it seems this conception has often been lost. More often, we see static, simplistic, and linear visualisations of the evidence system. Alternatively, another problem is not simplicity, but complexity. Some science and innovation maps can be hard to navigate, overburdened with too much information, the very opposite of good cartography. This event will see how we can move beyond simplistic or confusing maps by discussing and showcasing some useful approaches to mapping evidence systems. Potential questions for discussion: - What are the goals of mapping evidence and policy systems? - What are the best approaches to mapping science/policy/practice systems? - Do we have enough data to map interactions? - Are visual maps helpful - or not? Or is it better to describe in words or other means? - If we use a visualisation of this mapping - what is the best way to go about it? - How do we manage the complexity and fluidity of most systems without oversimplifying the maps? - What can we learn from other fields such as geography and data science - and across countries? - What should the future of mapping evidence and policy systems look like? We do hope you might be able to join us for this session. We would be very happy to discuss the content and approach in early January - or some other time that works for you. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Book launch: Systems Mapping: how to build and use causal models of systems, A Penn and P Barbrook-Johnson (Tuesday 27th September 2022, 13:00 - 14:00 BST) ONLINE |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Please join us for this virtual book launch for the new open access book (i.e. free PDF available) on systems mapping by CECAN members Dr Pete Barbrook-Johnson and Dr Alexandra Penn. Pete and Alex will introduce the book before we hear from a range of systems mapping experts as well as researchers and practitioners using systems mapping in their work. We will then close with an open Q&A session. Speakers: Speakers will include: Dan McGonigle (Head of Systems, Innovation and Futures, CSA's Office, UK Department for Environment, Food and Rural Affairs), Prof Birgit Kopainsky, (Professor in System Dynamics, University of Bergen), and Prof Petra Meier, MRC/CSO Social and Public Health Sciences Unit, University of Glasgow. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/events/book-launch-systems-mapping-how-to-build-and-use-causal-models-of-sys... |
| Description | Brian Castellani presented CECAN Evaluation methods to a Health Cascade workshop, Nov 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | Brian Castellani presented CECAN Evaluation methods to a Health Cascade workshop, Nov 2022, attended by young Post Docs, who were very enthusiastic about CECAN's work and some have since contacted Brian for further info. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://healthcascade.eu/ |
| Description | CECAN Blogs (active and ongoing - regularly updated) |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | 15 CECAN Blog posts (active and ongoing - regularly updated): Members of the CECAN Team, CECAN Fellows, CECAN Partners and others regularly contribute to the CECAN Blog, which can be found on the CECAN website. Posts to date include: Health CASCADE Workshop: Co-producing complex systems interventions for public health (SEP 30, 2022), It's time for the UK government to embrace systems mapping (SEP 9, 2022), Pluralistic evaluation framework published in Ecological Economics (MAY 3, 2022), What would a complexity-appropriate evaluation framework for England's action on nature over the next 10 years look like? (NOV 16, 2021), Evaluation as a Pathway to Transformation for a Sustainable Future (JUL 27, 2021), Part 5 - Why we Need Locally/Nationally Interdependent Models to Successfully Exit COVID-19 Lockdown (Jul 9, 2020), Part 4 - Social Networks and the Coronavirus: The Importance of Complexity Science for Public Health (Apr 1, 2020), Part 3 - Simulation and the Coronavirus: The Importance of Complexity Science for Public Health (Mar 30, 2020), Part 2 - Modelling COVID-19: So, What Does an Effective Approach to Simulation Look Like? ( Mar 30, 2020) Part 1 - Modelling the Coronavirus: Why All Public Health Models are Not the Same (Mar 30, 2020), CECAN contributes to Norway Anniversary Evaluation Conference (08/10/2019 - 09:38), Complexity and Evaluation Failure (07/08/2019 - 10:04), Value, Values and Valuing Nature (25/06/2019 - 20:58), Slipping into simplicity (24/06/2019 - 09:47), How to evaluate complex research impact (26/04/2019 - 08:31). The following blog posts are still present on the CECAN website but were created before 1/3/19 so have been logged under CECAN1 award. FOR REFERENCE ONLY: Complexity and Evaluative Thinking (21/01/2019 - 09:44) What has economic pluralism got to do with understanding complexity? (17/12/2018 - 09:40) The Story of the Pluralistic Evaluation Framework (19/11/2018 - 10:55) Power and Hope: Using Evaluation in Volatile Contexts (24/10/2018 - 14:27) The Nexus: A New Approach to Sustainability Transformations - What, Why and How (12/09/2018 - 11:38) CECAN Fellowship Blog: Evaluative methodologies for the transition towards an institutional ecosystem approach 'system-cultures' within Natural Resources Wales and CECAN (27/07/2018 - 13:22) A Pluralistic Evaluation Framework (11/06/2018 - 21:06) The Defiance of Global Commitment (15/05/2018 - 21:36) Looking Through a Glass Darkly: Public Goods and Agricultural Policy(09/03/2018 - 14:38) Humility and courage: strategy and evidence in our complex world (27/02/2018 - 20:25) Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes (22/02/2018 - 22:20) Finding The Common Ground (21/02/2018 - 17:46) Carillion may have collapsed, but public-private partnerships can be so much more... (08/02/2018 - 18:07) Teaching Evaluation of Complex Policy and Programmes (12/01/2018 - 16:29) Putting Values Back in Evaluation. (03/01/2018 - 15:30) Building Collaborative Narratives and Developing Trust in Policy and its Evaluation. (13/12/2017 - 20:13) An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!(15/11/2017 - 12:26) Travels in Evaluation - Systematising the Complex, Enjoying the Journey.(15/11/2017 - 12:20) CECAN's 'The Visual Representation of Complex Systems' at Environment, Economy, Democracy: Flourishing Together #RSD6.(13/11/2017 - 15:51) A Whole New World - What Next? (12/09/2017 - 16:38) Complexity Settings to the Rescue: A New Lease of Life for Evidence-Based Policy?(28/08/2017 - 21:14) Models as 'Interested Amateurs' (09/08/2017 - 10:33) Taking The Temperature of Trust - Henry Leveson Gower (03/07/2017 - 20:44) One Researcher's Anecdote is Another Researcher's Data (09/03/2017 - 11:44) That Way Lies Prosperity: Sustainability and the Nexus(07/02/2017 - 17:24) Agriculture Fit For A Complex World (16/11/2016 - 11:28) Why We Need Network Analysis to Understand the Future of Economics(08/11/2016 - 13:59) CECAN Complexity in Evaluation Workshop: What we Did and What we Learned (01/11/2016 - 13:35) Why Carry Out Economic Evaluation? (01/11/2016 - 09:49) Access to Data is Crucial(25/10/2016 - 15:42) Aligning Policy and Evidence for the Age of Complexity(24/10/2016 - 15:43) Complexity High on the Agenda at the EES 2016 Biannual Conference(11/10/2016 - 14:20) Likelihoods (06/10/2016 - 14:02) Is Policy Evaluation Fit For Purpose? (26/09/2016 - 13:51) After the Brexit vote: What Next for the UK's Environment? (29/06/2016 - 13:51) How Should Academics Interact with Policy Makers? Lessons on Building a Long-term Advocacy Strategy.(22/06/2016 - 12:09) Reflections on Language and Complexity (22/06/2016 - 11:59) Should Academics be Expected to Change Policy? Six Reasons Why it is Unrealistic for Research to Drive Policy Change.(26/05/2016 - 10:03) The Science of Using Research... (24/05/2016 - 21:52) Clearing The Fog(04/05/2016 - 14:44) Global Challenges Require Cross-Cutting Solutions(04/04/2016 - 16:35) Tools, Tools Everywhere and not a Hammer in Sight!(24/03/2016 - 18:48) |
| Year(s) Of Engagement Activity | 2019,2020,2021,2022,2023 |
| URL | https://www.cecan.ac.uk/blog |
| Description | CECAN Intra Project Meeting, 20 - 21 November 2019, Guildford |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Intra Project Meeting, 20 - 21 November 2019, Guildford Workshop with invited speakers: Jess Phoenix, Defra; Corey Schimpf (Complex-IT team); Kathryn Oliver, London School of Hygiene and Tropical Medicine. Circa 25 participants including CECAN team members" |
| Year(s) Of Engagement Activity | 2019 |
| Description | CECAN Ltd took part in a University of Surrey hosted "Pitch@Palace" event, 23 Sept 19 |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Industry/Business |
| Results and Impact | CECAN Ltd took part in a University of Surrey hosted "Pitch@Palace" event in an effort to publicise university spin-outs and engage with new organisations. HRH the Duke of York - founder of Pitch@Palace - attended the event. |
| Year(s) Of Engagement Activity | 2019 |
| Description | CECAN Ltd: launch event for Defra CEF, June 2019 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CEF launch event, 5th June 2019, London CECAN Ltd presented a first look at the new Complexity Evaluation Framework to an audience of policy-makers, analysts and commissioners in the Department for Environment, Food and Rural Affairs (Defra). The workshop included a discussion with Roisin O'Brien (Principal Social Researcher - Resources and Waste Evidence) and Tom Pye (Senior Policy Advisor - Resources, Waste and Plastics Strategy), hosted by CECAN Deputy Director Ben Shaw." |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.co.uk/projects/ |
| Description | CECAN Newsletter - Dec 2019 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter Dec 2019 https://mailchi.mp/d58409fb1855/46i5hbo49y-4710021 |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://mailchi.mp/d58409fb1855/46i5hbo49y-4710021 |
| Description | CECAN Newsletter - June 2019 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter June 2019 https://mailchi.mp/f518f8346b87/46i5hbo49y-2048137 |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://mailchi.mp/f518f8346b87/46i5hbo49y-2048137 |
| Description | CECAN Newsletter - March 2019 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter, March 2019 https://mailchi.mp/bd4dde53ae9a/46i5hbo49y-1961397 |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://mailchi.mp/bd4dde53ae9a/46i5hbo49y-1961397 |
| Description | CECAN Newsletter - Sept 2019 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter Sept 2019 https://mailchi.mp/e542b558be3b/46i5hbo49y-2130917 |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://mailchi.mp/e542b558be3b/46i5hbo49y-2130917 |
| Description | CECAN Newsletter April 2020 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 2-3 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://mailchi.mp/3c822aaeeb0c/46i5hbo49y-4881686?e=4698f05d27 |
| Description | CECAN Newsletter December 2020 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 2-3 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://mailchi.mp/ff7dc9ae2d7f/46i5hbo49y-5027490 |
| Description | CECAN Newsletter February 2021 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 2-3 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://mailchi.mp/507318e1afc1/46i5hbo49y-5052613?e=4698f05d27 |
| Description | CECAN Newsletter February 2022 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 3-4 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://mailchi.mp/3b4d02a1500c/46i5hbo49y-5999182 |
| Description | CECAN Newsletter July 2020 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 2-3 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://mailchi.mp/7d924ce0da9b/46i5hbo49y-4948877 |
| Description | CECAN Newsletter June 2021 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 3-4 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://mailchi.mp/252e4c1834ce/46i5hbo49y-5087318 |
| Description | CECAN Newsletter October 2021 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN's newsletter is sent approximately every 3-4 months to the CECAN mailing list, which includes over 1000 members (e.g. policymakers, policy analysts, evaluation professionals) |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://mailchi.mp/cf342bb7985b/46i5hbo49y-5159194 |
| Description | CECAN Newsletter, January 2023 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter, Jan 2023. Sent to mailing list via email and also posted to website. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.cecan.ac.uk/cecan-newsletters/ |
| Description | CECAN Newsletter, May 2022 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter, May 2022 |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/cecan-newsletters/ |
| Description | CECAN Newsletter, September 2022 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Newsletter, Sept 2022. Circulated via email to mailing list and posted on website. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/cecan-newsletters/ |
| Description | CECAN Press Release (1 April 2020): CECAN Guidance on Complexity and Policy Evaluation Published by HM Treasury |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN has produced supplementary guidance for the 2020 revision of the Magenta Book, The Magenta Book, published by HM Treasury, is the key UK Government resource on evaluation, setting out central government guidance on how to evaluate policies, projects and programmes. Following publication of the guide in April 2020, CECAN promoted this through a range of activities (in person events (pre-lockdown), online webinars) including this press release. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/news/cecan-press-release-1-april-2020-cecan-guidance-on-complexity-and-polic... |
| Description | CECAN Press Release (15 September 2021): New Toolkit to Help Evaluate Complex Policies |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN issued a press release to promote its Complexity Evaluation Toolkit. Contents as follows: New toolkit to help evaluate complex policies One of the country's top research hubs for evaluating complex policies has published a new toolkit that could help government officials judge the effectiveness of their work. The Centre for the Evaluation of Complexity Across the Nexus (CECAN), based at the University of Surrey, has published The Complexity Evaluation Toolkit. The document covers areas of complex evaluation, including key issues in commissioning, designing and managing an evaluation. The toolkit draws from, summarises and builds upon concepts and guidance in the contributions that CECAN has made to the Magenta Book Supplementary Guidance (HM Treasury 2020) and the Complexity Evaluation Framework (Defra 2019). The toolkit is aimed at those with some basic knowledge or experience of evaluation but who may need guidance on handling complexity. These professionals could include: · Policy analysts in government · Evaluation consultants in the private sector · Evaluators working in NGOs and charities. Professor Nigel Gilbert CBE, Director of CECAN at the University of Surrey, said: "The UK, and indeed the world, is facing some grand challenges in the next decade - climate change being chief amongst them. These complex policy problems must and should be tackled on all fronts - from local to national government and all sectors of our economy. "We believe our new toolkit will give guidance and reassurance to professionals who are crafting and making sense of complex policies, and we urge them to download the document." |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/news/cecan-press-release-15-september-2021-new-toolkit-to-help-evaluate-comp... |
| Description | CECAN Twitter |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Twitter: CECAN has a Twitter account @cecanexus which is regularly used to post about CECAN events, seminars, blogs, papers, policy practice notes, other achievements (eg. syllabus, toolkits). As of 14/3/23, the account has 1,723 Followers (last count: 1,620 on 15/3/22), is following 223 accounts (last count: 219 on 15/3/22) and has made 1,059 Tweets (last count: 973 on 15/3/22). Almost all Tweets indicate some level of engagement with Followers, eg. usually 10 re-tweets / likes / comments, if not more. NOTE that this Twitter account started under CECAN1 in 2016 and has run continuously ever since. As advised by ESRC, all impacts from 1 March 2019 are recorded under CECAN's continuation funding, ES/S007024/1 hence this new entry. |
| Year(s) Of Engagement Activity | 2019,2020,2021,2022,2023 |
| URL | https://twitter.com/cecanexus?lang=en |
| Description | CECAN Webinar - Citizens' juries: how public policy should be made?, Tuesday 9th July 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar - Citizens' juries: how public policy should be made?, Tuesday 9th July 2019, 13:00 - 14:00 BST Presenter: Dr Malcolm Oswald" |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-citizens-juries-how-public-policy-should-be-made |
| Description | CECAN Webinar - COMPLEX-IT: User-friendly computational modelling software for exploring complex policy data, 18 June 2020, Pete Barbrook Johnson |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This webinar covered - an overview of what COMPLEX-IT is, a demonstration of how to use it and some discussion around extending it with ABM. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.youtube.com/watch?v=YuRXdWss9tE |
| Description | CECAN Webinar: A Systems Approach to Environmental Policy in Defra Online, 24 Nov 2020, Tom Oliver |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Tuesday 24th November 2020, 13:00 - 14:00 GMT Presenter: Tom Oliver, University of Reading Webinar Overview: The need to take a whole-systems approach to policy to avoid unanticipated outcomes and burden shifting across sectors is now widely recognised. In this presentation I will introduce the Systems Research Programme established by the UK government's Department for Environment Food and Rural Affairs (Defra) in March 2019, with the aim to better understand multiple outcomes of policy decisions and how suites of policies interact. The programme has six academic fellows reflecting policy areas of air quality, resources and waste, rural land use, marine and food. My role involves the advising on overall design, synthesis and coordination of the programme and working on crosscutting issues. I will discuss in particular our current work on some particularly wicked policy problems: net zero policy design and environmental target setting, and conclude by describing a forthcoming project with CECAN and several universities developing new protocols for the governance of systemic risk, including COVID-19 relevant case studies for biosecurity, respiratory health and food security. Presenter Biography: Tom Oliver is Professor of Applied Ecology at the University of Reading UK and leads their Ecology and Evolution research division. He sits on the European Environment Agency scientific committee and is a senior Fellow with Defra working to coordinate their Systems Research Programme. Tom's primary research focuses on understanding the interacting impacts of drivers upon multiple environmental outcomes. A key aspect of this involves developing methods and tools to better quantify and communicate environmental risk to support environmental decision-making. Tom has published more than eighty scientific papers in world-leading interdisciplinary journals. His writing has appeared in the Guardian, Independent and BBC Science Focus and he is author of the critically acclaimed book The Self Delusion: The Surprising Science of Our Connection to Each Other and the Natural World. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-a-systems-approach-to-environmental-policy-in-defra/ |
| Description | CECAN Webinar: An update of the Medical Research Council (MRC) Framework for Developing and Evaluating Complex Interventions Online, 15 Jun 2022, 1pm Kathryn Skivington & Laurence Moore |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Webinar Overview: Complex interventions are commonly used in the health and social care services, public health practice, and other areas of social and economic policy that have consequences for health. Examples include a new surgical procedure, the redesign of a healthcare programme, and a change in welfare policy. In this webinar Kathryn and Laurence will present their recent paper in BMJ, which updated a 2008 Medical Research Council framework for developing and evaluating complex interventions. They will discuss the motivations for updating this framework, including the need to discuss complexity and systems thinking more explicitly in the context of development and evaluation of interventions. They will introduce the key points of the framework, aiming to help researchers work with other stakeholders to identify the key questions about complex interventions, and to design and conduct research with a diversity of perspectives and appropriate choice of methods. Kathryn Skivington is a Research Fellow at the MRC/CSO Social and Public Health Sciences Unit at the University of Glasgow. She has used various qualitative and quantitative methods to explore, develop and evaluate public health interventions, and has a broad research interest in health inequalities. Kathryn led the writing of the MRC/NIHR Framework to Develop and Evaluate Complex Interventions (2021). More recently she has been developing the process of taking a systems approach to intervention development, as PI on a CSO funded project to explore drug-related deaths in Scotland (as developmental work for co-produced interventions) and as investigator on a project related to physical activity and social connections for healthy ageing. Laurence Moore is Director of the MRC/CSO Social and Public Health Sciences Unit at the University of Glasgow as well as Co-Programme Leader of the Complexity Programme within the Unit. He is senior author on the MRC/NIHR Framework to Develop and Evaluate Complex Interventions (2021). Until 2013, Laurence was Professor of Public Health Improvement at Cardiff University and founding Director of DECIPHer, a UKCRC Public Health Research Centre of Excellence. He is a social scientist and statistician with a track record in the development and evaluation of interventions to improve public health. He has completed mixed methods evaluations of diverse interventions and programmes (e.g. related to exercise, diet, smoking), which have then had a direct impact on policy and practice. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-an-update-of-the-medical-research-council-mrc-framework... |
| Description | CECAN Webinar: Capturing Social Dynamics for Evaluation: Trajectory-Based Qualitative Comparative Analysis Online, 11 Nov 2020, Lasse Gerrits and Sofia Pagliarin |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Wednesday 11th November 2020, 13:00 - 14:00 GMT / 14:00 - 15:00 CET Presenters: Lasse Gerrits and Sofia Pagliarin Webinar Overview: Research methods usually struggle with the complexity of social processes, such as policy processes, and how they play out over time. As commonly acknowledged, the outcome of these processes observed 'here and now' is a result of what happened 'back then'. We therefore need to account for the conditions back in time to explain what is happening now. We present a novel version of Qualitative Comparative Analysis, called Trajectory-based QCA (TJ-QCA). This method compares processes over time to highlight the combinations of conditions that contribute to the outcome. In the presentation, we will discuss the technique of TJ-QCA as well as the causal reasoning behind the technique. We will use examples from policy evaluation to demonstrate how the technique works. Of course, there will be ample time to ask questions. Presenter Biographies: Prof. Dr. Dr. Lasse Gerrits is professor in urban planning at the Institute for Housing and Urban Development Studies at the Erasmus University Rotterdam (NL). Dr. Sofia Pagliarin is professor ad interim in political science at the Otto-Friedrich University of Bamberg (DE). They work together in aligning QCA with actual social complexity for the sake of policy and planning research. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-capturing-social-dynamics-for-evaluation-trajectory-bas... |
| Description | CECAN Webinar: Case-Based Modelling and Scenario Simulation for Ex-Post Evaluation Online, 6 Jul 2021, 3pm Corey Schimpf, Brian Castellani and Pete Barbrook-Johnson |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This seminar introduces case-based scenario simulation (CBSS) to evaluators. CBSS is well aligned with existing evaluation approaches, most directly realist evaluation which emphasizes 'what works for whom in which circumstances'. CBSS's approach enables users to explore in a simplified simulation environment the possibility of driving different clusters of outcomes in a desired evaluation direction, as well as explore counterfactuals. CBSS starts with the notion that interventions or policies may themselves not always be complex in nature, but that such interventions always happen in complex systems and that policy evaluators and analysts are in need of accessible tools or methods for studying how interventions unfold in these systems. In light of this challenge, we introduce a new approach that integrates two key methodologies for modeling and studying complexity: case-based methods and scenario simulation. The integrated approach of CBSS enables the identification and modeling of underlying groups of cases within a study and simulating hypothetical or 'what-if' scenarios to further explore how the system under study may respond to additional or alternative interventions. CBSS achieves this by modelling unique case types, their trajectories and the heterogeneous effects interventions may have on them. This work is grounded in a pluralist view of methodology, meaning that CBSS is not meant to replace other evaluation methods but rather to complement, support and extend them. More concretely, for case-based modelling CBSS uses k-means and the self-organizing map, a type of neural network, together to uncover underlying case groups within the data so that the unique profiles of case groups and their dynamics or responses can be modelled. The scenario simulation part of CBSS maps the cases to an n x m grid representing different configurational possibilities for the case groups and allows the analyst to make targeted interventions on case groups, representing 'what if' or counterfactual scenarios, to explore if this improves, worsens, or alters case groups trajectories. We briefly present COMPLEX-IT, which is a lightweight open-source web application that supports the use of CBSS regardless of an analyst's or researcher's experience with these methods. In this presentation we discuss CBSS in relation to evaluation purpose (learning or accountability) and evaluation design (cross-sectional or longitudinal). From these dimensions we highlight four evaluation situations as combinations of each dimension and identify four documented evaluation cases that map to each situation. We discuss how CBSS could complement and extend evaluations in each situation, noting the ways in which CBSS can help attend to differences within cases, alternative scenarios, and attention to complex dynamics operative in the systems under study. We conclude with a discussion of the possibilities CBSS opens up for different types of evaluation as well as the commonalities across these situations and the other methodological implications it has for evaluation research. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-case-based-modelling-and-scenario-simulation-for-ex-pos... |
| Description | CECAN Webinar: Co-creation of innovation: Group Concept Mapping (GCM) to value and engage More Knowledgeable Others in authoring and valuing complex systems, Tuesday 24th September 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: Co-creation of innovation: Group Concept Mapping (GCM) to value and engage More Knowledgeable Others in authoring and valuing complex systems, Tuesday 24th September 2019, 14:00 - 15:00 BST / 09:00 - 10:00 EDT Presenter: Mary Kane, Concept Systems, Inc. (US)" |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-co-creation-innovation-group-concept-mapping-value-and-... |
| Description | CECAN Webinar: Cynefin - Navigating Uncertainty Online, 18 Oct 2022, 12:00 - 13:00 Dave Snowden |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Webinar Overview: The recently published EU Field Guide to managing in complexity (and chaos) represents an important milestone in the shift of the complexity sciences into operational management. Aside from a simple five step approach to dealing with a crisis it also contains a body of pragmatic methods and tools to create a resilient organisation able to deal with rapidly changing and largely un-forecastable events. The guide is based on the Cynefin framework. This presentation will outline a new process for mapping strategic opportunities based on the guide, along with the core steps needed to build a future-fit organisations. Presenter Biography: Dave Snowden divides his time between two roles: founder Chief Scientific Officer of Cognitive Edge and the founder and Director of the Centre for Applied Complexity at the University of Wales. His work is international in nature and covers government and industry looking at complex issues relating to strategy, organisational decision making and decision making. He has pioneered a science based approach to organisations drawing on anthropology, neuroscience and complex adaptive systems theory. He is a popular and passionate keynote speaker on a range of subjects, and is well known for his pragmatic cynicism and iconoclastic style. He holds positions extra-ordinary Professor at the Universities of Pretoria and Stellenbosch as well as visiting Professor at Bangor University in Wales. He has held similar positions at Hong Kong Polytechnic University, Canberra University, the University of Warwick and The University of Surrey. He held the position of senior fellow at the Institute of Defence and Strategic Studies at Nanyang University and the Civil Service College in Singapore during a sabbatical period in Nanyang. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-cynefin-navigating-uncertainty/ |
| Description | CECAN Webinar: Demi-Regs and the Characteristics of Realist Evaluation, Thursday 5th December 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: Demi-Regs and the Characteristics of Realist Evaluation, Thursday 5th December 2019, 16:00 - 17:00 GMT Presenter: Justin Jagosh, Director for the Centre for Advancement in Realist Evaluation and Synthesis (CARES) " |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-demi-regs-and-characteristics-realist-evaluation |
| Description | CECAN Webinar: Developing Systems Approaches in the Water Sector Online, 21 Sep 2021, 1pm Brendan Bromwich |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenter: Brendan Bromwich, Principal Engineer, Asset Management and Investment Planning, Mott MacDonald Webinar Overview: There is a consistent appeal for a systems approach to be applied to water and environmental problems in new and emerging policy in England and Wales. Mott MacDonald and CECAN have a fruitful collaboration developing practical responses to this appeal. Systemic approaches that bridge engineering systems, such as infrastructure, with social-ecological systems, such as catchments or landscapes, create particular challenges for systems management. These challenges can be addressed by recognising and accommodating different implicit assumptions around risk, system boundaries, control and optimisation of systems that different organisations may hold. The webinar will review the application of Participatory Systems Mapping and Plural Rationality in practical system challenges in water and environment. We will draw on project experience including Water Resources South East's Regional Resilience Plan and Defra's Systems Analysis for Water Resources. We will conclude by looking at emerging challenges in the water sector including nested operation of catchment, regional and national water resource management systems. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-developing-systems-approaches-in-the-water-sector/ |
| Description | CECAN Webinar: Diagnostic Evaluation with Simulated Probabilities Online, 27 Apr 2021, 1pm Barbara Befani & Corinna Elsenbroich |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | In this webinar, Barbara and Corinna will present their recent paper in Evaluation on 'Diagnostic Evaluation with Simulated Probabilities'. As policy makers require more rigorous assessments for the strength of evidence in Theory-Based evaluations, Bayesian logic is attracting increasing interest; however, the estimation of probabilities that this logic (almost) inevitably requires presents challenges. Probabilities can be estimated on the basis of empirical frequencies, but such data is often unavailable for most mechanisms that are objects of evaluation. Subjective probability elicitation techniques are well established in other fields and potentially applicable, but they present potential challenges and might not always be feasible. We introduce the community to a third way: simulated probabilities. We provide proof of concept that simulation can be used to estimate probabilities in diagnostic evaluation and illustrate our case with an application to health policy. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-diagnostic-evaluation-with-simulated-probabilities/ |
| Description | CECAN Webinar: Evaluating government spending: Findings and recommendations from the latest National Audit Office report Online, 15 Mar 2022, 1pm Phil Bradburn |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenter: Phil Bradburn, National Audit Office Webinar Overview: At the National Audit Office (NAO), we believe good evaluation is key to helping government to learn whether its interventions are working and to demonstrate accountability for the use of public money. Our 2021 report Evaluating Government Spending finds that Government has recently taken steps to strengthen the way it evaluates its activity, but evaluation continues to be variable and inconsistent. Much of its work is either not evaluated robustly or not evaluated at all, which means government has little information in most policy areas on what difference is made by the billions of pounds being spent. Hear from the NAO on: the actions we found that government has taken since our previous report on evaluation in 2013; progress in addressing systemic barriers to good evaluation and good use of evaluation evidence; and What further steps the centre of government should take to build on their reforms to date. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-evaluating-government-spending-findings-and-recommendat... |
| Description | CECAN Webinar: Handling Complexity in Policy Evaluation - Introducing the new Magenta Book 2020 Supplementary Guide Online, 28 Apr 2020, Martha Bicket et al |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Tuesday 28th April 2020, 13:00 - 14:00 GMT Presenters: Martha Bicket, Dione Hills and Helen Wilkinson Webinar Overview: Complex systems are all around us. Their characteristics make their behaviour hard to predict and they present challenges to policy making and evaluation. CECAN was invited to produce a supplementary guide on complexity for the 2020 revision of the Magenta Book - UK Government's central guidance on how to evaluate policies, projects and programmes. The Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation is based on three years' research and development of evaluation methods by CECAN. With the help of useful examples, illustrations and tables, the guide explains what complexity is, its implications, and how evaluators and policy makers can plan, deliver and use complexity-appropriate evaluation to work with this complexity. In the first part of this webinar, Martha will present an overview of the Supplementary Guide and some of its key highlights. Martha will then be joined in conversation by Dione and Helen as they reflect together on their experiences of developing the guide, and on its implications for evaluation practice and policymaking in the UK. The Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation is written by Martha Bicket, Ian Christie, Nigel Gilbert, Dione Hills, Alexandra Penn and Helen Wilkinson. It is published by HM Treasury and accompanies the 2020 edition of the Magenta Book, released on 1st April 2020. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-handling-complexity-in-policy-evaluation-introducing-th... |
| Description | CECAN Webinar: How can System Dynamics Support Policy Evaluation? Online, 9 Sep 2020, Jonathan Nichols |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Wednesday 9th September 2020, 13:00 - 14:00 BST Presenter: Jonathan Nichols, Environmental Consultant Webinar Overview: System Dynamics is a mature, model-based policy design and evaluation method which has been used to harness the power of complexity on a wide range of policy issues, both in the UK and elsewhere, over the past 60 years. This webinar will introduce the basic concepts and tools which underly the method and will make the case that it presents evaluators with a "ready to go" complexity-appropriate toolkit. The webinar will illustrate the kind of policy insights the method can generate by presenting the results of a CECAN Fellowship project conducted in partnership with the European Master in System Dynamics programme. In this project a System Dynamics simulation model was built to identify risks and opportunities for using natural capital investment initiatives to achieve DEFRA's 25 Year Environmental Plan ambition of ensuring all soils in England are managed sustainably by 2030. The webinar will conclude with suggestions for how the rich history and vibrant contemporary practice of System Dynamics can be drawn on to support methodological innovation in complexity evaluation and tackle the practical challenges of applying complexity-appropriate methods in policy practice, and thus contribute to the broader CECAN agenda. Session Objectives: The webinar aims to address the following key questions: What is System Dynamics? How can System Dynamics help tackle some of the challenges of policy evaluation in complex systems? What insights has CECAN's research using System Dynamics generated so far? Presenter Biography: Jonathan is an environmental consultant with experience in the engineering, water and financial sectors, specialising in natural capital and soil science. As a graduate of the European Master in System Dynamics programme he holds a joint MPhil-MSc in System Dynamics from the University of Bergen (Norway) and the New University of Lisbon (Portugal) and an MSc in Business Administration from Radboud University (Netherlands). In 2019 he completed a CECAN Fellowship and Master's thesis project which used System Dynamics to explore the potential for public policies focused on natural capital investment to tackle soil degradation. Jonathan's enthusiasm for systems thinking and simulation modelling also extends to the commercial realm, having run a small business in the Netherlands using System Dynamics to help SMEs design their business growth strategies. He is now based in London, currently between roles, and will soon be joining a leading engineering and management consultancy to support decision makers at the creative frontier of the natural capital approach. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-how-can-system-dynamics-support-policy-evaluation/ |
| Description | CECAN Webinar: Inequality in a Context of Climate Crisis After COVID - the Role of Applied Social Science Online, 24 May 2022, 1pm David Byrne |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenter: David Byrne, Emeritus Professor of Sociology and Applied Social Science, University of Durham Webinar Overview: Widely acknowledged as the key driver of political discontent and social instability, economic inequality across high and middle-income countries is profoundly interconnected with climate change. Both of these issues are now set within the particularly acute context of COVID-19 and its aftermath. Confronting the crisis of these inherently interwoven issues is now the major problem for all political and governance systems. This Webinar will draw on the complex realist frame of reference to understand the character of social-cultural-economic-political-ecological systems AND to consider the role of applied social and ecological science combined in an inter or even post disciplinary fashion in informing policy and practice for what must be done. I will reflect on the role of scientists and practitioners, particularly people in roles like those of the analytical professions of the UK Civil Service, who can in Gramsci's terms be thought of as "Organic Intellectuals", in acting towards necessary system transformation. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-inequality-in-a-context-of-climate-crisis-after-covid-t... |
| Description | CECAN Webinar: Mapping the Cognitive Landscape of Productivity in Northern Ireland: A Systems Approach to Understanding Productivity Policy, Wednesday 12th February 2020 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: Mapping the Cognitive Landscape of Productivity in Northern Ireland: A Systems Approach to Understanding Productivity Policy, Wednesday 12th February 2020, 13:30 - 14:30 GMT Presenter: Jen Nelles, Centre for Regional Economic and Enterprise Development (CREED), University of Sheffield Management School, Productivity Insights Network (PIN)" |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-mapping-cognitive-landscape-productivity-northern-irela... |
| Description | CECAN Webinar: Participatory Systems Mapping for Policy Evaluation Online, 31 Mar 2021, 1pm Pete Barbrook-Johnson & Alex Penn |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | In this webinar, Pete and Alex will present their recent paper in Evaluation on the role 'Participatory Systems Mapping' can play in policy evaluation. As well as presenting the paper, they will give a sneak-peek of their forthcoming book on causal systems mapping, considering a range of related systems mapping methods and presenting some tentative advice on getting started with these methods themselves. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-participatory-systems-mapping-for-policy-evaluation/ |
| Description | CECAN Webinar: Redefining Evaluation to Support System Change: Theoretical and Practical Insights Online, 9 Dec 2021, 1pm Emily Gates |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenter: Emily F. Gates, Assistant Professor of Evaluation in the Measurement, Evaluation, Statistics, and Assessment Department at Boston College. Webinar Overview: What role should evaluation play in systems change and transformation? How must we rethink evaluation itself to adapt to this role? Emily Gates, assistant professor at Boston College, will share her thoughts on these questions grounded in theoretical scholarship and a case study. She draws on her work with Thomas Schwandt in their new book, Evaluating and Valuing in Social Research, to argue that the way we define evaluation needs to expand. Evaluating has conventionally been framed as an assessment of the value of a discrete intervention (i.e. policy, program) at a single point in time. Evaluating traditionally assumes relative clarity and agreement on the boundaries of an intervention, the underlying problem it addresses, and what constitutes success, such that the evaluation scope and methods focus primarily on evidence generation. For efforts to change systems amidst complexity, evaluating needs to expand to address boundary and value conflicts and support ongoing learning and adaptation. This calls for shifting evaluation from a determination of the value of an intervention to an ongoing process of co-developing the value of initiatives as they unfold in changing environments. To illustrate this shift, she will share a case study of evaluation within the Rippel Foundation's ReThink Health initiative to change the systems that influence health and wellbeing in the U.S. The case provides support that these two ways of seeing evaluation are not mutually exclusive but rather complementary for leaders who simultaneously work on discrete interventions and ongoing initiatives. The case study is featured in the summer issue of New Directions for Evaluation focused on systems- and complexity-informed evaluation. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-redefining-evaluation-to-support-system-change-theoreti... |
| Description | CECAN Webinar: Reflecting and Learning: Using Contribution Analysis, 18-Jan-23, Catherine Currie |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Webinar Overview: In 2019, building on the Centre for Evaluation Innovation's interest in Contribution Analysis as a flexible evaluative tool, Laura Hopkins and Catherine Currie presented a short paper at the UK Evaluation Society annual conference. It was called Tips and Tools for implementing Contribution analysis: Lessons Learnt from a series of advocacy evaluations. Three years on, one pandemic later and with the experience of several relevant new consultancies from a range of clients, Catherine has paused, reflected and updated the lessons with some new observations. This session explores practical evaluation management as well as lessons that touch on embracing major and emerging trends in monitoring, evaluation and learning. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-reflecting-and-learning-using-contribution-analysis/ |
| Description | CECAN Webinar: Systems Approaches to Water Management and Infrastructure Planning for Zero-Pollution Targets Online, 17 Jun 2021, 1pm Dr Ana Mijic |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenter: Dr Ana Mijic, Centre for Systems Engineering and Innovation Co-Director, Senior Lecturer in Water Systems Integration, Imperial College London Webinar Overview: At a time of system shocks, significant underlying challenges have been revealed in current approaches to delivering infrastructure, including the need for holistic assessment and that infrastructure users in many societies feel distant from nature. In this seminar, I will present methods for integrated land use planning and sustainable water management. The methods are based on a systems approach to infrastructure and environmental management, which identifies systems integration, integrated modelling and net-zero thinking as key research themes that can improve our understanding of the environmental impacts and the multifunctionality of infrastructure interventions and operational decisions. I will introduce a novel Systems Water Management (SYWM) framework, developed as part of a NERC Innovation Placement with the Environment Agency, and how its meta-model can be used to analyse a range of water management and land use planning challenges. I will also give examples of how the SYWM meta-model can be used to inform the development of integrated water management models at city and catchment scales to evaluate adaptation options designed to offset adverse impacts of development on water resources, water quality and flood risk. These models provide much needed systems-level evidence that has a potential to inform environmental policy, local planning and water infrastructure decisions. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-systems-approaches-to-water-management-and-infrastructu... |
| Description | CECAN Webinar: The Human, Learning, Systems approach to managing in complexity, Tuesday 1st October 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: The Human, Learning, Systems approach to managing in complexity, Tuesday 1st October 2019, 13:00 - 14:00 BST Presenter: Dr Toby Lowe, Senior Lecturer in Public Leadership and Management, Newcastle Business School " |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-human-learning-systems-approach-managing-complexity |
| Description | CECAN Webinar: The Practice of Blue Marble Evaluation Online, 5 Oct 2021, 4pm BST Michael Quinn Patton & Glenn G. Page |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Presenters: Michael Quinn Patton, Utilization Focused Evaluation & Glenn G. Page, SustainaMetrix/COBALT (Collaborative for Bioregional Action Learning & Transformation) Webinar Overview: The great acceleration of human activities over the past 10, 50, 100 years has changed the very nature of life support for all species who live together on this big "blue marble" we call planet Earth. As we navigate into the Anthropocene, with system-wide turbulence emerging all around us, we recognize the dire need for transformative change towards more just, equitable and regenerative futures. The first Nobel Prize Summit was held this year focusing on this dramatic challenge and concluded that "emerging technologies, social innovations, broader shifts in cultural repertoires, as well as a diverse portfolio of active stewardship of human actions in support of a resilient biosphere are highlighted as essential parts of such transformations". In order to evaluate such transformations, the field of evaluation itself requires transformation. Blue Marble Evaluation as a core set of principles was launched in November 2019 with this urgent need in mind and today features a worldwide network of over 600 people who are learning how to practice Blue Marble evaluation. As a global network, we are bearing witness to, and participating in, a major turn in the story of humanity. Through a growing planetary awareness, we are the first generation to see the powerful strengths and life threatening costs of globalization driven by dominant economic paradigm of profit. There are no silver bullet solutions and simplistic linear logic models confound the problem. To confront this reality, we need to let go of mechanistic, narrowly focused theories of change and meaningless, fabricated SMART Goals. We must let go of the anachronistic, controlling, and compliance-driven results-based management and outdated evaluation criteria. To transform systems, such as global food systems, it will require dismantling and transforming this dysfunctional underlying development architecture. This presentation will focus on the practice of Blue Marble Evaluation by connecting diverse local contexts to global issues and trends. For example, in preparation for the United Nations World Food Systems Summit Michael Quinn Patton will share results of how a global team of Blue Marble Evaluators are capturing and reporting the knowledge and voices of women who grow food, indigenous people who have worked their land for hundreds of years, smallholder farmers who feed their communities, and youth who are the future of agriculture. Through Blue Marble Evaluation, we put a spotlight on innovative ideas and practices while being diligent and thoughtful to integrate what is already working well. The World Food Program has documented that a child dies of hunger every 5 seconds. The COVID Pandemic has dramatically increased world hunger. To feed the world with nutritious food without damaging land and water requires food systems transformation worldwide. That's where Blue Marble Evaluation comes in, examining how food systems can be transformed to eliminate hunger while saving the environment. Glenn Page will emphasize how this work is applied to global hot-spots or "tipping elements" from the Amazon Rainforest to the Arctic Sea Ice that are central in regulating the state of the planet. He will illustrate how Blue Marble Evaluation emphasizes the concept of bioregionalism and the integrated nature of local to global (influence of chemical and physical compositions of the atmosphere, how biodiversity contributes to generating and maintaining soils, controlling pests, pollinating food crops, and the flow of biogeochemical cycles). He will illustrate examples of the practice of Blue Marble Evaluation and how it may be essential as our "hedge" against unanticipated ecosystem change that threatens the very life-support for our collective future on earth. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-the-practice-of-blue-marble-evaluation/ |
| Description | CECAN Webinar: The difference between Adrenaline and Core Strength - Investing in Resilience as an Organisational Capability Online, 2 Feb 2021, Jon Ainger & Henrietta Curzon |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Tuesday 2nd February 2021, 13:00 - 14:00 GMT Presenter: Jon Ainger, Director & Henrietta Curzon, Insight Lead (IMPOWER) Webinar Overview: Covid has undeniably pushed public sector organisation to their limits to overcome ongoing challenges. Their ability to bounce forward from these challenges is not only a testament to their resilience, but to their willingness to grasp everyday disruption. Resilience is often seen as the ability to 'keep going' - we believe a better definition is required. Our work with nine local authorities has sharpened our definition to recognise the core strength required to anticipate, adapt to and bounce forward from disruption. Our book 'The EDGEWORK Manifesto' describes how to deliver change and impact in complex systems - we now need to build resilience into that complexity. In our webinar, we will share key insights and have a rich discussion about what resilience really means, with a focus on practical elements. We are also keen to get feedback on the framework and its potential application from the webinar participants. Presenter Biography: Prior to IMPOWER, Jon worked in a range of public policy and strategy consulting roles, with a consistent focus on public service reform. After six months on secondment at the Cabinet Office, Jon joined IMPOWER at its inception in September 2000 and has spent the past 20 years working to build innovative approaches to solving complex public sector challenges. Jon has pioneered our work on demand management, applied behavioural science and codified our unique approach to delivering change in complex systems - EDGEWORK. Jon continues to work every day with clients, and on pushing IMPOWER to find new ways to help those clients improve and become more resilient. Henrietta worked in the police and local government for over 15 years and was seconded to the Public Service Transformation Network, a cross-government initiative, before joining IMPOWER in June 2014. Henrietta is IMPOWER's Insight Lead and has been instrumental in developing our understanding of resilience and testing and iterating our new framework to assess organisational resilience. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-the-difference-between-adrenaline-and-core-strength-inv... |
| Description | CECAN Webinar: Theory of Change - Getting the most out of it - Insights from a Practitioner, Tuesday 2nd July |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: Theory of Change - Getting the most out of it - Insights from a Practitioner, Tuesday 2nd July, 13:00 - 14:00 BST Presenter: Bridget Dillon, Head of Evaluation Profession, Department for International Development, UK Government (DFID)" |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-theory-change-getting-most-out-it-insights-practitioner |
| Description | CECAN Webinar: Trophic Analysis of Directed Networks Online, 22 Sep 2020, Robert MacKay |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Tuesday 22nd September 2020, 13:00 - 14:00 BST Presenter: Professor Robert MacKay, University of Warwick Webinar Overview: Directed networks are used in many domains. Particularly relevant examples to evaluation and appraisal are systems maps and influence maps. Trophic analysis indicates how far up or downstream each node is and computes a quantity called trophic incoherence, but perhaps better called circularity, which indicates the extent to which the network fails to line up with each node feeding to nodes at the next level. Robert will suggest incorporating trophic analysis into systems mapping, and present relations of trophic incoherence with some other network properties, and end with an application of incomplete pairwise comparison to football leagues. Learning Outcomes: 1. How to quantify up/down-streamness in directed networks. 2. How to quantify the directedness (or its complement, the circularity) of a directed network. Presenter Biography: Professor Robert MacKay is a Professor in the Mathematics Institute of the University of Warwick and Director of Mathematical Interdisciplinary Research at Warwick. He was Professor of Nonlinear Dynamics at Cambridge (1995-2000), founding Director of Warwick's Centre for Complexity Science from 2007-15 and President of the (UK) Institute of Mathematics and its Applications for 2012-13. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-trophic-analysis-of-directed-networks/ |
| Description | CECAN Webinar: Using the System Effects methodology to understand the user experience of complex systems, Tuesday 16th July 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Webinar: Using the System Effects methodology to understand the user experience of complex systems, Tuesday 16th July 2019, 09:00 - 10:00 BST (18:00 - 19:00 ACT) Presenter: Dr Luke K Craven, Research Fellow (Public Service Research Group), University of New South Wales, Canberra " |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-using-system-effects-methodology-understand-user-experi... |
| Description | CECAN Workshop - Commissioning Complex Evaluations, Monday 29th April 2019, London |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | "CECAN Workshop - Commissioning Complex Evaluations, Monday 29th April 2019, 10:00 - 16:00, Prince Philip House, 3 Carlton House Terrace London SW1Y 5DG In this workshop, we will: Share experience on procuring complexity-appropriate methods, and how to get round some of the rigidities in tender processes Pinpoint barriers to complexity appropriate methods in competitive tendering processes Develop immediately applicable suggestions and solutions; and have an opportunity to debate these with procurement staff (outside the context of an actual tendering exercise) to suggest mutually workable solutions. The workshop will bring together policy analysts from government departments who have to commission the evaluation of complex policies and contractors who have experience of bidding against ITTs for policy evaluation." |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-workshop-commissioning-complex-evaluations |
| Description | CECAN Workshop for BEIS: Dealing with complexity in policy evaluation, BEIS Conference Centre, 4th October 2019 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | "Dealing with complexity in policy evaluation, BEIS/CECAN Briefing and Workshop, BEIS Conference Centre, 4th October 2019 CECAN held a briefing and workshop session with BEIS on dealing with complexity in policy evaluation. Held at the BEIS conference centre in central London the session was well attended by over 20 BEIS policy-makers and analysts. The event was chaired by CECAN Deputy Director, Ben Shaw, and included presentations from Dione Hills and Helen Wilkinson (providing an overview of the principles of complexity plus a taster of some of the issues included in the forthcoming CECAN Annex on complexity to the revised Magenta Book - the Government's guidance on evaluation); Simon Shrimpton (from Defra) and Martha Bicket (re: the development of Defra's Complexity Evaluation Framework which CECAN produced); Alex Penn and Pete Barbrook-Johnson (a practical example of complexity appropriate methods being used in government, focused on Participatory systems mapping). |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/news/dealing-complexity-policy-evaluation-beiscecan-briefing-and-workshop |
| Description | CECAN Workshop: "Participatory Systems Mapping", 18th - 20th March 2019, University of Surrey, Guildford |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Alex Penn and Pete Barbrook-Johnson delivered a ""Participatory Systems Mapping"" workshop 18th - 20th March 2019, University of Surrey, Guildford, UK This course will introduce participants to the innovative participatory systems mapping methodology being developed by researchers in CECAN, which allows researchers to bring together diverse knowledge and stakeholders in order to construct shared causal maps of an issue or system. The course will introduce participants to the theory behind, and analytical approaches to, systems maps, as well as giving participants significant time spent in hands-on practical workshops building a map of their own. Time will also be given to considering the wider value and use of participatory modelling approaches such as systems mapping. Indicative content includes: Theory • Background on fuzzy cognitive mapping, causal mapping, and dependency modelling approaches • Analysis approaches (including simple dynamic modelling, network analysis, map interpretation for scenario exploration and intervention design combining quantitative nalysis and stakeholder beliefs) and their strengths and weaknesses. • Broader consideration of the use and value of participatory modelling Model construction • Practical skills and tools to build, verify and refine a systems map alone or in a group. Practical hands-on workshops will provide participants with experience of • Building, verifying and refining a map of an area of participants' interest, alone or in a group • Reflection on the faciltiation process • Observation of, and reflection on, group dynamics during model construction • Preliminary observation and discussion of map structure and potential implications Learning Outcomes: On successful completion of this course, participants will be able to: • Demonstrate a critical understanding of the ideas behind systems mapping (C,K) • Build a map on their own (C,P) • Facilitate a group to build a map (C,P,T) • Understand when a map might be used, and how it fits in with other methodologies. (C,K) • Be able to critically engage with existing research using this and similar methods (C,T) • Be able to analyse and interpret maps (C,K,P) Key: C-Cognitive/Analytical; K-Subject Knowledge; T-Transferable Skills; P- Professional/ Practical skills |
| Year(s) Of Engagement Activity | 2019 |
| Description | CECAN members delivered a systems mapping presentation as part of the Sunbelt Conference in Montreal, 20 June 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Former CECAN Fellow, Verena Knerich, and CECAN Senior Research Fellows Pete Barbrook-Johnson and Alex Penn delivered a presentation as part of the Sunbelt Conference in Montreal. The presentation was called "Structure-based Identification of Threats and Opportunities from Participatory Systems Mapping". |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://sunbeltconference2019.sched.com/event/Qudq/innovations-in-social-network-based-interventions... |
| Description | Introduction to CECAN's Magenta Book Supplementary Guide "Handling Complexity in Policy Evaluation", part of the "Policy That Works" conference for civil servants (Cabinet Office). Martha Bicket, 28 Feb 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Martha Bicket presented an Introduction to CECAN's Magenta Book Supplementary Guide "Handling Complexity in Policy Evaluation", a presentation as part of a session on "Complex Evaluation: The Art of the Possible" as part of the 4-day "Policy That Works" conference for civil servants, run by the Evaluation Task Force (Cabinet Office). A note on numbers from the tech team facilitating the conference: "Total no of HMG people who'd signed up as of yesterday was 2080. 94 attended your slot. Three slots running in parallel - yours had the second highest numbers." |
| Year(s) Of Engagement Activity | 2022 |
| Description | Invited blog post: Participatory Systems Mapping (Gilbert), University of Surrey "Technology and Society at Surrey" series, Feb 2021 |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Nigel Gilbert was invited to contribute a blog post to the University of Surrey's "Technology and Society at Surrey" blog, which highlights spotlights and showcases from the University of Surrey's Strategic Research Theme Technology and Society. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://blogs.surrey.ac.uk/technology-and-society/2021/02/12/participatory-system-mapping/ |
| Description | Liz Varga was invited to present about CECAN at UCL's OMEGA Centre seminar series on 7 Dec 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | "Evaluation of Complexity Across the Nexus: A Summary of CECAN's Research Findings of Relevance to Mega Infrastructure Policy and Planning". Response after the event: From: Dimitriou, Harry Date: 11 December 2022 at 08:44:00 GMT To: Liz Varga Subject: Thanks for your OMEGA Seminar Presentation Hi Liz, Thank you once again for your very clear and articulate OMEGA Seminar presentation last Wednesday. It went down very well. As you will have appreciated, the audience (especially the students) very much welcomed the holistic perspective presented. Having run a research programme with similar sized funding, I wish to reiterate my congratulations to your team for the most effective dissemination of your insightful findings and taking this forward into institutional capacity building and training. Really impressive - this is something the OMEGA team has only achieved through its IPAD MSc programme It is very reassuring to see how we (what is left of the OMEGA research team) are on the same page despite my nip-picking and even though we were essentially covering different sectors. Thank you once again for your efforts and to you Nigel for agreeing for Liz to make the presentation. Maybe John and you/your team can share our respective insights into the difference between project appraisal and evaluation for megaprojects and the necessity of employing boundaries of systems thinking that are not too suffocating to appreciate bigger picture concerns. Unfortunately, the recording quality of the seminar as streamed was not fit for purpose and will not be used. We will, however, with your permission, post the ppts on the OMEGA website and produce a report of the event to also be posted on the website once you have approved its content. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.ucl.ac.uk/bartlett/planning/events/2022/dec/omega-seminar-evaluation-complexity-across-n... |
| Description | Magenta Book Annex Roadshow Event 1 - Handling Complexity in Policy Evaluation: Introducing the Magenta Book 2020 Supplementary Guide on Complexity, Tuesday 3rd March 2020, London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Magenta Book Annex Roadshow Event 1 - Handling Complexity in Policy Evaluation: Introducing the Magenta Book 2020 Supplementary Guide on Complexity, Tuesday 3rd March 2020, 13:30 - 16:30, Church House, Deans Yard, Westminster, London SW1P 3NZ. Circa 90 people registered with c. 70 in attendance. Policy analysts, consultants, evaluators etc. Invited speaker: Steven Finch from DfT and Government Cross Evaluation Group. Event was professionally filmed and will be posted on the CECAN website after the second of the two Roadshow events has taken place in Birmingham (16th March 2020). |
| Year(s) Of Engagement Activity | 2020 |
| Description | Martha Bicket and Helen Wilkinson were invited to present at UKES Webinar July 6th 2022 - Evaluation Guidance |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Martha and Helen were invited to present at a UKES webinar (July 6th 2022) - Evaluation Guidance: "Professor Laurence Moore from University of Glasgow will be discussing the new framework for developing and evaluating complex interventions - an update of the Medical Research Council guidance, in light of the above. Initial comments from Dr Helen Wilkinson (Risk Solutions) and Dr Martha Bickett (CECAN) who worked on the supplementary guidance for Handling complexity in policy evaluation." At the event Helen and Martha discussed maintaining and updating guidance as a more 'living' resource. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.evaluation.org.uk/event/webinar-evaluation-guidance-what-makes-the-difference/ |
| Description | Martha Bicket on Advisory Panel for Centre for Evidence and Implementation |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Martha Bicket was invited to join the Advisory Group at the Centre for Evidence and Implementation who (as part of a wider collaboration with Monash University and the Violence Reduction Centre at Cambridge University) had been writing a guidance document for the Youth Endowment Fund, drawing on a number of existing guidance documents, including CECAN's Magenta Book supplementary guidance on evaluating complexity. |
| Year(s) Of Engagement Activity | 2022,2023 |
| Description | Nigel Gilbert and colleagues invited to give the Keynote presentation at Surrey's Dept of Sociology Research Afternoon event, 22 May 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Nigel Gilbert, Ben Shaw and Pete Barbrook-Johnson were invited to give the Keynote presentation at Surrey's Dept of Sociology Research Afternoon event. 'What is policy evaluation and does it support evidence-based policy?' Event summary: This event considers the 'real world' challenges of the policy/research nexus. Drawing on the perspectives and experiences of academics in the Department of Sociology we will consider the role of policy evaluation and examine what evidence-based practice in the 'real world' might look like. Professor Nigel Gilbert, Dr Ben Shaw, Dr Pete Barbrook-Johnson will examine 'What is policy evaluation and does it support evidence-based policy?' and Professor Nigel Fielding will consider 'evidence-based practice in the real world'. Our panel will then broaden this out and discuss: 'what policy needs from research and what research needs from policy.' The panel comprises academics working in diverse policy environments. Dr Sara Dicerto is a Senior Commissioning Officer at Surrey County Council. Ms Megan Georgiou is a Programme Manager, Royal College of Psychiatrists. Dr Paul Quinton is an Evidence and Evaluation Advisor at College of Policing. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.eventbrite.com/e/evidence-based-policy-and-the-social-sciences-tickets-59853840448?utm_s... |
| Description | Nigel Gilbert delivered a bespoke presentation ("Evaluating Complexity") for the Department of Health and Social Care, 11 Sept 19, London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nigel Gilbert delivered a bespoke presentation ("Evaluating Complexity") for the Department of Health and Social Care. The presentation took place at 39 Victoria Street, London, SW1H 0EU and was live-streamed to DHSC offices in Leeds. The presentation was delivered to DHSC Ops Research (OR) analysts covering complex evaluation and how it links to soft OR methods. It also included signposting to useful further reading/learning resources etc available via the CECAN website. c. 40 participants. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Nigel Gilbert presented CECAN's work at University of Surrey's Faculty of Arts and Social Sciences' Festival of Research, 23 Jan 2020 |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | Nigel Gilbert gave a presentation and presented a poster about CECAN's work at the University of Surrey's Faculty of Arts and Social Sciences' Festival of Research |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.surrey.ac.uk/news/celebrating-years-fass-festival-research |
| Description | Nigel Gilbert was invited to present at the UKES Conference in May 2022 - about PRSM |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nigel gave a presentation about PRSM at the UKES online conference in May 2022. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.evaluation.org.uk/app/uploads/2022/05/UKES-PDF-Programme-13-May-2022.pdf |
| Description | Nigel Gilbert was invited to talk at the Cross Government Evaluation Group meeting, 23 May 2019 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nigel Gilbert was invited by Steven Finch, Head of Evaluation, Evaluation Centre of Excellence, Department for Transport, to talk at the Cross Government Evaluation Group meeting on Thurs 23 May at HMT. Nigel gave an hour-long presentation, which covered: 1. Update on CECAN's programme if work, learnings to date, current plans following the Centre's extension from ESRC; 2. Reflections from the Commissioning Complex Evaluations workshop. Other attendees at the meeting included Marianne Law (BEIS), Bridget Dillon (DFiD), Tarran Macmillan (Defra), Mike Daly (DWP) and Steven Finch (DfT). |
| Year(s) Of Engagement Activity | 2019 |
| Description | Pete Barbrook-Johnson delivered an informal seminar "Exploring complex data from a case-based perspective: the COMPLEX-IT software", 19 Nov 19, Guildford |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Other audiences |
| Results and Impact | At University of Surrey, Dr. Pete Barbrook-Johnson (Senior Research Fellow, Surrey) and Dr. Corey Schimpf (Learning Analytics Scientist, Concord Consortium) delivered an informal seminar entitled "Exploring complex data from a case-based perspective: the COMPLEX-IT software". In this seminar, we will present our recent work which brings together, and makes accessible, traditional and machine learning cluster analysis in an easy-to-use software interface called COMPLEX-IT. We will describe how you can use COMPLEXIT to explore your data and uncover complex patterns. In addition, we will outline current plans to extend the software to include 'quick-and-dirty' agent-based modelling of your topic. Corey's visit was paid for as part of a Surrey Faculty pump-priming award and the work on COMPLEX-IT is part of the wider research of CECAN. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Presentation of Pluralistic Evaluation Framework methodology at a British Ecological Society conference, Bristol, 3 March 2020 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Adam Hejnowicz and and Richard Gunton presented the Pluralistic Evaluation Framework methodology (developed during Richard's CECAN Fellowship) at a British Ecological Society and Valuing Nature conference in Bristol |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://www.britishecologicalsociety.org/wp-content/uploads/2020/02/BES-symposium-brochure-feb-2020-... |
| Description | Transforming complex public policy evaluation and design through co-production: A Defra Case Study. Presented by Amy Proctor at the Royal Geographical Society Annual Conference, 2nd September 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | On behalf of CECAN, Amy presented the following paper at this conference: Proctor, A., Hejnowicz, A, Rowe, F and Phillipson, J. Transforming complex public policy evaluation and design through co-production: A Defra Case Study. Conference paper presented by Amy Proctor at the Royal Geographical Society Annual Conference, 2nd September 2021 |
| Year(s) Of Engagement Activity | 2021 |
| Description | UKES Annual Conference 25-27th May 2021, Magenta Book - Booth demo Martha Bicket, Helen Wilkinson |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Martha and Helen presented at the UKES Annual Conference on behalf of CECAN |
| Year(s) Of Engagement Activity | 2021 |