Centre for the Evaluation of Complexity Across the Nexus (CECAN)
Lead Research Organisation:
University of Surrey
Department Name: Sociology
Abstract
Contributing to the design of public policy that can respond to the problems facing society is challenging. Not only are policies difficult to design well, but it can also often be difficult to know whether the policy was successful. Evaluating whether changes have been caused by the policy or for other reasons can be hard to determine. Policy planning and evaluation are especially challenging when social and environmental problems are intertwined. For example, designing and evaluating a policy on the risk of flooding needs close liaison with experts, local authorities and citizens on issues such as coastal erosion, drainage, farming, housing, transport - and the policy needs to be both flexible and robust across these areas. Complex problems require new solutions with special kinds of methods and approaches to study them.
But how do we design policies that can keep up with environmental and social issues that are fundamentally dynamic? How do we evaluate whether a policy has been effective when the policy has been deliberately designed to adapt over time? What kinds of new methods do we need to be able to design and evaluate complex policies that cross sectors? How do we act quickly to respond to changing policy problems? How do we bring together and mobilise expertise across many different subject areas to work on a common problem and produce new solutions? How can we ensure that policymakers, evaluators, practitioners and students have the skills to design and evaluate policies that need constant updating? These are some of the questions this new Centre will begin to answer.
To do this, the Centre will foster an 'open research' culture of knowledge exchange founded on a growing network of policy-makers, practitioners and researchers. At its core is a strong group of academic and non-academic experts with many years' experience of working on these kinds of problems. The Centre is uniquely positioned to address some of the big policy challenges because:
* each member of the core Centre team is a global figure in their particular research area, has worked with or for policy makers or in policy relevant areas, and is a methodological pioneer who in his or her own way has contributed and created cutting-edge methodological approaches to understanding these complex problems;
* supplementing the core Centre team, a network of academic and non-academic Fellows and Associates will join the Centre for weeks or months. They will offer expert advice on areas that need specialist input for particular policy initiatives and innovations.
The Centre's work will include developing new and enhancing existing methods for the evaluation of policies in complex settings; piloting these methods on a range of evaluation projects; organising educational programmes for practitioners, academics and policymakers; and publishing guides and toolkits for evaluators, those commissioning evaluations, and policy audiences.
In the first year, we will:
* Link to on-going and forthcoming policy evaluations, so that we can advise on more advanced or cost-effective
approaches that might be used.
* Organise workshops that bring together evidence teams, policy makers and academics to work with us to understand the policy challenge and develop new ways to frame policy evaluation problems;
* Host cross-sectoral seminars on the state of the art of evaluating complexity;
* Initiate training events for evaluation commissioners, practitioners and academics to disseminate emerging lessons quickly;
* Publish short summary and agenda setting briefings on policy design and evaluation in complex settings;
* Publish a guide to what a complexity-sensitive evaluation toolkit might look like. This will become a 'living document', updated as knowledge develops.
Overall, this new and exciting Centre will enable the integration of a complex systems approach into the policy evaluation process and hence contribute to more effective policy-making.
But how do we design policies that can keep up with environmental and social issues that are fundamentally dynamic? How do we evaluate whether a policy has been effective when the policy has been deliberately designed to adapt over time? What kinds of new methods do we need to be able to design and evaluate complex policies that cross sectors? How do we act quickly to respond to changing policy problems? How do we bring together and mobilise expertise across many different subject areas to work on a common problem and produce new solutions? How can we ensure that policymakers, evaluators, practitioners and students have the skills to design and evaluate policies that need constant updating? These are some of the questions this new Centre will begin to answer.
To do this, the Centre will foster an 'open research' culture of knowledge exchange founded on a growing network of policy-makers, practitioners and researchers. At its core is a strong group of academic and non-academic experts with many years' experience of working on these kinds of problems. The Centre is uniquely positioned to address some of the big policy challenges because:
* each member of the core Centre team is a global figure in their particular research area, has worked with or for policy makers or in policy relevant areas, and is a methodological pioneer who in his or her own way has contributed and created cutting-edge methodological approaches to understanding these complex problems;
* supplementing the core Centre team, a network of academic and non-academic Fellows and Associates will join the Centre for weeks or months. They will offer expert advice on areas that need specialist input for particular policy initiatives and innovations.
The Centre's work will include developing new and enhancing existing methods for the evaluation of policies in complex settings; piloting these methods on a range of evaluation projects; organising educational programmes for practitioners, academics and policymakers; and publishing guides and toolkits for evaluators, those commissioning evaluations, and policy audiences.
In the first year, we will:
* Link to on-going and forthcoming policy evaluations, so that we can advise on more advanced or cost-effective
approaches that might be used.
* Organise workshops that bring together evidence teams, policy makers and academics to work with us to understand the policy challenge and develop new ways to frame policy evaluation problems;
* Host cross-sectoral seminars on the state of the art of evaluating complexity;
* Initiate training events for evaluation commissioners, practitioners and academics to disseminate emerging lessons quickly;
* Publish short summary and agenda setting briefings on policy design and evaluation in complex settings;
* Publish a guide to what a complexity-sensitive evaluation toolkit might look like. This will become a 'living document', updated as knowledge develops.
Overall, this new and exciting Centre will enable the integration of a complex systems approach into the policy evaluation process and hence contribute to more effective policy-making.
Planned Impact
WHO WILL BENEFIT FROM THE CENTRE'S WORK?
The key users of the Centre's outputs will be policy analysts, policy professionals, regulators and evaluation practitioners both within and outside government. These users are well versed in the Magenta Book, its evaluation approaches and their limitations. During discussions held for this proposal, they indicated that they want to know how to go beyond the Magenta Book to address pressing and complex policy evaluation challenges. These users are located in government departments and agencies, local partnerships (such as local enterprise partnerships and nature partnerships), local authorities, and third sector and commercial consultancy organisations.
The primary beneficiaries will be those working in the policy process - in policy analysis, development and implementation - in relation to Energy-Environment-Food Nexus domains. However, the work will be of relevance and interest far beyond this, since they interface with a multitude of other complex global problems, e.g. health, inequalities, education and international development. The methodological advances made by the Centre will also be of interest to a wider academic and practitioner community in the UK, Europe and globally as well as business, consultancy and civil society organisations who operate within the Nexus and beyond.
The ultimate beneficiary of the Centre's work will be the public and society through improved and more cost-effective policymaking.
WHAT IMPACTS WILL THE CENTRE GENERATE?
The Centre will deliver instrumental, conceptual and capacity building impacts for economy and society.
The main instrumental impact of the Centre will be to enable better evaluation, evidence, policy-making and understanding of 'what works' - whether in public policy, business, or the third sector across the Nexus.
Better policy design and implementation will deliver benefits to stakeholders, the general public and the environment, delivering improved value for money for the taxpayer, a better environment for businesses to thrive and grow, and enhanced individual experiences of the environment, with potential for improved health and well-being.
The Centre will strengthen the links between policy-makers, evaluation practitioners and academia. This will enhance the ability of government to draw on expert resources and increase the understanding of government evidence and evaluation needs in academia and consultancy.
The conceptual impact of the Centre will be the development and availability of novel and tested evaluation methods suited to complex settings. The Centre will also develop and shape the framings of issues associated with complexity in the Nexus, providing value, clarity and insight to policy generalists and specialists and wider beneficiaries alike.
The Centre will improve the capacity of policy-makers to conduct, commission and use evaluations in complex settings through awareness raising, development of knowledge and provision of tools. Similarly, capacity building amongst evaluation practitioners will enhance their ability to provide advanced evaluation skills and methods to government and other users.
The development of this evaluation capacity will enhance the standing of the UK internationally and also create opportunities for evaluation practitioners to access and be competitive in international markets with consequent economic benefits and impact for the UK.
The academic impact of the Centre will include the development of new knowledge and methods for policy evaluation, the building of a broad research community working in this area, the promotion of interdisciplinary working between diverse areas, the possibility of new approaches that go beyond the sum of their parts and the development of the research and policy engagement skills of those involved in the Centre over a wide range of stages of their careers. These will enhance the reputation of UK academia internationally.
The key users of the Centre's outputs will be policy analysts, policy professionals, regulators and evaluation practitioners both within and outside government. These users are well versed in the Magenta Book, its evaluation approaches and their limitations. During discussions held for this proposal, they indicated that they want to know how to go beyond the Magenta Book to address pressing and complex policy evaluation challenges. These users are located in government departments and agencies, local partnerships (such as local enterprise partnerships and nature partnerships), local authorities, and third sector and commercial consultancy organisations.
The primary beneficiaries will be those working in the policy process - in policy analysis, development and implementation - in relation to Energy-Environment-Food Nexus domains. However, the work will be of relevance and interest far beyond this, since they interface with a multitude of other complex global problems, e.g. health, inequalities, education and international development. The methodological advances made by the Centre will also be of interest to a wider academic and practitioner community in the UK, Europe and globally as well as business, consultancy and civil society organisations who operate within the Nexus and beyond.
The ultimate beneficiary of the Centre's work will be the public and society through improved and more cost-effective policymaking.
WHAT IMPACTS WILL THE CENTRE GENERATE?
The Centre will deliver instrumental, conceptual and capacity building impacts for economy and society.
The main instrumental impact of the Centre will be to enable better evaluation, evidence, policy-making and understanding of 'what works' - whether in public policy, business, or the third sector across the Nexus.
Better policy design and implementation will deliver benefits to stakeholders, the general public and the environment, delivering improved value for money for the taxpayer, a better environment for businesses to thrive and grow, and enhanced individual experiences of the environment, with potential for improved health and well-being.
The Centre will strengthen the links between policy-makers, evaluation practitioners and academia. This will enhance the ability of government to draw on expert resources and increase the understanding of government evidence and evaluation needs in academia and consultancy.
The conceptual impact of the Centre will be the development and availability of novel and tested evaluation methods suited to complex settings. The Centre will also develop and shape the framings of issues associated with complexity in the Nexus, providing value, clarity and insight to policy generalists and specialists and wider beneficiaries alike.
The Centre will improve the capacity of policy-makers to conduct, commission and use evaluations in complex settings through awareness raising, development of knowledge and provision of tools. Similarly, capacity building amongst evaluation practitioners will enhance their ability to provide advanced evaluation skills and methods to government and other users.
The development of this evaluation capacity will enhance the standing of the UK internationally and also create opportunities for evaluation practitioners to access and be competitive in international markets with consequent economic benefits and impact for the UK.
The academic impact of the Centre will include the development of new knowledge and methods for policy evaluation, the building of a broad research community working in this area, the promotion of interdisciplinary working between diverse areas, the possibility of new approaches that go beyond the sum of their parts and the development of the research and policy engagement skills of those involved in the Centre over a wide range of stages of their careers. These will enhance the reputation of UK academia internationally.
Publications
Calder M
(2018)
Computational modelling for decision-making: where, why, what, who and how.
in Royal Society open science
| Description | [Note that, as advised by ESRC, all impacts from 1 March 2019 are recorded under CECAN's continuation funding, ES/S007024/1.] The main conclusions we have drawn from the work of CECAN over three years are: Linear methods such as Randomised Control Trials are simply not suitable for the effective evaluation of the sorts of complex policies that many Government Departments are responsible for. All government interventions in social and environmental domains are acting on complex systems. The best policy teams, analysts and evaluators in government acknowledge this, and are taking steps to address it. There are some occasions where evaluations may not need to be complexity-appropriate, though these are rare, and the emphasis should be on being sure that this is the case, rather than assuming simplicity. Government departments crave the time and space to develop capacity for complexity-appropriate evaluation. Without a clear picture of available solutions, they may struggle to articulate clearly their complexity-related needs; evaluators must give a strong and creative lead to government for successful complexity-appropriate evaluation. There is not enough of a focus on cross-departmental evaluation to deal with unintended consequences and Nexus interactions beyond one policy area. Nexus-related objectives and outcomes must be considered. Acknowledging the complexity of an area highlights the importance of decisions around the scope of an evaluation; artificially separating questions about the intervention, knock-on effects, and its context, may be counter-productive. The policy evaluation landscape is shifting; government is evaluating more varied policies, often made up of multiple projects, delivered in collaboration with multiple stakeholders. This means effective evaluation designs will be more bespoke, co-produced, and flexible; echoing the implications of complexity. Evaluation should be embedded throughout the policy process. This improves the quality of evaluations and increases their ability to deal with complexity, by improving, for example, data availability and collection, and policy design for learning. Practitioners intuitively want to evaluate in a complexity-appropriate way but are constrained by practicalities, time, and capacity. Government should be clear in its demand for complexity-appropriate evaluation from contractors. There is both a need and an opportunity to respond to the growing appetite and demand for approaches that embed complexity thinking into policy making across the full 'policy cycle' and at all levels of government. The demand from policy-makers and analysts has been very clear throughout our interactions with them. CECAN was formed to explore how to deal with complexity in policy evaluation but complexity needs to be dealt with at all stages and levels of policy design and implementation. Commissioning will need to change to be more complexity-appropriate. We have highlighted the importance of the commissioning process for delivering successful evaluation outcomes and a need to reshape this process to more effectively incorporate the challenges posed by complexity. Complexity-appropriate evaluation methods are: co-produced with users, adaptable, iterative, and often a combination or hybrid of other methods. Rather than delivering single definitive but flawed answers, they provide ranges and likelihoods of outcomes, and deliver enormous learning in the process of their use. Users must be willing to accept and use preliminary or provisional findings, and evaluators must be willing to provide these. Many methods can be re-purposed for complex policies and contexts; the innovation is getting them in the right place, in the right hands, and using them in the right combination, at the right time. Using complexity-appropriate methods to develop theory of change maps and log frames that include feedbacks and interaction between levels and paths is a key first step. Computational models are increasingly being used to assist in developing, implementing and evaluating public policy. Policy models can have an important place in the policy process because they could allow policy makers to experiment in a virtual world, and have many advantages compared with randomised control trials and policy pilots. However, there are still some challenges that need to be overcome if policy models are to make an effective contribution. CECAN's work would suggest that policy modelling will continue to grow in importance as a component of public policy making processes, but if its potential is to be fully realised, there will need to be a melding of the cultures of computational modelling and policy making. Evaluation teams, hierarchical structures within and between organisations, and contracts need to be set up to have the capacity to adapt to the emergent requirements of complex-appropriate evaluations. Departments must be prepared to operate in a manner that mirrors the complexity of the systems within which they operate thus allowing for an organisational culture that facilitates dialogue between different areas of expertise required for effective evaluation. Interactional expertise (i.e. understanding a subject, concept, or method well-enough to discuss it appropriately, without being an expert) is vital. Evaluation planners and leads must have high interactional expertise, must develop it as quickly as possible, and regularly maintain it. We have learned that funding is very difficult to draw out of government departments. Budgets are tight and restrictions on spending (procurement rules) are very limiting. Mechanisms for accessing funding are unclear. Traditional academic structures are incompatible with bidding for government contracts (we have learned that responding to ITTs requires you to move much more quickly and with almost no negotiating power, which would not work through university infrastructure). We have created CECAN Ltd to overcome this difficulty. Novel approaches to running research centres, which embed researchers in policy settings, are required to successfully address real world challenges. Utilising a co-production process between researchers, evaluation practitioners, policy makers and analysts allows for direct innovation and impacts on the conduct of policy evaluation in government and research, and secondly, wider evaluation insights relevant to users in government, evaluation practitioners and academic researchers. |
| Exploitation Route | The main impact of our work will be on policy professionals, policy analysts in government and evaluation practitioners in the private sector and third sector. We envisage that they will gradually become better informed, more comfortable with, and more able to engage with complexity as a result of our work and that ultimately the practice of policy evaluation will be transformed to become complexity-appropriate. |
| Sectors | Agriculture, Food and Drink,Environment,Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice,Culture, Heritage, Museums and Collections,Transport |
| URL | https://www.cecan.ac.uk |
| Description | [Note that as advised by ESRC, all impacts from 1 March 2019 are recorded under CECAN's continuation funding, ES/S007024/1.] A unique coalition of academic experts and senior practitioners, CECAN has proven itself to be ideally positioned for bringing academic depth and breadth to the development and understanding of complexity appropriate evaluation, while maintaining a strong practitioner perspective to the work. We have made advances towards transforming the practice of policy evaluation in nexus areas through leadership, methodological innovation, methodological application, and capacity building. Our co-production approach has been a learning experience for both CECAN and practitioners and ensured policy analysts and evaluators are more aware of, and use, more powerful, complexity appropriate methods to tackle their evaluation challenges. CECAN has also fostered new partnerships and relationships within and between departments and more widely, providing an extensive network for the exchange of expertise and best practise. The gains from our case study work (spanning a multi-billion pound policy portfolio with co-funders Defra, BEIS, the Environment Agency and the Food Standards Agency) have been two-way: the co-funders have learned about better approaches to their evaluation tasks, and the Centre has profited from the opportunity to develop and apply new or modified evaluation methods to real and practical issues: a true instance of 'knowledge exchange'. CECAN's capacity building portfolio has created a large cadre of analysts who have had exposure to and strengthened familiarity with complexity appropriate evaluation tools and methods. Our Syllabus on Evaluation of Complex Policy and Programs is also stimulating interest from practicing and future policy analysts and evaluators. A range of methods have been developed, tried and tested in case studies and our learning synthesised in a draft Annex to the government's Magenta Book. Compliance with the Magenta Book is often stated as a requirement for evaluations and is referred to by commissioners and practitioners alike, hence its strategic importance. The Annex is due for publication alongside the new edition of the Magenta Book (expected Summer 2019), giving it the potential to have national impact on evaluation practice across government. Most recently, CECAN has launched an independent commercial arm, CECAN Ltd., which will contribute to the financial sustainability of CECAN's work beyond the period supported by Research Council funding. CECAN Ltd has already been successful in securing a contract to develop a Complexity Evaluation Framework for Defra, which is intended for use as an actionable resource by Defra policy analysts and a first (important) step in facilitating much needed dialogue between evaluators and senior policy professionals. |
| First Year Of Impact | 2016 |
| Sector | Agriculture, Food and Drink,Energy,Environment,Government, Democracy and Justice |
| Impact Types | Economic,Policy & public services |
| Description | (May 2018) Corinna Elsenbroich was asked to contribute to Surrey PG/T programmes delivered by the Sociology Dept |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Corinna was asked by Surrey's Department of Sociology to contribute to the Evaluation Research PG/T module, where she delivered two weeks on "Complexity in Evaluation" (informed by CECAN Syllabus). She was also asked to take over the Evaluation Research PG/T module from Sept 2018 so that has been "revamped" closely along the lines of the CECAN Syllabus. |
| Description | Agent Based Modelling short course delivered at University of Surrey |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | http://www.cecan.ac.uk/news/agent-based-modelling-course-a-great-success |
| Description | Bespoke two-day day workshop for the Department of Education broadly based on CECAN's Evaluation of Complex Policies and Programmes syllabus, 19th and 20th April 2018 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | On February 8, 2018, Catherine Newsome (Strategic Research Team Leader, Strategic Analysis and Research Division) at Department for Education contacted CECAN to request training in evaluation of complex interventions to support development of a process and impact evaluation of a intervention which will include multiple components working across schools, health and public health, with a change in systems and processes as well as workforce and individual interventions delivered within this. We ran a two day workshop for the Department of Education broadly based on CECAN's Evaluation of Complex Policies and Programmes syllabus. A pre course questionnaire ascertained that most participants had considerable evaluation and commissioning experience, with several having prior experience of theory based evaluation and some knowledge of complexity. The participants were very interested in the topic, and engaged enthusiastically in small and large group discussions. Overall, the course content and delivery was rated well and the feedback was positive. We appear to have found the right 'level' of content to match the group's prior experience, but sufficiently demanding that people felt stretched. Overall aim of workshop: To provide • an understanding of complexity and the challenges that complexity can pose to the planning, delivery and use of policy evaluations. • guidance on some of the strategies, approaches and methods that an evaluator or evaluation commissioner might want to use to tackle the challenges of complexity |
| Description | CECAN Fellowship Research Project: How to improve the evaluation of complex systems to better inform policymaking Learning from evaluating Defra's Reward & Recognition Fund: |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | https://drive.google.com/file/d/0B-yPZxBK8WjPTkdrN2R4OXUyYjQ/view?usp=sharing |
| Description | CECAN Sub-Project, Meta Evaluation of Past Policy Evaluations - Collingwood Environmental Planning |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | http://www.cecan.ac.uk/news/cep-undertaking-meta-evaluation-of-past-policy-evaluations |
| Description | CECAN Syllabus CPD Workshop: "Agent-Based Modelling for Policy Analysis" (Dr Pete Barbrook-Johnson) 16th January 2019, University of Surrey, Guildford |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | This 1-Day CECAN CPD course will cover aspects of model purpose and design, the use of data for model calibration and validation, and consider how to interpret and communicate modelling results in the social sciences and policy research. Attendees will also get a chance to build a simple model themselves. By the end of this session, students will have gained: • An understanding of the basics of agent-based modelling; • An understanding of how agent-based modelling can be used to model of complex adaptive systems; • An understanding of how agent-based modelling can be used in policy analysis and evaluation. |
| URL | https://cecan.us13.list-manage.com/track/click?u=feb21724b738fd54ff25c23a2&id=0dada54e96&e=615f44fb3... |
| Description | CECAN Syllabus CPD Workshop: "Agent-based Modelling for the Social Scientist" (Dr Corinna Elsenbroich and Dr Jennifer Badham ) 4th - 6th February 2019, University of Surrey, Guildford, UK |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | This 3-Day course will cover the process of agent-based modelling, from conceptualising a research question, where to obtain data, operationalisation and formalisation of data, model implementation, and model analysis and interpretation. In addition to the theoretical content, you will learn NetLogo as a programming language for agent-based models. On the basis of a detailed model of a social phenomenon (protective behaviour when faced with an epidemic) that is developed step-by-step in lab sessions, the major features of programming in NetLogo are learned through practical application. Through this guided implementation you will acquire basic to intermediate programming skills in NetLogo as well as engaging with the step-by-step development of a model. On successful completion of this course, particpants will be able to: Understand the foundations of agent-based modelling (K); Understand application areas of agent-based modelling (C,K); Understand different implementations of social phenomena (C,K); Be able to program in NetLogo (K,T,P); Be able to provide a basic model specification and a basic implementation (P) Key: C-Cognitive/Analytical; K-Subject Knowledge; T-Transferable Skills; P- Professional/ Practical skills |
| URL | https://www.cecan.ac.uk/index.php/events/agent-based-modelling-social-scientist-4-6-february-2019 |
| Description | CECAN Syllabus CPD Workshop: "Methods for Complex Systems", Dr Corinna Elsenbroich (a three-day workshop offered to Surrey MSc students and also as a short course to externals), 11-13 March 2019, Guildford |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Complexity social science represents an important new suite of methods that moves researchers beyond the traditional reliance on quantitative or qualitative approaches. This module introduces participants to complex social systems and the challenges of complexity social science and policy research. The course introduces students to a range of methods to tackle these challenges, in particular focussing on methods to understand complex causality. Two methods are covered in particular: Qualitative Comparative Analysis (QCA) and Process Tracing/Bayesian Updating. Both methods will be introduced through real world case studies and situated within the research and policy process. |
| URL | https://www.surrey.ac.uk/events/20190311-complex-social-systems |
| Description | CECAN Syllabus CPD Workshop: 'Designing Evaluations for Complex Policy Interventions 2018', 15th May 2018 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | https://www.cecan.ac.uk/events/cecan-syllabus-cpd-workshop-designing-evaluations-for-complex-policy-... |
| Description | CECAN Syllabus CPD Workshop: 'Qualitative Comparative Analysis', Tuesday 19th June 2018, 11:00 - 15:00 at University of Surrey, Guildford, UK |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | CECAN Syllabus CPD Workshop: 'Qualitative Comparative Analysis' CECAN - Qualitative Comparative Analysis Tuesday 19th June 2018, 11:00 - 15:00 (registration 10:30) University of Surrey, Guildford, UK Course Details: This workshop is part of the CECAN Module for future policy analysts and evaluators. The syllabus was developed to support capacity building nationally and internationally in the evaluation of complex policy and programmes. This session introduces qualitative comparative analysis (QCA) - a case-based approach that relies on Boolean algebra, which crudely speaking is a way of sorting differentiating units or cases according to whether they share or do not share particular configurations of attributes. |
| URL | https://www.cecan.ac.uk/events/cecan-syllabus-cpd-workshop-qualitative-comparative-analysis |
| Description | CECAN Syllabus CPD Workshop: Evaluation Fundamentals - Choosing Appropriate Methods and Assessing Overall Quality of Evaluations (Barbara Befani, Research Fellow, University of Surrey), 19 Nov 2018 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | The course introduces participants to the fundamental aspects of the evaluation processes that precede and follow data collection, analysis, and report drafting; and that are relevant when drafting Invitations to Tender (ITT) or Terms of Reference (ToR), when evaluating proposals, and assessing the quality of intermediate and final evaluation products / reports. Intended Audience: Policy analysts, commissioners of evaluation, professional evaluators, social science postgraduate students. By the end of this session, students will: •Know how to assess the overall quality of a broad range of evaluation products, including ITT, proposals, business cases; and inception, intermediate, and final draft reports •Know which methods are most appropriate under different circumstances and how to choose and combine methods at various stages of the evaluation process, taking into account stakeholders' preferences and organisational constraints. |
| URL | https://www.cecan.ac.uk/events/cecan-syllabus-cpd-workshop-evaluation-fundamentals-choosing-appropri... |
| Description | Evaluating Complex Interventions with Process Tracing and Bayesian Updating - 2 day CPD Course at University of Surrey - 15th & 16th February 2017 - Dr Barbara Befani |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | http://www.cecan.ac.uk/events/cecan-short-course-evaluating-complex-interventions-process-tracing-ba... |
| Description | Getting To Grips With Wicked Issues Using Exploratory Qualitative Comparative Analysis (QCA) - Professor Dave Byrne - CPD Course at University of Surrey - 21st February 2017 |
| Geographic Reach | National |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | http://www.cecan.ac.uk/events/getting-to-grips-wicked-issues-using-exploratory-qualitative-comparati... |
| Description | Jeremy Phillipson is providing expert advice to Defra in relation to fisheries (From Jan 2019) |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Magenta Book Refresh |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| URL | https://www.gov.uk/government/publications/the-magenta-book |
| Description | Reference to CECAN in Defra ITT (CEF), Oct 2018 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| Description | Reference to CECAN in Food Standards Agency Areas of Research Interest 2017-2018 |
| Geographic Reach | National |
| Policy Influence Type | Citation in other policy documents |
| URL | https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/8235... |
| Description | Actively anticipating the unintended consequences on air quality of future public policies (ANTICIPATE) |
| Amount | £429,208 (GBP) |
| Funding ID | NE/T001852/1 |
| Organisation | Natural Environment Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 02/2019 |
| End | 02/2022 |
| Description | Defra Invitation to Tender: Invitation to Tender for Complexity Evaluation Framework, Tender Reference: ITT_4883 (bid via CECAN Ltd) |
| Amount | £58,304 (GBP) |
| Organisation | Department For Environment, Food And Rural Affairs (DEFRA) |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2019 |
| End | 05/2019 |
| Description | New approaches to the Participatory Steering and Evaluation of Complex Adaptive Systems (added to the existing CECAN grant ES/N012550/1 under International Collaboration for ESRC investments scheme 2018) |
| Amount | £21,947 (GBP) |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2018 |
| End | 05/2019 |
| Description | RCUK/UKRI Innovation Fellowship (Pete Barbrook-Johnson) |
| Amount | £377,843 (GBP) |
| Funding ID | ES/S000402/1 |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2018 |
| End | 01/2021 |
| Title | Complex-IT App |
| Description | CECAN has developed the Complex-IT app. Complex-IT is a beta-tested R Shiny available online and in downloadable form on GitHub. It is designed to increase access to the tools of computational social science (i.e. artificial intelligence, micro simulation, predictive analytics), which it does through its tab-driven interface. The result is a seamless and visually intuitive learning environment for data classification, visualisation, exploring simulated interventions and policy changes and data forecasting. |
| Type Of Material | Data handling & control |
| Year Produced | 2018 |
| Provided To Others? | Yes |
| Impact | In Oct 18 Brian Castellani reported having been contacted by the Dept for Work and Pensions enquiring about how to use the app. |
| URL | https://www.art-sciencefactory.com/complexit.html |
| Description | Project with HSE Exploring Complexity Appropriate Modelling Approaches (Pete Barbrook Johnson and Helen Wilkinson) |
| Organisation | Health and Safety Executive (HSE) |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Members of the CECAN Team are working with HSE on a small project exploring how complexity-appropriate modelling approaches could be useful to them. It will be ongoing throughout 2018, and the main output is a report we will write for them, as well as at least one workshop. |
| Collaborator Contribution | N/A |
| Impact | Too early at this stage |
| Start Year | 2017 |
| Company Name | CECAN LTD |
| Description | CECAN Ltd is the commercial arm of the ESRC funded Centre for Evaluating Complexity across the Nexus (CECAN) and all members of the team are members of CECAN. A unique coalition of academic experts and senior practitioners, CECAN has proven itself to be ideally positioned for bringing academic depth and breadth to the development and understanding of complexity appropriate evaluation, while maintaining a strong practitioner perspective to the work. Each member of the team has direct access to the vast expertise of the broader CECAN research group and its associated network. |
| Year Established | 2018 |
| Impact | CECAN Ltd has been successful in responding to an Invitation to Tender launched by Defra to commission a Complexity Evaluation Framework. This project is currently underway, having started in January 2019 and is due for completion in early May 2019. |
| Website | https://www.cecan.co.uk/ |
| Description | "Complexity, Power and Evidence in the UK Healthcare Sector: A Case Study of E-Health Research" - CECAN Seminar - Professor Trish Greenhalgh November 2016 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Trish began by discussing what we mean by e-health research and that research of technologies that only focuses on 'technologies which will change the world', is aligned with a vague modernist vision, where technology is meant to solve many of society's problems. Why? Because it does not consider the complex elements of peoples' everyday lives. Of course, e-health technologies have not changed the world, as highlighted by Tony Blair's 'National Programme for IT'. This £12.8billion programme was meant to improve transparency, efficiency and accountability within the health service. It didn't happen and this became a paradigm case of inflated hopes and underestimated challenges in e-health. Trish was involved in evaluating the programme and fears that Jeremy Hunt is now going down a similar route, with a recent headline of '£4billion NHS tech investment', suggesting the same lack of attention is being paid to the system elements in evaluation. Trish described the research she has been carrying out into 'smart homes' including e-health technologies for health, wellness and assisted living. She was quick to point out that in reality nobody lives in a smart home or has room for social assistance robots and the biggest determinant of an individual's health remains their social position, with poverty, poor housing, unsafe streets, ethnicity/culture, low health literacy, social exclusion and digital exclusion being additional contributory factors. So, the vision of e-health does not match up with reality. Through the case study of the 'Whole Systems Demonstrator Programme', a multi-million pound randomised control trial of telehealth and telecare, Trish explained how evaluations of such trials were biased because they could never capture the 'whole' system of influences on individuals' lives. She also explained that Government were quick to use very early data in order to 'demonstrate' the positive outcomes of the programme before the scientists had finished evaluating and analysing. Scientists eventually produced a very different picture to that which had been made public highlighting the struggle of conducting randomised control trials that are politically driven. Trish suggests that RCT evidence is of limited value in e-health research, because 'today's trial relates to yesterday's technology'. Technologies are becoming cheaper and faster all the time. Also, neither people nor technologies can be meaningfully studied in isolation from the socio-technical system in which they are embedded. She also discussed the importance of case studies in social research, suggesting a social science discipline that lacks in-depth case studies is impoverished because we need detailed narratives of what happened and why. Trish talked us through various models of evaluation including those of Professor Saville Kushner and Professor Barry Macdonald, and suggested we need more democratic (deliberative) evaluation, where we critically question what is going on, who is not being heard, where are the hidden agendas. Despite the fact that funding panels seem intent on recruiting more RCTs, Trish suggests RCTs reduce complex questions to over simplified scientific questions, particularly the question of 'what works.' Trish summarised her seminar by suggesting that: E-health research is dominated by a naïve, modernist mindset that envisages a rational technological utopia. The RCT, borrowed from drug experiments and promising simple answers about 'what works', aligns with this vision and is strongly favoured by funders. E-health programmes are complex, multifaceted and politically situated, they fail chiefly because no one fully understands what's going on. In depth case studies offer rich understanding but such approaches are misunderstood. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/news/cecan-seminar-complexity-power-and-evidence-uk-healthcare-sector-a-case-... |
| Description | 1.5 degrees conference in Oxford - 20/22 Sept |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Candice Howarth Discussing / Introducing CECAN at 1.5 degrees conference in Oxford - 20/22 Sept |
| Year(s) Of Engagement Activity | 2016 |
| Description | 16-18 November 2016 Title: EA Conference "Mastering 'Grand Challenges' - what can be the contribution of technology assessment?" Summary: conference on technology assessment w/ a strong focus on "Grand Challenges" and covering various areas of research, e.g. demographic change, digitilisation, energy, mobility, lifesciences, health and food Type of activity: conference w/ podium, sessions |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | 16-18 November 2016 Title: EA Conference "Mastering 'Grand Challenges' - what can be the contribution of technology assessment?" Summary: conference on technology assessment w/ a strong focus on "Grand Challenges" and covering various areas of research, e.g. demographic change, digitilisation, energy, mobility, lifesciences, health and food Type of activity: conference w/ podium, sessions - Professor Petra Ahrweiler on behalf of CECAN |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.ea-aw.de/nta7-2016 |
| Description | 18th July 2016, CECAN Seminar Series #2. "Policy Making Using Modelling in a Complex World" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The consequences of complexity in the real world are discussed together with some meaningful ways of understanding and managing such situations. The implications of such complexity are that many social systems are fundamentally unpredictable by nature, especially when in the presence of structural change (transitions). This implies consequences for the way we model, but also for the way models are used in the policy process. We discuss the problems arising from a too narrow focus on quantification in managing complex systems, in particular those of optimisation. We criticise some of the approaches that ignore these difficulties and pretend to approximately forecast using the impact of policy options using over-simple models. However, lack of predictability does not automatically imply a lack of managerial possibilities. We will discuss how some insights and tools from "Complexity Science" can help with such management. Managing complex systems requires a good understanding of the dynamics of the system in question - to know, before they occur, some of the real possibilities that might occur and be ready so they can be reacted to as responsively as possible. Agent based simulation will be discussed as a tool that is suitable for this task, especially in conjunction with model-informed data visualisation. 40 people attended this seminar which influenced their thinking on a new methodology and sparked questions, and debate amongst practitioners. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/events/18th-july-2016-cecan-seminar-series-2-policy-making-using-modelling-a-... |
| Description | 23 Jan Professor Robert MacKay attended Royal Society Meet the Media to promote CECAN |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | 23 Jan Professor Robert MacKay attended Royal Society Meet the Media to promote CECAN |
| Year(s) Of Engagement Activity | 2017 |
| Description | Attended UK Evaluation Society (UKES) conference over 2 days |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN attended UKES conference over 2 days, engaged with approx 100 people from the evaluation community. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.evaluation.org.uk/index.php/events-courses/ukes-conference/232-ukes-2017-annual-evaluati... |
| Description | BEIS - Contracts for Difference - CECAN acting as a "critical friend" |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN team members (PBJ, MB, RM) have been providing expert input in the current evaluation work being conducted by BEIS and contractors on the Contracts for Difference policy. Operating as a 'critical friend' in this case study, this has involved attending presentations made by contractors to BEIS, and commenting on draft evaluation reports |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.gov.uk/government/publications/contracts-for-difference/contract-for-difference |
| Description | British Academy/Government Social Research event |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | British Academy/Government Social Research event on Social sciences and humanities in policy making, London, 13 April: Invited participant |
| Year(s) Of Engagement Activity | 2016 |
| Description | CECAN (Adam Hejnowicz and Pete Barbrook-Johnson) alongside the James Hutton Institute led two sessions on 'nexus governance' at the 2018 Royal Geographical Society's annual conference in Cardiff (RGS-IBG Annual International Conference 2018), 28 - 31 August 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | "Session title: Adaptive management and governance of the food-energy-water-environment Nexus (1): Speed Talks Session organisers: Kirsty Blackstock (James Hutton Institute, UK), Pete Barbrook-Johnson (University of Surrey, UK), Adam Hejnowicz (University of York, UK) Session chair: Kirsty Blackstock (James Hutton Institute, UK) Session authors, presenters and titles: 1. Adaptive management and governance of the Water-Energy-Food Nexus, Sally J. Watson (Mott MacDonald) (presenter) 2. Understanding the potential of innovative public-private-partnerships at the Nexus, Pete Barbrook-Johnson (University of Surrey, UK) (presenter) 3. Power, democracy and poly-centric governance in water management: 'social science into practice' in the Environment Agency, Anna Lorentzon (Environment Agency, UK) (presenter) 4. Decentralising the Governance of the Nexus: the challenges of politics and power in finding workable solutions to a democratic management of the landscape, Paddy Abbott (LTS International, UK) (presenter), Melvin Woodhouse (Independent) 5. Who is responsible for integration? Responsibility in WEEF nexus governance, Kirsty Blackstock (James Hutton Institute, UK) (presenter), Alba Juarez-Bourke (James Hutton Institute, UK), Jessica Maxwell (James Hutton Institute, UK), Katrin Prager (James Hutton Institute / University of Aberdeen, UK), Sophie Tindale (James Hutton Institute / Durham University, UK), Kerry Waylen (James Hutton Institute, UK) 6. 'Operationalising an ecosystem approach for NEXUS governance: The case of Natural Resources Wales', Nick Kirsop-Taylor (University of Surrey / University of Exeter, UK) (presenter), Adam Hejnowicz (University of York, UK), Karen Scott (University of Exeter, UK) 7. Landscaping Ecosystem Services - A Landscape-Nexus Approach: Meaning, Narrative and Integration, Adam Hejnowicz (University of York, UK) (presenter), Sue E. Hartley (University of York, UK), Jeremy Phillipson (Newcastle University, UK), Frances Rowe (Newcastle University, UK), Murray A. Rudd (World Maritime University, Sweden), Piran CL White (University of York, UK) 8. Multi-aspectual evaluation: an integrative approach to Nexus policies, Richard Gunton (Sciteb Ltd / CECAN, UK) (presenter), Ian Christie (University of Surrey, UK), Adam Hejnowicz (University of York, UK), Sue E. Hartley (University of York, UK) 9. Governing the Water-Energy-Food Nexus: The Role of Blockchain Technologies, Katrien Steenmans (Kings College London, UK) (presenter), Phillip Taylor (University of Warwick, UK), Ine Steenmans (University College London, UK) 10. The Role of Quantifying Material Flows in the Governance of the Water-Energy-Food Nexus, Thomas Voelker (Joint Research Centre), Kerry Waylen (James Hutton Institute, UK) (presenter), Zora Kovacic (Universitat Autonoma de Barcelona, Spain), Roger Strand (University of Bergen, Norway), Jan Sindt (Climate Analytics) " AND "Session title: Adaptive management and governance of the food-energy-water-environment Nexus (2): Workshop Session organisers: Kirsty Blackstock (James Hutton Institute, UK), Pete Barbrook-Johnson (University of Surrey, UK), Adam Hejnowicz (University of York, UK) Session chair: Adam Hejnowicz (University of York, UK) |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.rgs.org/research/annual-international-conference/history/ |
| Description | CECAN / BEIS workshop (Energy Innovation Programme (EIP) Case Study with BEIS), 25 April 2018 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | On April 25, BEIS and CECAN held a workshop bringing together members of the CECAN team with analysis and policy leads in BEIS, responsible for the £505million Energy Innovation Programme (EIP). The workshop explored options for the development of an evaluation strategy for the programme; it is the first activity in the ongoing support CECAN will offer in this key policy area. The programme is made of many projects with diverse aims and contexts, and has key objectives around energy innovation, meaning there are many evaluation challenges. |
| Year(s) Of Engagement Activity | 2018 |
| Description | CECAN Annual Conference 2018 - "Policy Evaluation for a Complex World: The Challenges that Complexity Poses for Policy - Solutions and Benefits", One Birdcage Walk, Westminster, London, 11th July 2018 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Annual Conference 2018 - Policy Evaluation for a Complex World: The Challenges that Complexity Poses for Policy - Solutions and Benefits One Birdcage Walk, Westminster, London, SW1H 9JJ 11th July 2018, 9.30am to 3.45pm The world is complex: it is made up of many different components which interact in messy, adaptive and unpredictable ways. Complexity poses great challenges to policy-making and evaluation because complex systems are fundamentally unpredictable. Ignoring it brings the risk of making costly inadequate decisions and policies failing unexpectedly. Understanding and embracing complexity means exploring uncertainty with rigour and the possibility of effective policy-making. Our keynote speaker Matthew Taylor, Chief Executive, RSA will talk about the challenges of complexity for government. Two panel sessions will discuss how well Government is dealing with complexity in policy making and evaluation, and what needs to happen to build capacity and enable better management of complexity in future. Panel members will include Penny Hawkins (Independent Evaluation Consultant), Gary Kass (Deputy Chief Scientist - Natural England) and Charlotte Allen, (Head of Central Social Research - DEFRA), Siobhan Campbell (Head of the Central Research Team, Deputy Chief Scientific Advisor, Department for Transport), Dione Hills (Principal Researcher - The Tavistock Institute), Jonas Schönefeld (Technische Universität Darmstadt/UEA). Concluding comments will come from Professor Eliott Stern and CECAN's Director, Professor Nigel Gilbert. CECAN's conference will be of value for all those involved in policy development, delivery and evaluation We registered 230 people and 77% of those registered were from the public sector, 15% from the private sector and 8% from academia, excluding the CECAN team |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.eventbrite.co.uk/e/policy-evaluation-for-a-complex-world-tickets-44742300425 |
| Description | CECAN Blogs (active and ongoing - regularly updated) |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Members of the CECAN Team, CECAN Fellows, CECAN Partners and others regularly contribute to the CECAN Blog, which can be found on the CECAN website. Posts to date include: Complexity and Evaluative Thinking (21/01/2019 - 09:44) What has economic pluralism got to do with understanding complexity? (17/12/2018 - 09:40) The Story of the Pluralistic Evaluation Framework (19/11/2018 - 10:55) Power and Hope: Using Evaluation in Volatile Contexts (24/10/2018 - 14:27) The Nexus: A New Approach to Sustainability Transformations - What, Why and How (12/09/2018 - 11:38) CECAN Fellowship Blog: Evaluative methodologies for the transition towards an institutional ecosystem approach 'system-cultures' within Natural Resources Wales and CECAN (27/07/2018 - 13:22) A Pluralistic Evaluation Framework (11/06/2018 - 21:06) The Defiance of Global Commitment (15/05/2018 - 21:36) Looking Through a Glass Darkly: Public Goods and Agricultural Policy(09/03/2018 - 14:38) Humility and courage: strategy and evidence in our complex world (27/02/2018 - 20:25) Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes (22/02/2018 - 22:20) Finding The Common Ground (21/02/2018 - 17:46) Carillion may have collapsed, but public-private partnerships can be so much more... (08/02/2018 - 18:07) Teaching Evaluation of Complex Policy and Programmes (12/01/2018 - 16:29) Putting Values Back in Evaluation. (03/01/2018 - 15:30) Building Collaborative Narratives and Developing Trust in Policy and its Evaluation. (13/12/2017 - 20:13) An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!(15/11/2017 - 12:26) Travels in Evaluation - Systematising the Complex, Enjoying the Journey.(15/11/2017 - 12:20) CECAN's 'The Visual Representation of Complex Systems' at Environment, Economy, Democracy: Flourishing Together #RSD6.(13/11/2017 - 15:51) A Whole New World - What Next? (12/09/2017 - 16:38) Complexity Settings to the Rescue: A New Lease of Life for Evidence-Based Policy?(28/08/2017 - 21:14) Models as 'Interested Amateurs' (09/08/2017 - 10:33) Taking The Temperature of Trust - Henry Leveson Gower (03/07/2017 - 20:44) One Researcher's Anecdote is Another Researcher's Data (09/03/2017 - 11:44) That Way Lies Prosperity: Sustainability and the Nexus(07/02/2017 - 17:24) Agriculture Fit For A Complex World (16/11/2016 - 11:28) Why We Need Network Analysis to Understand the Future of Economics(08/11/2016 - 13:59) CECAN Complexity in Evaluation Workshop: What we Did and What we Learned (01/11/2016 - 13:35) Why Carry Out Economic Evaluation? (01/11/2016 - 09:49) Access to Data is Crucial(25/10/2016 - 15:42) Aligning Policy and Evidence for the Age of Complexity(24/10/2016 - 15:43) Complexity High on the Agenda at the EES 2016 Biannual Conference(11/10/2016 - 14:20) Likelihoods (06/10/2016 - 14:02) Is Policy Evaluation Fit For Purpose? (26/09/2016 - 13:51) After the Brexit vote: What Next for the UK's Environment? (29/06/2016 - 13:51) How Should Academics Interact with Policy Makers? Lessons on Building a Long-term Advocacy Strategy.(22/06/2016 - 12:09) Reflections on Language and Complexity (22/06/2016 - 11:59) Should Academics be Expected to Change Policy? Six Reasons Why it is Unrealistic for Research to Drive Policy Change.(26/05/2016 - 10:03) The Science of Using Research... (24/05/2016 - 21:52) Clearing The Fog(04/05/2016 - 14:44) Global Challenges Require Cross-Cutting Solutions(04/04/2016 - 16:35) Tools, Tools Everywhere and not a Hammer in Sight!(24/03/2016 - 18:48) |
| Year(s) Of Engagement Activity | 2016,2017,2018,2019 |
| URL | https://www.cecan.ac.uk/blog |
| Description | CECAN Fellow Richard Gunton gave a lecture at the Valuing Nature Keynote Lecture Series "Beyond Ecosystem Services and Natural Capital: Valuing the Invaluable." 3rd July 2017 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN Fellow Richard Gunton gave a lecture at the Valuing Nature Keynote Lecture Series "Beyond Ecosystem Services and Natural Capital: Valuing the Invaluable." 3rd July. The 45 minute lecture was followed by responses from the following Panelists: Michael Winter (Chair) - Professor of Land Economy and Society, Land, Environment, Economics and Policy Institute (University of Exeter). Member of Valuing Nature Programme Coordination Team Professor Emily Brady - Professor of Environment and Philosophy, The University of Edinburgh. Guy Duke Member of Valuing Nature Programme Coordination Team and Deputy Chair of the Joint Nature Conservation Committee. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://valuing-nature.net/event/valuing-nature-keynote-lecture-series-valuing-invaluable-edinburgh |
| Description | CECAN International Symposium |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | As part of the ESRC Festival of Research, CECAN proudly hosted an International Symposium on complexity approaches to evaluate global nexus policy challenges. Delegates, including representatives from the European Commission and UK Government, came together to share expertise and join the debate. We were thrilled to connect with leaders and influencers in the evaluation community, from ten different countries; Germany, USA, Switzerland, Australia, Senegal, Netherlands, South Africa, Belgium, Ireland and Italy. Over the course of this two-day event topic areas included; The need for complexity sensitive evaluation, exploring complex thinking for sustainable futures, cutting edge evaluation methods and how to increase the uptake of them, and how to get government to deal with complexity. Knowledge sharing on evaluation methods included; Qualitative comparative analysis, incorporation of agent based modelling into case based methods, and application of complexity science in assessing and evaluating policy. The symposium was seamlessly facilitated by Paul Brand from Risk Solutions, and delivered through a combination of panel sessions, presentations and interactive breakout groups with valuable Q&A time made available for further discussion amongst delegates. Some of the key messages that have emerged from the CECAN International Symposium are; To understand our influence and impact on the complex world around us, we need to use the very best in evaluation practice. We need to manage complex systems and evaluations adaptively. Expect change! Complexity requires us to be humbler about our perspective and influence. On first appearances, complexity doesn't make a strong sales pitch for how to make a policy-maker or programme manager's life easier. But without an appreciation of complexity, interventions and evaluations may fail to achieve their goals - ignore it at your peril. Finally, appreciating and managing complexity requires interdisciplinarity, bringing diverse disciplines and perspectives together. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/news/cecan-international-symposium-on-complexity-approaches-to-evaluate-glob... |
| Description | CECAN International Workshop: New Approaches to the Participatory Steering and Evaluation of Complex Adaptive Systems Barnett Hill Hotel, Surrey, 12-14 Feb 2019 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN International Workshop: New Approaches to the Participatory Steering and Evaluation of Complex Adaptive Systems Barnett Hill Hotel, Surrey This workshop is a first step towards building an international community of those developing innovative, complexity-appropriate aids to participatory evaluation, to help people understand and steer their own complex systems and evaluate the effects of policies or other interventions. Our focus will be on exploring and synthesising new approaches to collective understanding, experiencing and interacting with socio-ecological-technical systems in which numerous scales of dynamics and complexity are at play. We will examine the value of such approaches and how they could be used in collaborative system design and co-management and policy evaluation and appraisal, in particular to allow ordinary people, rather than simply policy makers or experts, to understand, steer and evaluate change in their own complex social systems. This will be a highly interactive and focused small workshop in which we will share, generate and kick start ideas and collaborations and all attendees will be expected to participate fully. Contributors include: Petra Ahrweiler, Johannes Gutenberg University Mainz, Germany; Luis Enrique García Barrios, College of the South Frontier, Mexico; Nigel Gilbert, University of Surrey, UK; Mizuki Oka, University of Tsukuba; Alex Penn, University of Surrey, UK; Pascal Perez, SMART Infrastructure Facility, University of Wollongong, Australia; Alex Smajgl, Deakin University, Australia; Sha Xin Wei and Brandon Mechtley, Synthesis Center, Arizona State University, USA Resulting impacts include: 7 International Topic Groups have formed to develop ideas and discussions further into Research Project Proposals and papers. An international collaborative network is being maintained with more events and continuing work planned. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-workshop-new-approaches-to-participatory-steering-and-evaluatio... |
| Description | CECAN Newsletter August 2017 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter sent to c. 1100 people on the CECAN Mailing list |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://mailchi.mp/795f6f49f684/all-the-latest-news-from-cecan?e=6e12331a14 |
| Description | CECAN Newsletter Autumn 2016 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E-newsletter sent to 984 recipients national and international. Topics included: Report/film from the CECAN launch event in London, CECAN progress report, Fellowship scheme update, CECAN blog list, Introducing new members of the CECAN team, links to new EPPN publications, report on September complexity residential workshop, CECAN animation, films of recent seminars, promotion of upcoming CPD courses. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://eepurl.com/clsaAP |
| Description | CECAN Newsletter December 2017 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter sent to c 1100 people on te CECAN mailing list |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://mailchi.mp/8b011f69b936/happy-christmas-from-cecan-our-final-newsletter-of-2017 |
| Description | CECAN Newsletter December 2018 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter reaching over 1100 people on the CECAN mailing list |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://mailchi.mp/f0ced3762882/46i5hbo49y-1874233 |
| Description | CECAN Newsletter February 2017 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E-newsletter reaching 1027 people on our mailing list. Topics included: Promotion of joint CECAN/ Nexus Network/CUSP event 'Sustainability in Turbulent Times' - March 2017, Film of James Wilsdon seminar, advertising new fellowship opportunity with University of York, advertising CPD courses, advertising two new CECAN seminars in specific methods/tools for evaluation, introducing new members of the CECAN team, |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://eepurl.com/czQu5P |
| Description | CECAN Newsletter June 2016 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E-Newsletter informing our growing database of policy makers, analysts, scientists, academics and professionals about the work of CECAN. Topics included: Who we are, Fellowship Scheme, Launch Event, Complexity Workshop, Opportunities to Join CECAN, Reading Group, Seminar Series. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://eepurl.com/b2YUAT |
| Description | CECAN Newsletter June 2018 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter sent to c 1100 people on the CECAN mailing list |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://mailchi.mp/eb574cf46cf8/news-update-from-cecan |
| Description | CECAN Newsletter March 2018 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter sent to c 1100 people on te CECAN mailing list |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://mailchi.mp/b41c0641bf8d/all-the-latest-news-from-cecan?e=615f44fb38 |
| Description | CECAN Newsletter May 2017 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter reaching over 1100 people on the CECAN mailing list |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://us13.campaign-archive.com/?u=feb21724b738fd54ff25c23a2&id=d871bf96b8 |
| Description | CECAN Newsletter November 2017 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter sent to c 1100 people on CECAN mailing list |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://mailchi.mp/183fa97a4b45/all-the-latest-news-from-cecan |
| Description | CECAN Newsletter September 2018 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | E Newsletter reaching over 1100 people on the CECAN mailing list |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://mailchi.mp/df3499dd65e6/46i5hbo49y-1787781 |
| Description | CECAN Project kick off meeting 10th & 11th March 2016 - 2 day residential workshop |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN had its official project launch meeting in March 2016 at the beautiful Barnett Hill Conference Centre, in the heart of the Surrey hills. We had a fantastic attendance of team members and co-funders, getting to know one another and sharing ideas around how the CECAN Centre can best operate to achieve its short and long term objectives. Through a mixture of plenary and break-out sessions, we discussed the difficulties currently faced by policymakers in evaluating complex social policies, explored the various tools and methodologies CECAN will use to try and overcome these difficulties, and came up with a range of 'real world' Nexus based problems that our co-funders wish us to work upon. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/events/10th-11th-march-2016-cecan-kick-off-meeting |
| Description | CECAN Reading Group #1 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Reading Group #1 - "Complexity theory in applied policy worldwide" by Yaneer Bar-Yam. Chapter 11 in the book Modeling Complex Systems for Public Policies/Editors: Bernardo Alves Furtado, Patricia A.M Sakowski, Marina H Tovolli - Brasilia: IPEA, 2015. CECAN team members enhance their own knowledge and skills by reading a publication each month and then getting together to discuss it by means of virtual meeting. |
| Year(s) Of Engagement Activity | 2016 |
| Description | CECAN Reading Group #2 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | "Adding It Up: Improving Analysis and Modelling in Central Government" by the Performance and Innovation Unit (PIU), Cabinet Office, 2000. |
| Year(s) Of Engagement Activity | 2016 |
| Description | CECAN Seminar - The Spectrum of Overwhelming Systems, with Professor Claes Andersson of Chalmers University of Technology, hosted at BEIS, 1 Victoria Street, London, 16th May 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | In a 2014 paper in Futures we explored the relatively straightforward proposition that societal systems combine two qualities that are commonly referred to as complexity and complicatedness. Why do societal systems remain recalcitrant despite the fact that we have powerful approaches for dealing with both these qualities. We claim that the combination between complexity and complicatedness is emergent: fundamentally and irreducibly different from either quality in isolation. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/cecan-seminar-claes-andersson-spectrum-of-overwhelming-systems |
| Description | CECAN Seminar - Applying Process Tracing Within a Realist Evaluation, Adam Stiles, 5th September 2018, hosted at BEIS, 1 Victoria Street, London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | This seminar, led by Adam Stiles, will examine the application of process tracing methods within a realist evaluation. The evaluation of the 'Transitional Arrangements (TA) for Demand Side Response' scheme was commissioned by the Department of Business, Energy and Industrial Strategy and has been led by CAG Consultants in partnership with Charles Michaelis (CECAN). Our approach to process tracing has been informed by the work of Dr. Barbara Befani (CECAN). 53 people registered for this event, 35 attended and it has subsequently been viewed online more than 200 times |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.eventbrite.co.uk/e/cecan-seminar-applying-process-tracing-within-a-realist-evaluation-ti... |
| Description | CECAN Seminar - How evaluation in the SDG era can mislead, and what to do about it |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar - How evaluation in the SDG era can mislead, and what to do about it Tuesday 27th February 2018, 12:45 - 14:00 - BEIS, 1 Victoria Street, London, SW1H 0ET About the seminar: Professionals in evaluation pay a lot of attention to how to evaluate. We are less thoughtful about exactly what to evaluate, and why. Stakeholders' interests and questions tend to determine what will be useful to focus on at a particular point in time. Or we use a predetermined set of criteria, usually the so-called DAC criteria - relevance, efficiency, effectiveness, impact and sustainability - to direct and focus evaluations. Yet this widely accepted practice can prompt us to neglect important issues that should be on our agenda if we are serious about the role of evaluation in supporting sustainable development, and understanding whether we are on track to achieve the SDGs. Failure to focus our evaluations appropriately can lull us into dangerous complacency about accomplishment and success. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://cecan.ac.uk/events/cecan-seminar-how-evaluation-sdg-era-can-mislead-and-what-to-do-about-it |
| Description | CECAN Seminar - Mixed Methods Evaluation, using QCA and NVIVO with Wendy Olsen, 20th April 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar with Professor Wendy Olsen, University of Manchester. 20th April 2018. BEIS, 1 Victoria Street, London. 12.45pm-2pm. Theorising Bangladesh Indebtedness and Using Mixed-Methods Evaluation, Taking Into Account Gender/Class Realities: Using QCA and NVIVO In development studies, one wants an evaluation team to face up to the challenge of combining surveys with semi-structured interview data. In this seminar I explain and show how these linkages are made with concrete data about villages in Bangladesh and India. The problem is that complex data can overwhelm the interpreting team. The solutions I offer here are methodologically sound. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.eventbrite.co.uk/e/cecan-seminar-on-mixed-methods-evaluation-using-qca-and-nvivo-tickets... |
| Description | CECAN Seminar - Professor David Hunter, Durham University "Understanding Health Policy in the Third Era Through a Complex Systems Lens" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar - 12th September 2017 - Professor David Hunter, Durham University BEIS, 1 Victoria Street, London 12.45pm - 2pm "Understanding Health Policy in the Third Era Through a Complex Systems Lens" Health policy is complex, arguably becoming more so. What have been termed 'wicked issues' are a central feature of health policy where there are often no simple or single solutions, if any at all. This is especially evident in regard to public health issues including obesity, alcohol and substance misuse, health inequalities. But health systems more generally are undergoing major transformation in response to changes in epidemiology, aging populations, lifestyle-related illnesses, multi-morbidities, and so on. We have entered the third era of thinking about health policy which has introduced new challenges and complexities. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://cecan.ac.uk/events/cecan-seminar-understanding-health-policy-third-era-through-a-complex-sys... |
| Description | CECAN Seminar - What works at the Nexus? - Professor James Wilsdon. 25th January 2017 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The CECAN team would like to extend thanks to Professor James Wilsdon from the University of Sheffield for his thought provoking discussions on 'What works at the Nexus' on 25th January 2017. Drawing on insights from his role as director of the ESRC's Nexus Network, and vice-chair of the International Network for Government Science Advice (INGSA), James Wilsdon reflected on how the landscape for transdisciplinary, policy-engaged research is changing, and what opportunities may emerge through the Global Challenges Research Fund, reforms to the Research Excellence Framework and the creation of UK Research and Innovation. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/news/what-works-at-nexus-a-thought-provoking-seminar-from-professor-james-wil... |
| Description | CECAN Seminar Professor Brian Castellani of Kent State University - COMPLEX-IT and the SACS Toolkit |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Seminar with Professor Brian Castellani of Kent State University. given at BEIS to an audience of 50 people, mainly government policy makers, evaluators, analysts, researchers. Addressing Complexity in Nexus Issues: A case based approach to evaluation research. Drawing upon two recent studies - one on health trajectories and the other on grid reliability - Brian will demonstrate how evaluation researchers can use case-based complexity to more effectively model nexus issues across time/space. More specifically, seminar participants will use these two case studies to explore a new Case-based Complexity App that Brian and colleagues have developed, which researchers can use to: treat the variables in a database (big-data or otherwise) as a nexus of complex and inter-dependent factors; use this nexus of factors to construct a nexus profile for each and every case in a database - which can be cross-sectional or longitudinal; cluster these nexus profiles into the major and minor clusters or trends across some defined framework of evaluated time/space; use these clusters/trends to explore how a nexus issue manifests itself as an array of multiple trajectories and divergent outcomes; and finally, use these results to improve the differential utility or impact of a policy or public service. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-seminar-addressing-complexity-nexus-issues-a-case-based-approach... |
| Description | CECAN Seminar Series: NEKS Ltd Agent Based Modelling Training Course: What is the big deal for policy? |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar Series: NEKS Ltd Agent Based Modelling Training Course: What is the big deal for policy? Monday 11th September 2017 Location: The School of Economic Science, 11-13 Mandeville Place, Marylebone, London, W1U 3AJ Time: 10am - 4pm Agent Based Modelling (ABM) is a key new technique for exploring complex economic problems. It can represent complex interactions between agents over time under different scenarios. It can be used to better understand and mitigate risks and identify and accentuate opportunities for policy. It is a key tool recommended by people such as Andy Haldane, Chief Economist of the Bank of England and Richard Bookstabber, former Securities and Exchange Commission regulator and Treasury department adviser. Current users include government depts and agencies, banks, major retailers and the military amongst others. This course would allow you to better understand how ABMs are best used to inform policy and regulatory strategies. It is given by people who have both built and used ABMs. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://cecan.ac.uk/events/neks-ltd-agent-based-modelling-training-course-what-big-deal-for-policy |
| Description | CECAN Seminar with Clare Twigger Ross and Owen White from Collingwood Environmental Planning (CEP) "Learning lessons from practical policy evaluation: reflecting on a meta-evaluation of UK/EU policy and practice evaluations across the nexus" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CEP has been carrying out these types of evaluations over the past decade and as part of the CECAN centre took time to analyse 23 of their evaluations to learn lessons specifically around the evaluation of complexity, the role of methods and the nature of evaluation impact. This brought out the key factors that affect evaluation progress and enable some understanding of how to navigate an evaluation through dynamic policy landscapes so as to provide value and insight. In this seminar Dr Clare Twigger-Ross and Owen White from CEP will present the findings from that research and facilitate a discussion around the key findings and approaches for managing complexity in evaluations. The seminar is aimed at those involved in policy evaluation specifically with respect to the natural environment, consultants, academics, policy analysts and policy makers. given at BEIS to an audience of 50 people, mainly government policy makers, evaluators, analysts, researchers. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/events/cecan-seminar-learning-lessons-from-practical-policy-evaluation-refle... |
| Description | CECAN Seminar with Dorothy Lucks: "Evaluating Sustainability - Is It Possible? A Case for Focusing on Evaluating Decision-Making for Sustainable Development." |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar: "Evaluating Sustainability - Is It Possible? A Case for Focusing on Evaluating Decision-Making for Sustainable Development." 8th November 2017, 12.45 - 2pm, Food Standards Agency (FSA), 125 Kingsway, London, WC2B 6NH You are warmly invited to participate in a CECAN Seminar with Dr Dorothy Lucks, Director of Sustainable Development Facilitation (SDF Global Pty Ltd) About the Seminar: The complexity of sustainability is immense, yet many policies and plans now have sustainability as an aim. While it is difficult to empirically evaluate sustainability outcomes, it is possible to evaluate the process of decision-making to identify if decision-making is becoming more informed and working towards balancing social, economic and environmental priorities. Examples from doctoral research of two case studies, one in Australia, one in the Philippines on decision-making processes. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://cecan.ac.uk/events/cecan-seminar-evaluating-sustainability-it-possible-a-case-for-focusing-o... |
| Description | CECAN Seminar with Helen Wilkinson of Risk Solutions, and Harry Walton of Environment Agency: The use of dependency modelling to support evaluation in complex environments - a case study application. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Understanding and quantifying the value of flood risk management activities. The 'theory of change' evaluation strategy (described in the Magenta book) was developed to help tackle evaluation of complex multi-level, multi-intervention initiatives. Elements of theory of change mapping are now widely used to both plan and evaluate new initiatives. However, the approach has been criticized for being too 'linear' to effectively reflect the complexity inherent in many programmes or situations. Dependency modelling provides one way of meeting this challenge. Risk Solutions worked with the Environment Agency to develop and quantify a dependency model to evaluate the contribution of the different types of flood risk management activity the Environment Agency engage in on the achievement of outcomes. As well as providing, for the first time, a clear comparison of the value of 'softer' activities (such as stakeholder engagement around community resilience) with capital investment in flood defences, the model enabled the Environment Agency to demonstrate the potential impact of changes in activities on flood risk outcomes. The work therefore effectively bridged the gap between evaluation and the strategic planning of activities in complex, changing environments. Its greatest value lay in the deeper understanding of how the FCRM system works that was generated by undertaking the analysis, rather than in the results themselves. In this seminar Dr Helen Wilkinson (Risk Solutions) and Harry Walton (Environment Agency) will describe the construction of the model and its use to support evaluation. They will describe the strengths and weaknesses of the approach, and facilitate discussion of how the approach could deliver value in other applications. The seminar is aimed at those involved in policy evaluation, specifically with respect to the natural environment, including consultants, academics, policy analysts and policy makers. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-seminar-use-of-dependency-modelling-to-support-evaluation-comple... |
| Description | CECAN Seminar with Henry Leveson-Gower: "Can We Have Our Cake and Eat It Post-Brexit? Effective Regulation Without Redtape?" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar: "Can We Have Our Cake and Eat It Post-Brexit? Effective Regulation Without Redtape?" Wednesday 6th December 2017, 12.45 - 2pm, BEIS, 1 Victoria Street, London You are warmly invited to participate in a CECAN Seminar with Henry Leveson-Gower, CECAN Research Fellow. About the Seminar: New methodological perspectives, from economics and other disciplines, and research into practice are suggesting more effective approaches to regulation that could also avoid escalatory pressures towards red tape. This seems to require us to rethink dichotomies between tough and light touch regulation. Henry Leveson-Gower has looked at the case of the Water Industry - in particular the role and prevalence of trust - through interviews and a survey as part of a research fellowship at CECAN. He will present his results and their potential implications for future regulation. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://cecan.ac.uk/events/cecan-seminar-can-we-have-our-cake-and-eat-it-post-brexit-effective-regul... |
| Description | CECAN Seminar with Jan Kwakkel of University of Delft. "Supporting decision making under deep uncertainty: on adaptive plans and exploratory modeling." |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar with Jan Kwakkel of University of Delft. "Supporting decision making under deep uncertainty: on adaptive plans and exploratory modeling." Decision making on complex systems requires coming to grips with irreducible uncertainty. In the literature, there is an emerging consensus that any decision regarding a complex system should be robust with respect to the various uncertainties. A plan is robust if its expected performance is only weakly affected by deep uncertainty. Over the last decade a new decision support paradigm, known as 'decision-making under deep uncertainty', has emerged that aims to support the development of robust plans. This paradigm rests on three key ideas. The first idea is exploratory modeling. In the face of deep uncertainty, one should explore the consequences of the various presently irreducible uncertainties for decision-making. Typically, in the case of complex systems this involves the use of computational scenario approaches. The second idea is adaptive planning. Decision robustness is to be achieved through plans that are designed from the outset to be adapted over time in response to how the future actually unfolds. In this way, adaptation over time is planned for, rather than taking place in an ad hoc manner. The third idea is decision support. Decision analysis should move away from trying to dictate the right choice, and instead aim at enabling joint sense making amongst the various parties to decision. In this talk, I will introduce the idea of decision making under deep uncertainty, explain the basic idea of adaptive policy making and adaptation pathways, briefly discuss how exploratory modeling can be used to support the design of such adaptive plans, and illustrate this with examples drawn from a variety of application domains including transport and climate adaptation. given at BEIS to an audience of 50 people, mainly government policy makers, evaluators, analysts, researchers. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-seminar-supporting-decision-making-under-deep-uncertainty-on-ada... |
| Description | CECAN Seminar with Kerstin Junge and Richard Allen, of the Tavistock Institute "An Introduction to Theory of Change" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | This seminar was given at BEIS to an audience of c. 50 people, mainly government policy makers, evaluators, analysts, researchers. Theory of Change (TOC) is not a novel evaluation tool in itself. As a member of a family of theory based approaches to outcome evaluation it has become a key approach to assessing the outcomes of complex interventions where implementation strands are multiple, causal paths are not straightforward and feedback loops do not lend themselves to a linear cause-effect analysis. Yet, all too often we seem to get 'stuck', visualising theories of change in linear models even in the space of complexity. Greater clarity of purpose of theory of change for complex interventions appears to be needed. Against this background, this seminar will explore different purposes and uses of theories of change in complex interventions and hence approaches to constructing and visualising them. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/index.php/news/cecan-seminar-theory-of-change |
| Description | CECAN Seminar with Professor David Hunter, Durham University, "Understanding Health Policy in the Third Era Through a Complex Systems Lens" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Health policy is complex, arguably becoming more so. What have been termed 'wicked issues' are a central feature of health policy where there are often no simple or single solutions, if any at all. This is especially evident in regard to public health issues including obesity, alcohol and substance misuse, health inequalities. But health systems more generally are undergoing major transformation in response to changes in epidemiology, aging populations, lifestyle-related illnesses, multi-morbidities, and so on. We have entered the third era of thinking about health policy which has introduced new challenges and complexities. The talk will be illustrated with examples from the Centre of Public Policy and Health's research programme. The Centre is engaged in the Fuse partnership made up of the five North East universities in addressing these challenges to research and leads the complex systems research theme. It is also working with WHO Europe on a health system transformation programme of work focusing on the How of change and not the What. Two principal objectives of the complex systems research programme are to: • Explore the nature of transformational change in complex health systems with a view to identifying the key enablers and barriers in regard to sustainable change • Evaluate the impact on people's health and wellbeing of policy changes including the return of public health to local government in England in 2013, the programme of new models of care outlined in the NHS Five Year Forward View (2014), and the focus on place-based change arising from the Sustainability and Transformation Programme being introduced across England. In addressing the challenges facing health systems and how to transform them, the implications for research and researchers will be explored with reference to issues such as knowledge exchange, co-production, and getting evidence into policy and practice. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-seminar-understanding-health-policy-third-era-through-a-complex-... |
| Description | CECAN Seminar with Professor Phil Haynes, University of Brighton: Dynamic Pattern Synthesis |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar with Professor Phil Haynes, University of Brighton, given at BEIS to an audience of 50 people, mainly government policy makers, evaluators, analysts, researchers. Dynamic Pattern Synthesis (DPS) : A mixed method for exploring longitudinal patterns in social science data. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-seminar-dynamic-pattern-synthesis-dps-a-mixed-method-for-explori... |
| Description | CECAN Seminar with Stefan D'Errico - Evaluation in follow up and review processes of the SDGs: future vision and current practice. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Seminar - Evaluation in follow up and review processes of the SDGs: future vision and current practice. Tuesday 16th January 2018, 1-2pm - BEIS, 1 Victoria Street, London, SW1H 0ET About the seminar: The 2030 Agenda for Sustainable Development adopted by the United Nations in 2015 puts follow-up and review processes at the heart of global and national efforts to achieve the 17 Sustainable Development Goals (SDGs). It says the follow-up and review processes will be informed by country-led evaluations and data which is high-quality, accessible, timely and reliable. Stefano D'Errico, will be talking about the role and current use of evaluation in assessing the SDGs and will present findings from research conducted by IIED and EVALSDGs on M&E processes reported by the Voluntary National Reviews submitted by 64 countries to the High Level Political Forum in 2016 and 2017. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://cecan.ac.uk/events/cecan-seminar-evaluation-follow-up-and-review-processes-of-sdgs-future-vi... |
| Description | CECAN Seminar: Evaluating Policy Scenarios with Micro-Simulation - 3rd October 2016 Professor Peter Davis |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Peter briefly summarised his career history, which spans 30 years in a medical school environment as a social scientist and the challenges that presented in terms of sociologists being viewed as 'lacking reality, not engaging with the real world.' He also highlighted the fact that even in a medical environment there was an acute need for evidence, in both medicine and policy spheres. Peter then took the audience through a series of case studies to help explain micro-simulations (computer models that look like society) and how these techniques allow social scientists and policy analysts to ask the necessary counterfactuals. He feels micro-simulation is under-developed and under-intellectualised, especially because many projects involving these techniques close without models or data being replicated and made accessible, instead they are hidden within bureaucratically protected silos. He suggests the greatest use of simulation techniques are in representing social processes (e.g. pension schemes, life course, tax and employment policies), as they are based on real data and real people, and they can represent transitions at the level of individuals. Computer models can also be projected forward or parameters altered to test new scenarios, so you can pursue a policy idea which you couldn't do in the real world, helping envisage what the world would look like in terms of a specific policy framework. The greatest challenge it seems, according to Peter, is to get better access to archived models and data from past research, policies and evaluations. He stresses the need to be talking more openly with each other (researchers and government departments), and poses the question of how best to do this. Thankfully that is part of CECAN's remit and we thank Peter for raising the profile of this challenge! Peter Davis is Professor of Sociology, specialising in health and well-being, at the University of Auckland, New Zealand, with cross-appointments at that university in the Department of Statistics and in the School of Population Health. He is also founding Director of the COMPASS Research Centre, a grant-funded centre that has promoted the application of advanced methodological techniques in the social sciences for over a decade. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/news/micro-simulation-seminar-from-professor-peter-davis |
| Description | CECAN Seminar: Revaluation |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | 23rd June will go down in history as a very difficult day in Westminster - Brexit Day. Nonetheless, Andrew Darnton and Andrew Harrison boldly produced an engaging and thought provoking complexity seminar in Whitehall and we thank them wholeheartedly for a superb event. The seminar topic was 'Revaluation: Measuring Paradigm Shift'. 'Revaluation', a new innovative method for measuring change in complex systems, was presented and discussed. Revaluation comes out of a year's work evaluating 'bottom-up' change within the NHS. It is an approach based on participative methods, in which evaluators and evaluated are equal actors in a system of measurement, designed to reveal the 'full value' of the change intervention or movement. Since its evolution in the 'messy' system of the NHS, Revaluation has been applied to the challenge of tackling biodiversity, and to repositioning the public arts. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/news/cecan-seminar-revaluation-measuring-paradigm-shift |
| Description | CECAN Team members presented a Methodological Innovations "Special Event" at the Annual British Sociological Association (BSA) Conference 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN presented multiple sessions as part of this "Special Event" at the BSA 2018 Annual Conference. These were: "Getting Governance to Engage with Complexity" Byrne, D., Barbrook-Johnson, P., Byrne, D., Gilbert, N., Hills, D., Stoltz, L., Varga, L. (This session will be based on a set of CECAN papers. In this session there will be a presentation of what CECAN does coupled with an account of the innovative methods and methodological frameworks which are deployed in the programme's work. We will also address the implications of a programme like CECAN for the relationships between social science and governance, widely defined); "Wicked Issues: The Way into Governance" Byrne, D. (This session will show how the 'wicked issues' discourse has opened up the 'civil service style of governance' to the implications of the complexity frame of reference and will attempt to specify how attention to the real politics of conflict might be part of the social science reflection on this); "Evaluating Complexity", Gilbert, N. (In this talk, we will review what we mean by 'complex' and by 'Nexus', and then explain why there are particular methodological and process challenges for evaluating complex public policies. The talk will also touch briefly on the way that CECAN is organised as a cross-disciplinary, multi-partner distributed Centre closely involving stakeholders and using an 'agile' project management methodology); "Methodological Innovations in Systems Mapping for Policy Evaluation" Barbrook-Johnson, P. (We will present the innovations in the methodology of Systems Mapping - including new data collected during mapping workshops, and the novel analysis of maps this allows - that we have developed and tested in these case studies. We will also reflect on the practicalities and future possibilities of using this approach more widely in policy evaluation); "Capacity Building for Policy Evaluation in the Nexus: Collaborative Governance" Varga, L., Hills, D. (This talk discusses the contribution that the science of complex systems can make, helping in the appreciation of inter-connections and relationships between heterogeneous systems and sub-systems at multiple scales, taking account of feedback and uncertainty, as well as the non-linear emergence of outcomes whether desired, expected or otherwise. Nexus evaluation capacity building thus suggests the importance of close collaboration and co-learning, not only between evaluators and researchers who bring to the task different disciplinary skills, but also with those involved in the design and delivery of the policies being implemented. Complex systems are unique and approaches need to adapt to determine the boundaries and contexts of the policy as well as the evaluation needs of the commissioners); "Using Visualization to Explore the Evaluation of Complexity" Stoltz, L. (In this presentation, I will discuss how I collected visualisations from the Journal 'Evaluation' and reflect on those images and some of the implications this exercise raises specifically in relation to the limits and opportunities relating to visualisations use in complex policy and evaluation research). |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.britsoc.co.uk/media/24644/ac2018_programme.pdf |
| Description | CECAN Twitter |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | CECAN has a Twitter account @cecanexus which is regularly used to post about CECAN events, seminars, blogs, papers, policy practice notes, other achievements (eg. syllabus, toolkits). As of 13/3/19, the account has 914 followers (last count: 640 on 15/5/18), is following 185 people (last count: 140 on 15/5/18) and has made 637 Tweets (last count: 456 on 15/5/18). Almost all Tweets indicate some level of engagement with Followers, eg. usually at least 4 or 5 re-tweets / likes / comments. |
| Year(s) Of Engagement Activity | 2016,2017,2018,2019 |
| URL | https://twitter.com/cecanexus |
| Description | CECAN Webinar - The Pluralistic Evaluation Framework for Policy Evaluation (Richard Gunton) 20 Nov 18 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | In a democratic system, how can we show that a policy has improved a complex situation in a way that represents good use of public funds? "Improve" and "good" are very general words, calling for pluralistic evaluation. That is, to do justice to the complexity of a situation, evaluations should account for (a) the different kinds of systems that a policy may impact simultaneously, (b) the values of different stakeholders, and (c) the different ways in which any stakeholder may value changes to a situation. A comprehensive evaluation ought to account for the wide range of interests that may be held by diverse stakeholders concerning the multiple components of a situation of interest |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-pluralistic-evaluation-framework-for-policy-evaluation |
| Description | CECAN Webinar - "How to evaluate or commission an evaluation when everything is messy (Or - don't panic, evaluation might just be the answer!)" presented by Dione Hills and Helen Wilkinson, facilitated by Martha Bicket, 30 Jan 19 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Helen, Dione and Martha are all part of a CECAN team that has been developing guidance for evaluation practitioners and commissioners struggling with complex interventions - and particularly those working in complex policy areas. The guidance was requested by the government team of evaluators who are revising the Magenta Book (Cross government guidance on evaluation) with the idea of having an annex on complex evaluation ready for when this is published. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-how-evaluate-or-commission-evaluation-when-everything-m... |
| Description | CECAN Webinar - Developing Microsimulation Models to Generate Evidence to Support Policy Change in Public Health, Abbygail Jaccard, 7 Nov 18 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Over the last decade the UK Health Forum has been developing microsimulation models to estimate the future impact of public health policies targeting both behavioural (e.g. obesity, smoking) and environmental risk factors (e.g. air pollution) on the disease and economic burden to the health system and wider society. The public health policies investigated have ranged from individual level (e.g. a weight management intervention) to population level (e.g. tobacco duty escalator, a 20% sugar-sweetened beverage tax). The UKHF microsimulation model simulates individuals within a synthetic population through time. Each individual within the model has a risk of developing a risk factor related disease, dying from an existing condition or dying from other causes. Abbygail will present the UKHF microsimulation model and highlight key findings which have aimed to support policy change. In addition, she will present related tools which have been developed to enable policy makers to generate evidence at local and national levels. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-developing-microsimulation-models-to-generate-evidence-... |
| Description | CECAN Webinar - Handling subjective views and bringing rigour to contribution analysis: Bayesian Belief Networks and evaluating likelihoods in action, Simon Henderson and Stuart Astill, 16 Oct 18 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Simon Henderson and Stuart Astill will give a talk showing how influence maps analysed using Bayesian Belief Networks can be used to add rigour to contribution analysis (and other evaluative judgements) in situations where subjective views rather than objective outcome data is the main source of knowledge. They will look at how a well-planned approach can structure and aggregate subjective views using influence mapping, Bayesian Belief Networks (BBNs) and conceptions around evaluating likelihoods. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-handling-subjective-views-and-bringing-rigour-to-contri... |
| Description | CECAN Webinar - Risk Analysis at the Food Standards Agency post EU-Exit: The role of economics and social science in informing risk management (Vanna Aldin, Head of Analytics and Chief Economist, Food Standards Agency) 6th Feb 19 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The Food Standards Agency, a non-ministerial government department, is currently revising its risk analysis process as part of the preparatory work for EU Exit. The food and feed safety risk analysis process contributes to the FSA objective to ensure that the high standard of food safety and consumer protection we enjoy in the UK is maintained when the UK leaves the EU. As the FSA's role changes after EU Exit, we are working on the FSA framework to show how we undertake risk analysis. The FSA is already following globally recognised frameworks for risk analysis (notably the Codex Principles). This webinar will highlight the central role of economics and social sciences in informing "the other legitimate factors" - which is the internationally accepted terminology for issues other than risk related to human health- risk managers will consider alongside the risk assessment to develop, analyse and identify risk management recommendations. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-risk-analysis-food-standards-agency-post-eu-exit-role-e... |
| Description | CECAN Webinar with Dr Barbara Befani - "Choosing Appropriate Evaluation Methods: a Tool for Assessment and Selection" |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The range of methods available to produce useful, credible and rigorous evidence in evaluation is growing but selecting the right method or combination of methods can be difficult. CECAN will host a webinar with Dr Barbara Befani who has designed a tool to help understand evaluation methods in more depth. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-choosing-appropriate-evaluation-methods-a-tool-for-asse... |
| Description | CECAN Webinar with Gill Westhorp - Complexity theory and "invisible mechanisms": Implications for methods and commissioning |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Webinar - Complexity theory and "invisible mechanisms": Implications for methods and commissioning Wednesday 31st January 2018, 09:00 - 10:00 UK Speaker: Dr Gill Westhorp, Charles Darwin University Australia About the webinar: This presentation continues the CECAN series on realist research and evaluation and their use in relation to complexity. It focuses on the issue of causation. Causation is of critical interest to policy and program authors, who seek to cause (or steer) change; to evaluators who seek to attribute outcomes to interventions, and to researchers who seek to understand particular aspects of the operations of complex systems. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://cecan.ac.uk/events/cecan-webinar-complexity-theory-and-invisible-mechanisms-implications-for... |
| Description | CECAN Webinar with Justi Jagosh PhD - Realist methodology part 3 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | This webinar is 3rd in a series on realist methodology hosted by Dr. Justin Jagosh for the Centre for Complexity across the Nexus (CECAN). Working with Pawson and Tilley's realist approach to evaluation brings the idea that the functioning of a programme can be determined from realist programme theories that sketch out its basic architecture. This becomes increasingly evident in the process of theory testing using the context-mechanism-outcome configuration in which we try to understand how resources, when placed in environments trigger responses to produce outcomes. In understanding how programmes work, we may arrive at questions about dosage (e.g., how many times should we expect the resource to be delivered before the desired response will be triggered?) and timelines of impact (e.g., what are reasonable estimations of time delay between the introduction of resources and the triggering of response?). Building on the material presented in the previous CECAN webinars on realist methodology, here it is argued that these facets of programme architecture are not immediately apparent, and require consideration when developing our realist programme theories to open us to the possibility of 'ontologically deep' empirical testing using qualitative and quantitative data collection tools and techniques. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-webinar-realist-methodology-part-3 |
| Description | CECAN Webinar with Justin Jagosh - The "Context+Mechanism" Association: Mastering a Key Heuristic in Realist Evaluation for Innovating Complex Programmes and Policy. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Webinar to approx 100 people with Justin Jagosh PhD - The "Context+Mechanism" Association: Mastering a Key Heuristic in Realist Evaluation for Innovating Complex Programmes and Policy. Confusion often arises in determining when data should fit under 'context' or else 'mechanism' in the process of configuring. This webinar will offer a simple set of definitions for context, mechanism and outcome and will introduce a number of examples of 'raw data' that we will transform into CMO configurations during the webinar to exemplify how to do it. The goal is to stimulate ideas around how to define concepts, theorize programmes and configure data in realist analysis, with the ultimate ambition of increasing capacity for using realist evaluation to innovate and transform programmes. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/events/context-mechanism-association-mastering-a-key-heuristic-realist-evalu... |
| Description | CECAN Webinar: "Evaluating Policy for Food Security - Before and After Implementation" with Dr Martine Barons of Warwick University, 4th May 2018 Chaired by Martha Bickett (CECAN) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | About the Webinar: Household food security is a classic complex system. Countries that are food secure when viewed at country-scale may nevertheless have considerable food security problems at household scale. Furthermore, local government wishing to design policies to ameliorate food insecurity and poverty for its citizens may find that their administrative boundaries cut across social boundaries and the areas covered by agencies required to collaborate may be misaligned. In order to provide decision support of local government, we have developed a method that networks together relevant data sources, models and expert opinions to evaluate candidate policies' relative effectiveness before implementation. The hard thinking required by stakeholders to fully populate this integrating decision support system and use it to compare and them select policies, can be leveraged to guide interim and end-of project evaluation. Martine is the Director of the Applied Statistics and Risk Unit at Warwick University |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/cecan-webinar-evaluating-policy-for-food-security-before-and-after-im... |
| Description | CECAN Webinar: Making policy and making policy work with developmental evaluation, Dr Martin Reynolds (The Open University), 24th July 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Developmental evaluation endeavours to reconfigure evaluation practice away from merely serving purposes of external accountability of policy towards making evaluation integral to the dualistic practice of policy making and policy implementation. Whilst ideas on developmental evaluation from Michael Quinn Patton have gained traction with increasing concern over interventions in contexts of rapid change and uncertainty, endeavours have focused on applying concepts principally from a tradition of complexity science. A key challenge is in making such concepts work for evaluators and other stakeholders. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/webinar-dr-martin-reynolds-open-university-making-policy-and-making-p... |
| Description | CECAN Website |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN Website set up in March 2016 as a primary means of engagement with target audiences and the general public. The website has grown to incorporate information about the project, its people, opportunities to engage, as well as being a repository for podcasts, EPPNs, news items and an active blog. |
| Year(s) Of Engagement Activity | 2016,2017 |
| URL | http://www.cecan.ac.uk/ |
| Description | CECAN Workshop hosted by the Alan Turing Institute and CECAN - What Good Data could do for Evaluation, 25th Feb 2019 at The Alan Turing Institute, British Library, London |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The rise of data science has opened up a number of opportunities for government policy planning and complex evaluation. Although both data science and evaluation involve using data to better understand a particular issue, the benefit of using data science for policy evaluation has still to be established. In what ways might future evaluation benefit from data science? Perhaps the nature of policy evaluation requires a new kind of data science, specifically shaped for complex evaluations? In what ways might complex policy evaluation challenge current ideas of what data science is and it can do? How might data science be used or adapted specifically for evaluating complex government policy? This workshop brings together policy practitioners, evaluators and researchers interested in exploring the applicability of data science and new methodological approaches for policy evaluation. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.cecan.ac.uk/events/cecan-workshop-what-good-data-could-do-evaluation |
| Description | CECAN and NEKS Workshop Public Infrastructure and Private Finance - Can there be a happy ever after? |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN and NEKS Workshop June 2017 approx 50 people Government and Private Finance Sectors. Post-workshop report available on CECAN website (link provided below) |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/sites/default/files/2018-01/Henry%20Leveson%20Gower%20Report.pdf |
| Description | CECAN presentation at CIED Summer school |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Anna Kaxira, CECAN PhD student gave a presentation of her PhD project and also spoke about CECAN to the audience at the CIED summer school. The summer school had 27 participants who are early PhD or postdoc researchers. However, the most interesting fact is that we had a session on 'Achieving Policy impact' during which Fraser Macleod, a Social Researcher at BEIS, mentioned his cooperation with CECAN on CfD. In addition, he presented CECAN as an example of cooperation with the academia. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cied.ac.uk/event/cied-summer-school-2017/ |
| Description | CECAN presentation at Royal Geographical Society Annual Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Candice Howarth presenting on CECAN at Royal Geographical Society Annual Conference (RGS-IBG), 'nexus theme' - 1st September |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.rgs.org/WhatsOn/ConferencesAndSeminars/Annual+International+Conference/Annual+Internation... |
| Description | CECAN presentation to Scottish Government |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Ian Christie giving presentation to Scottish Government's climate/environment team in their Climate Week programme on 21st September. |
| Year(s) Of Engagement Activity | 2016 |
| Description | CECAN ran a session at the 2018 ESRC Research Methods Festival, "Complex Evaluation Methods", on 3rd July at the University of Bath |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Pete Barbrook-Johnson and Alex Penn presented "Participatory Systems Mapping for Policy Analysis" (http://eprints.ncrm.ac.uk/4206/1/Sys_Mapping_NCRM_BATH_2018_for_sharing.pdf) Nigel Gilbert presented "Applying Agent Based Modelling (ABM) to Evaluation" (https://www.cecan.ac.uk/videos/applying-agent-based-modelling-abm-to-evaluation-prof-nigel-gilbert-2018-research-methods) Emma Uprichard presented "Big Data Evaluation?" (https://www.ncrm.ac.uk/RMF2018/programme/session.php?id=C6) |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.ncrm.ac.uk/RMF2018/programme/session.php?id=C6 |
| Description | CECAN talk at M2D 1st Annual Conference on Decision Making Under Uncertainty University of Exeter, UK |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Professor Liz Varga, University of Cranfield gave a talk at the M2D Annual Conference on Decision Making Under Uncertainty, entitled: The ability of models to predict the effects of policy decisions. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://blogs.exeter.ac.uk/models2decisions/events/m2d-2017-annual-conference/ |
| Description | CECAN team members ran a session at the 2018 UK Evaluation Society (UKES) Annual Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | On 2nd May 2018, Nigel Gilbert, Pete Barbrook-Johnson, Alex Penn and Dione Hills ran one of the parallel sessions at the 2018 Annual UKES Conference in London. The session was called "Using complexity and co-production for quality evaluation: The CECAN experience" and ran for 90 minutes |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.evaluation.org.uk/index.php/events-courses/ukes-conference/278-ukes-2018-annual-evaluati... |
| Description | CECAN was invited by HMRC to come and talk about complex evaluation (4th July 2018) |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | In June 2018, HMRC contacted CECAN to ask if someone could give a talk about complex evaluation to help them with looking to explore ways of evaluating a new suite of policy measures targeted at addressing offshore tax evasion. The nature of the policy issue means that some of the most concerning behaviours are also those we know the least about. It is also complex because behaviours are international and they cut across multiple different tax regimes. This was delivered at 3pm on 4 July at HMRC's Canary Wharf offices. |
| Year(s) Of Engagement Activity | 2018 |
| Description | CECAN webinar with Justin Jagosh (PhD) University of Liverpool. "Realist Methodology for Complexity-Mindful Evaluations in the Food, Energy, Water and Climate Sectors". |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN webinar with Justin Jagosh (PhD) University of Liverpool. "Realist Methodology for Complexity-Mindful Evaluations in the Food, Energy, Water and Climate Sectors". Approx 100 people registered. Many approaches to addressing complexity in evaluation design are burgeoning. In part this is in response to the inadequacy of 'complexity-thin' approaches to supporting solutions to entrenched and evolving problems. A second factor is witnessing new complex problems emerging with radical changes in society and sectors. In this context, the realist paradigm serves as a guide to expand complexity-mindedness, which presumably translates into more useful and relevant knowledge products, and a community of realist evaluation specialists dedicated to developing a lens on 'context-mechanism' interactions. This webinar will introduce realist methodological principles, and explain how realism can increase our complexity-mindedness and capacity to gain causal insight. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-webinar-realist-methodology-for-complexity-mindful-evaluations-f... |
| Description | CEE's First International Conference 'Better Evidence, Better Decisions, Better Environment' |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Introducing CECAN at CEE's First International Conference 'Better Evidence, Better Decisions, Better Environment' was held on August 25-27th 2016 at the Swedish Museum of Natural History, Stockholm, Sweden (CEE is the Collaboration for Environmental Evidence) |
| Year(s) Of Engagement Activity | 2016 |
| Description | CPD Course with Professor Brian Castellani of Kent State University - COMPLEX-IT and the SACS Toolkit: |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CPD Course with Professor Brian Castellani of Kent State University - COMPLEX-IT and the SACS Toolkit: urpose: The complex socio-technical arenas (nexus issues) that government seeks to improve (e.g., health, food, water, safety, infrastructure) are not driven by a single factor or consequence. Instead, they are driven by multiple factors at multiple levels, which lead to different trends or outcomes for different areas/groups of people. The challenge is how to model such diversity and complexity? The complexity sciences, data mining and big-data offer some useful solutions. The challenge, however, is stitching these methodological solutions together into a user-friendly platform and APP, which policy makers, social scientists, evaluation commissioners and civil servants can use - hence our creation of COMPLEX-IT and the SACS TOOLKIT. Intended audience: This workshop is for anyone involved in evaluating the impact of policy (and its improvement) on complex nexus issues and would like to explore new software and mixed-methods options for doing so. At the end of this course, participants will: GOAL 1: Understand the theory behind case-based computational modeling, including Having a basic sense of the principles guiding case-based complexity. Understanding the philosophy behind data mining and computational modeling. Developing a working knowledge of COMPLEX-IT APP and SACS TOOLKIT. GOAL 2: Learn how to apply case-based computational modeling to their nexus topic, including how to: Build a complex systems model of their nexus issue. Explore how policy impacts different groups or areas across time/space. Use this information to create your study's case-based profile. Identify major and minor case-based clusters and key causal factors. Identify major and minor cluster trends (for longitudinal data). Identify key global-temporal dynamics, such as spiraling sources and saddle points. Use network analysis (where appropriate) to explore cluster links and structure. Examine how different clusters and trends lead to different outcomes. Run simulations to explore how policy can change outcomes. Compare resulting model to original theoretical formulation. GOAL 3: Learn how to use the COMPLEX-IT APP, including how to: Download and install the software. Run the software, including the R Studio environment in which it works. Upload the case study database. Identify key variables for case-based profiles. Explore how to deal with missing data and errors in variable choice. Use k-means cluster analysis to identify initial clusters, including how to identify. optimal solutions and run k-means for trend data. Use the SOM neural net to corroborate clusters and identify possible sub-clusters. Use SOM and k-means to identify underlying causal model. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/complex-it-and-sacs-toolkit-a-case-based-computational-modeling-platfo... |
| Description | CRESS Seminar - "Understanding innovative public-private partnerships at the food-energy-water-environment Nexus", Pete Barbrook-Johnson, 27th November 2018, Surrey |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | Pete Barbrook Johnson delivered a seminar at University of Surrey called "Understanding innovative public-private partnerships at the food-energy-water-environment Nexus". He presented preliminary findings from an overview and critical review of current public-private partnerships (excluding PFI projects) in the UK, working in food, energy, water and environment (or 'Nexus') domains. He considered what partnerships are out there, what forms they take, how they are funded, what affects their success and effectiveness, and how we can learn from them. |
| Year(s) Of Engagement Activity | 2017,2018 |
| Description | Candice presented CECAN overview at 'Realist Evaluators in Government' meeting - 7th July |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Candice Howarth presented CECAN overview at 'Realist Evaluators in Government' meeting - 7th July |
| Year(s) Of Engagement Activity | 2016 |
| Description | Complexity in Evaluation - CECAN Residential Workshop - 15/16th September 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | his 2 day residential workshop, conducted under the Chatham House Rule, will bring together evidence teams, policy makers, policy analysts, complexity scientists, evaluation experts and experts in Nexus subjects. Our aim for the workshop is to collectively generate deeper and shared understanding around complexity and evaluation whilst producing concrete outputs. In particular the co-construction of a general-purpose timeline for approaching evaluation in complex contexts. This will be designed to bring together your expertise and practical experience with our tools and approaches, to produce a workable, yet innovative, real world method. We will combine participatory design and collaborative problem solving with information and ideas exchange between CECAN and partners. We will pool the diverse expertise of everyone present to collectively frame and make progress on the key issues, whilst learning from each other, introducing novel approaches and connecting them to the realities of evaluation in a variety of contexts. Sessions will be designed to: scope the problem areas, practicalities and resources and tools at our disposal; make connections and develop ideas to find routes for progress; and co-develop a practical output constructed and illustrated using real-world evaluation examples. Sessions will include: Micro-presentations on innovative tools, approaches and case studies of evaluation of complexity in practice. Considering real examples and case studies-where is the complexity? Scoping the evaluation landscape and surveying the practical challenges and capacity to innovate. Framing and developing the key issues for evaluation of complexity via participant-driven, self-organised discussion. Problem-solving surgeries for specific issues, matching tools and problems and action planning in light of "landscape". Structured co-design of evaluation timeline. Plenary report-back session, summarising learning, outputs and next steps. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/events/15th-16th-september-surrey-2-day-residential-complexity-evaluation-wor... |
| Description | DPS Summer School University of Brighton 18th September 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | This training offers the chance to learn the basics of a new mixed method called DPS, that uses both Cluster Analysis and Qualitative Comparative Analysis (QCA) to identify dynamic patterns in longitudinal small sample data. For example, for comparing patterns over time in the paths taken by countries, regions or organisations. The method also has some potential to examine small samples of individuals. Aims of the Summer School: Explain the basics of the DPS method Allow participants to try out the basics of the methods using SPSS and Excel - this to run through the different stages of the research. Encourage participants to begin using the method with their own dataset, if they wish, (or using available data made available). |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/events/dynamic-pattern-synthesis-dps-methods-summer-school-18th-september-20... |
| Description | Dr Candice Howarth - Presentation on CECAN at the Water-Energy-Food-Nexus POST NOTE Event, London, 9th December 2016 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Dr Candice Howarth - Presentation on CECAN at the Water-Energy-Food-Nexus POST NOTE Event, London, 9th December 2016 |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://researchbriefings.parliament.uk/ResearchBriefing/Summary/POST-PN-0543 |
| Description | Dr Pete Barbrook-Johnson, Dr Amy Proctor and Dr Adam Hejnowicz presented at the British Science Festival in Hull on 12th September 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | CECAN researchers Dr Pete Barbrook-Johnson, Dr Amy Proctor and Dr Adam Hejnowicz presented at the British Science Festival in Hull on 12th September 2018. In a sold-out session exploring how Brexit might affect UK policy evaluation, the audience heard how CECAN research is supporting new approaches to waste crime and rural development policy evaluation before debating complexity and wicked issues in policy evaluation. Slides from the presentation are available here: https://www.dropbox.com/s/obet1sw41voacg4/Post%20Brexit%20policy%20whats%20the%20deal%20BSF%20Hull%2012%20Sept%20Final.pptx?dl=0 Title: Post-Brexit policy: what's the deal? Blurb: For better or worse, Brexit means government policy is undergoing the biggest upheaval in living memory. Many policies are likely to be reimagined, but will anyone ever ask whether they are working? This panel discusses the cutting-edge research surrounding this contentious area, exploring if and how the effectiveness of public policies is figured out by those who set them. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.britishscienceassociation.org/british-science-festival |
| Description | ESRC/NCRM Research Methods Festival |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Emma Uprichard of the University of Warwick, organised and presented a session on 'Complexity and Method' at the ESRC/NCRM Research Methods Festival. University of Bath, 5-7 July. |
| Year(s) Of Engagement Activity | 2016 |
| Description | EnAHRgie - Conception of sustainable land use and energy supply at the municipal level. Implementation in the model region - Innovation management workshop with BMBF project EnAhrgie. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Industry/Business |
| Results and Impact | EnAHRgie - Conception of sustainable land use and energy supply at the municipal level. Implementation in the model region Ahrweiler. Innovation management workshop with BMBF project EnAhrgie. - Professor Petra Ahrweiler on behalf of CECAN |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.ea-aw.org/research/overview/enahrgie.html |
| Description | Engagement with Environment Agency |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | General methods discussion with EA (Jon Greaves) - Dependency modelling, ABM and general discussion on evaluation in the face of complexity - 7th October - Helen, Paul, Michelle |
| Year(s) Of Engagement Activity | 2016 |
| Description | Enhancing Evaluation of Impact |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN's role is to help improve the effectiveness of evaluation in nexus settings. As well as exploring methodological innovations we need to understand the policy uptake and impact of evaluation. The use and influence of evaluation is inherently complex and contingent. This workshop will explore when and why do evaluations have impact and how can evaluation practice become more impactful in future. Workshop aims: 1) To share recent experience on successful evaluation impact in complex nexus policy areas 2) To share expertise in maximising the impact of evaluation in these policy settings 3) To generate practical recommendations for policymakers, analysts and evaluators 4) To generate ideas for focussing future R&D in this area 5) To produce a policy note on how evaluation practice can become more impactful in future |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/cecan-1-day-workshop-enhancing-evaluation-impact-complexity-settings |
| Description | Evaluating Complexity and Complexity in Evaluation - Workshop - 9th June 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Members of the CECAN team came together in Central London for a 1 day workshop to brainstorm what we mean by 'complexity in evaluation. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/events/9th-june-london-evaluating-complexity-and-complexity-evaluation |
| Description | First International Conference of the Collaboration for Environmental Evidence: Better Evidence, Better Decisions, Better Environment. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Ben Fagan-Watson, CECAN's Knowledge Integration Manager, presented to the First International Conference of the Collaboration for Environmental Evidence: Better Evidence, Better Decisions, Better Environment. August 25-27th 2016 Swedish Museum of Natural History, Stockholm, Sweden. The audience was a mixture of academics, policy people and NGOs. He delivered a presentation on evaluation of 'Nexus' issues in the UK government, based on the research PSI has done 2012-15 on Defra's evaluation capacity; and outlined the role and remit of CECAN in meeting these challenges. The emphasis was largely ex-ante methods, but Ben and few others challenged this and emphasised the importance of linking this to ex-post evaluation. Outcomes: CECAN webinar by another attendee. Link established to CIFF, a charitable funder that commissions large scale evaluations of their initiatives |
| Year(s) Of Engagement Activity | 2016 |
| Description | GESR Evaluation mini-conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel have a presentation about CECAN at the GESR Evaluation mini-conference, HM Treasury, 23 March. Invited presentation: 'Evaluating Complexity' to an audience 180 policy researchers |
| Year(s) Of Engagement Activity | 2016 |
| Description | IKnowFood launch event at University of York |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | IKnowFood launch event at University of York - CECAN marketing opportunity. On 8th June 2017, over a hundred people representing business, policy, academic and public interests gathered at the public launch of the Global Food Security-funded IKnowFood project. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://youtu.be/E2fV6kDf--s |
| Description | Ian C - talk on CECAN for audience of civil servants in Scottish Government working on climate policy, environment, economic development; venue - Scottish Government offices, South Quay, Edinburgh, 21st Sept. 2016; |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Professor Ian Christie - talk on CECAN for audience of civil servants in Scottish Government working on climate policy, environment, economic development; venue - Scottish Government offices, South Quay, Edinburgh, 21st Sept. 2016; audience - approx. 40. Follow-up meeting under discussion, to include CECAN director and knowledge integrator. |
| Year(s) Of Engagement Activity | 2016 |
| Description | Ian Christie was invited to speak at WZB / Univ. of Bielefeld Conference on Policy Evaluation and the Politics of Evaluation, in Berlin on 17th Nov 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Ian Christie was invited by Jonas Shoenefeld to speak at WZB / UNIV. OF BIELEFELD CONFERENCE ON POLICY EVALUATION AND THE POLITICS OF EVALUATION in Berlin. Jonas had previously presented at the 2018 CECAN Conference earlier in the year. He gave a presentation about CECAN's work and there was a lot of interest from the audience (made up of German and Swiss experts in evaluation and policy/politics). |
| Year(s) Of Engagement Activity | 2018 |
| Description | Invitation to join the BEIS external Peer Review Group (PRG) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN received and accepted a formal invitation from the Department of Business, Energy and Industrial Strategy (BEIS) to act as a subject expert to be included in their external Peer Review Group (PRG). The function of the PRG is twofold: - to improve the quality of evaluation within the department by providing an independent, transparent assessment of the quality of BEIS's evaluations prior to publication; and - to build a network of academics to share their expertise with BEIS. There are currently 17 academics and experts on the panel covering a variety of areas of interest and providing a mix of experience. Independent, transparent quality assurance of evaluations is an important element of BEIS's departmental strategy. This will be an opportunity to influence and provide advice on the development of the evidence-base across a range of high-profile policy areas. BEIS has recently published a summary of recommendations from the expert peer review group as to how it can continue to improve the quality of policy evaluations. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Keynote on 'Big Data, Complexity and Policy' |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Emma Uprichard, University of Warwick, gave a keynote on 'Big Data, Complexity and Policy'. International Symposium on Grids and Clouds (ISGC) 2016, Academia Sinica, Taipei, Taiwan, 13-18 March. |
| Year(s) Of Engagement Activity | 2016 |
| Description | Liz Varga Invited as discussant to Conference "Opening up to an Era of Innovation" 27-28 November 2017, Lisbon |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Prof Liz Varga was invited as a discussant to Conference "Opening up to an Era of Innovation" 27-28 November 2017, Lisbon, Portugal for the workshop on Evidence Informed Social Policy Innovation: Redesigning European welfare systems through ICT-based dynamic modelling. The session on "Redesigning European welfare systems through ICT-based dynamic modeling" was developed in order to address some of the most relevant questions on how social innovation can be applied to European welfare systems, in particular through the use of ICT-based dynamic modeling. The workshop was designed to be interactive, allowing participants to discuss possible applications of methodologies and tools for impact assessment of social innovation policies. Main takeaways from the session: The debate around "Redesigning European welfare systems through ICT-based dynamic modeling" is very timely and this was reflected in the diverse angles of the debate that were brought forward as well as the enthusiasm of the participants. The main conclusion of the discussion was that social innovation, being pluralistic and diverse, allows for the possibility to have a more complete toolbox of measures to respond to the challenges of the future. This creates the need for better evaluation models in order to measure impact and learn from diverse experiences. There is no one solution for this; it is more about evaluating continuously. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://ec.europa.eu/research/conferences/2017/era/index.cfm?pg=programme |
| Description | Liz Varga was an Invited Speaker, Resource Nexus Policy & Cluster Workshop "Stepping Up and CECAN nexus projects" 27 November 2018, EASME, Brussels |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The workshop brings together EU-funded projects related to the resource nexus as well as other (non-EU funded) related initiatives with the objective to stimulate peer-to-peer and science-to-policy discussions on how to get the most out of nexus research for the benefit of society and policy-making. The Resource Nexus Policy & Cluster Workshop is co-organized by the Executive Agency for SMEs (EASME) and the Horizon 2020 projects SIM4NEXUS MAGIC, and DAFNE. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://ec.europa.eu/easme/en/nexus-policy-and-cluster-workshop |
| Description | NERC Into the Blue event, Oct 2016, |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Candice Howarth running masterclass on 'Knowledge integration and delivering impact through research' at NERC Into the Blue event, Oct 2016, Manchester |
| Year(s) Of Engagement Activity | 2016 |
| Description | Nigel Gilbert gave a presentation about CECAN at the Defra group Science and Analysis Conference, 'Science for a Changing Future', 20th March 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel Gilbert was invited by Professor Ian Boyd (Department for Environment, Food and Rural Affairs' Chief Scientific Adviser) to the Defra group Science and Analysis Conference entitled 'Science for a Changing Future' on Tuesday 20th March 2018 at The Westminster Conference Centre, 1 Victoria Street, Westminster, where he gave a short presentation about CECAN. Objective: The conference will seek to explore the long-term challenges of Air quality, Land use (including Livestock and Food) and Marine management. These areas were identified for more in depth consideration at last year's joint Defra / Royal Society conference. This year we are seeking views on proposals to develop integrated models of these systems, explore the independencies between them and develop next steps for developing the models across the scientific community. It is our intention that by bringing together people with different expertise, valuable networking opportunities will arise. Format: There will be presentations on working with complex systems and delegates will have an opportunity, through workshops, to be involved in shaping this work in the future. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Nigel Gilbert gave a talk about CECAN to members of the Cabinet Office who visted Surrey on 1 June 2018 for a relationship building event |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel Gilbert was invited to gave a talk about CECAN to members of the Cabinet Office who visted Surrey on 1 June 2018 for a relationship building event chaired by Atti Amecz (Director of Research Strategy) |
| Year(s) Of Engagement Activity | 2018 |
| Description | Nigel Gilbert invited to participate in DEFRA Science and Evidence Assessment Group |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | 1st Feb 2018: Nigel Gilbert participated in a workshop for DEFRA Science and Evidence Assessment Group as an 'evidence specialist' |
| Year(s) Of Engagement Activity | 2018 |
| Description | Nigel Gilbert presented at the Government Operational Research Service (GORS) Conference 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel gave a talk to about 180 members of the Government Operational Research Service (GORS) on 31 Oct 2017 at the BEIS Conference Centre, London In this talk, I'll explain what we mean by complexity and by nexus, and then briefly introduce some evaluation methods and frameworks that are 'complexity-appropriate', with examples. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Nigel Gilbert was invited as an expert speaker to the Behave Summer School on Agent-Based Modelling for Social Scientists in Italy (Sept 2018) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Jointly organised by the ESLS PhD Programme in Economic Sociology and Labour Studies, the Behave Lab (University of Milan) and GECS - Research Group on Experimental and Computational Sociology (University of Brescia), this training school aims to introduce students to NetLogo, the most popular platform to build Agent-Based Models (ABM), by using modelling examples from social science research. Full-day hands-on training; 3 classic ABMs will be coded from scratch in NetLogo; Assistance and customized counselling on personal research projects; Personal project presentations on the final day; Evening informal talks by invited experts on ABM-related issues. Students will be provided with the theoretical background on the use of ABM in social science research and will learn how to develop an ABM from scratch. The examples will be on social impact theory, culture dynamics and social network effects on individual beliefs. Individual activities and exercise will be combined with group activities. No prerequisite on computing is needed. Students are expected to join the training with their own laptop. Students are also encouraged to develop a customized project starting from a personal research idea: bring your own model or data if you would like to work on it! |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://sites.google.com/view/abm-school/ |
| Description | Nigel Gilbert was the invited keynote speaker at "Research Colloquium in Application of Analytics, Modelling and Simulation in Complex systems", a workshop organised by the UKOR Society and Surrey Business School, 4th July 2018, Guildford |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nigel Gilbert was invited to be the keynote speaker at "Research Colloquium in Application of Analytics, Modelling and Simulation in Complex systems", a half-day workshop organised by the UK Operational Research Society and Surrey Business School at Wednesday 4th July 2018 12:00-16:00 at the University of Surrey. This was a Special Event as part of the Surrey Research Colloquium / UK Operational Research Society Simulation SIG Roadtrip with Paul Fishwick. This colloquium series at the University of Surrey aims to highlight and discuss the challenges of business analytics in the real world, create awareness on relevant cutting-edge technologies, tools, methods and processes in business analytics and data science, thus offering participants an exposure to various research trends in the industry and enabling them to engage with real-world business problems. This Research Colloquium is for Academics, Researchers, PhD students, Industry leaders and experts, practitioners or anyone else interested in Simulation and Business Analytics. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Nigel presenting at 'Challenges & Opportunities in New Funding Landscape: Creating Effective Research Policy Partnerships - British Academy |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel presenting at 'Challenges & Opportunities in New Funding Landscape: Creating Effective Research Policy Partnerships - British Academy 20th July - this invitation only conference took place on the 20 July 2016 at the British Academy in London with key speakers and panellists including Jane Elliot, (Chief Executive, Economic & Social Research Council, and RCUK champion of the Global Challenges Fund); Sarah Church, (Director of Food & Farming, Defra); Louise Heathwaite, (Chief Scientific Advisor for Environment & Rural Affairs, Scottish Government); Nigel Gilbert (Director, CECAN); Sarah Whatmore, (Professor of Environment and Public Policy, University of Oxford); and Alun Evans, (Chief Executive, British Academy). |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.geog.ox.ac.uk/news/articles/160720-OxDefraConf.html |
| Description | Nigel presenting at DFID 'Innovation in Evaluation' event 8th September Oxford |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Nigel presenting on CECAN at DFID 'Innovation in Evaluation' event 8th September Oxford |
| Year(s) Of Engagement Activity | 2016 |
| Description | Pete Barbrook-Johnson - Poster presentation at 'We need to talk about complexity' conference at Oxford Green Templeton College |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Dr Pete Barbrook Johnson presented poster at 'We need to talk about complexity' conference 13-14 June 2017 at Oxford Green Templeton College. Attended by 60+ health evaluation researchers and practitioners, from around the world. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.phc.ox.ac.uk/events/we-need-to-talk-about-complexity |
| Description | Pete Barbrook-Johnson - scoping evaluation options for TfL/Mayor of London policies on air quality/pollution |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | PBJ - attended workshop on scoping evaluation options for TfL/Mayor of London policies on air quality/pollution. Ongoing input expected into documents TfL are preparing on scoping out plans for Mayor's office. 5 July. 15 people. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Pete Barbrook-Johnson invited to give one-day course on ABM at University of Essex Summer School (1/2) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Pete Barbrook-Johnson invited to give two one-day courses on ABM at University of Essex Summer Schools. One on 11 July, one on 2 Aug. Total 20 students (11 and 9: PhDs, academics, and local government analysts) |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.blgdataresearch.org/blgsummerschool2017/ |
| Description | Pete Barbrook-Johnson invited to give one-day course on ABM at University of Essex Summer School (2/2) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Pete Barbrook-Johnson invited to give two one-day courses on ABM at University of Essex Summer Schools. One on 11 July, one on 2 Aug. Total 20 students (11 and 9: PhDs, academics, and local government analysts) |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.blgdataresearch.org/blgsummerschool2017/ |
| Description | Pete Barbrook-Johnson presented CECAN at Lorentz workshop on modelling socio-ecological systems |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | PBJ - 1-5 May attended Lorentz workshop on modelling socio-ecological systems - presented CECAN. Attended by 25 people. International |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.psi.org.uk/site/news_article/lorentz_center_workshop_on_simulating_socio_ecological_syste... |
| Description | Pete Barbrook-Johnson ran a workshop for UK Evaluation Society (UKES) - "Using Systems Mapping in Complex Evaluations" |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | The UKES London and Southeast Regional Network Event: "Using Systems Mapping in Complex Evaluations", with Pete Barbrook Johnson on 29th March 2018, 4.30 - 6.30pm in Seminar room 3, Resource for London, 356 Holloway Road, London N7 6PA Learn about the methodological principles of systems mapping and how to apply it in your areas of expertise. In this seminar, Pete will outline CECAN's use of systems mapping to support policy evaluation in complex settings. Pete will introduce the systems mapping approach, and the methodological developments being pioneered and tested in CECAN, using examples from work with DEFRA and BEIS. He will consider how this approach can be of use to evaluators and complement existing approaches and tools such as Theory of Change maps and policy maps. Finally, Pete will discuss practicalities and resources so that participants can consider using the approach themselves. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/using-systems-mapping-complex-evaluations-29th-march-2018 |
| Description | Policy Evaluation For A Complex World |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN: Policy Evaluation For A Complex World St Martin-in-the-Fields, Trafalgar Square, London 13:30pm onwards, Tuesday 13th September 2016 You are warmly invited by Prof. Nigel Gilbert, Director of CECAN, to an afternoon to consider Policy Evaluation For A Complex World. Introduced by Jane Elliott (Chief Executive, ESRC), with a keynote speech from Sir Mark Walport (Government Chief Scientific Adviser) and a panel discussion chaired by Roger Highfield (Dir External Affairs, Science Museum). The panel members are; Sir Amyas Morse (Comptroller and Auditor General of the National Audit Office), Dame Margaret Hodge (MP), Dr David Halpern (Behavioural Insights Team), and Dr Ulrike Hotopp (Simetrica Ltd). This event will entertain, engage and excite anyone with an interest in improving methods of policy evaluation across the energy, environment and food Nexus. The panel will encourage debate on the Nexus policy areas at the heart of the Centre's work, and raise critical questions around complexity, evaluation methodologies, and 'what works in practice'. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.cecan.ac.uk/events/13th-september-london-cecan-event-policy-evaluation-for-a-complex-worl... |
| Description | Policy Modelling - 2-slot session @ ESSA Special Interest Group (SIG) - Professor Petra Ahrweiler |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Policy Modelling - 2-slot session @ ESSA Special Interest Group (SIG) - 70 people - International Audience - Professor Petra Ahrweiler, speaking on behalf of CECAN |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://ssc2016.cnr.it/ |
| Description | Presentation at 5th European Environmental Evaluators Network Forum - 15/16 Sept |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Candice Howarth presenting at 5th European Environmental Evaluators Network Forum - 15/16 Sept |
| Year(s) Of Engagement Activity | 2016 |
| Description | Presentation on CECAN and Nexus Shocks - Dr Candice Howarth - UKADR Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Dr Candice Howarth gave a presentation at the UKADR conference in January 2017 about CECAN and her Nexus Shocks research. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.kcl.ac.uk/sspp/departments/geography/research/Activity-Hubs/CIRRR/UK-Alliance-for-Disaste... |
| Description | Presentation to the World Wildife Fund |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Gave talk to WWF: Interest in helping to test methods and tools; Interest in applications to WWF projects - e.g. land use dilemmas relating to climate change and resilience; trade-offs and ranking of priorities concerning complex conservation challenges; decision-making under uncertainties not only in relation to social systems but also in scientific analysis of physical systems; Offering speakers for CECAN events; possible interest in Fellowship; joint funding bids for further work; potential for some commissioning of work from CECAN . |
| Year(s) Of Engagement Activity | 2016 |
| Description | Professor David Byrne - presentation at Lisbon event - Interdisciplinary Futures: Open the Social Sciences 20 Years Later - using CECAN as an example of applied inter-discipinary work |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | David Byrne - presentation at Lisbon event - Interdisciplinary Futures: Open the Social Sciences 20 Years Later - using CECAN as an example of applied inter-discipinary work |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://ifoss20.wordpress.com/ |
| Description | Professor Ian Christie - talk on CECAN for Green Alliance think-tank, London; 3rd November 2016 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Professor Ian Christie - talk on CECAN for Green Alliance think-tank, London; 3rd November 2016 - Follow up meeting with Green Alliance being arranged. |
| Year(s) Of Engagement Activity | 2016 |
| Description | Professor Nigel Gilbert gave a presentation about CECAN to the ESRC 'all staff seminar' on 23rd February 2017 approx 50 people. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Supporters |
| Results and Impact | Professor Nigel Gilbert gave a presentation about CECAN to the ESRC 'all staff seminar' on 23rd February 2017 approx 50 people. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Professor Saville Kushner gave a talk on behalf of CECAN at the Food Standards Agency on 9th October. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Professor Saville Kushner gave a talk on behalf of CECAN at the Food Standards Agency on 9th October. In asking evaluators to define 'success' and 'failure' we ask too much - the range of views and criteria is too diverse and volatile to pin down; to limit the evaluator to measuring pre-specified results and to ignore life in programmes - aspirations, competitions, values, politics, humanity - is asking too little. We have, as a result, focused too hard on programme productivity and not enough on programme quality. But here we are. Over the past 20 years the field of evaluation has, arguably, advanced by shedding ideas and setting aside challenges and narrowed itself into a cul-de-sac of outcomes measurement and impact assessment - unwittingly cast itself as the outrider of public sector decline. As the austerity project nears its end, and as the country starts once again openly to question what counts as public value, is this an appropriate moment to return to Lee Cronbach's (1980) seminal proposal to reform the field? |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/news/cecan-seminar-we-ask-too-much-and-too-little-of-program-evaluators |
| Description | RDPE Workshop (Defra Case study) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Observations and learnings from Workshop 1 (evaluating current policy): ? The session was attended by Defra policy analysts as well as representatives from the Forestry Commission, Natural England and the devolved nations. This generated a lot of cross-learning. ? Attendees seemed to understand that they were dealing with something complex but struggled to articulate it well. ? Attendees were introduced to complexity appropriate evaluation methods to use in the commissioning process. This was done by working through Barbara Befani's evaluation toolkit (available on the CECAN website), which allowed participants to better understand the questions they were asking and therefore which methods would fit best. ? Initially participants were mainly interested in experimental / quantitative methods but by the end of the workshop they saw that theory-based methods were far better placed for answering their questions. ? Many participants were introduced to configurational theory-based methods for the first time. ? By the end of the workshop, participants understood that there was no single "gold standard" method for evaluation and that often a combination of methods is the best approach. ? Participants engaged well with the evaluation toolkit and it has been suggested in the post-workshop interviews that there is interest in arranging for Barbara Befani to teach evaluation teams about using the tool. ? The toolkit has provided a way of approaching this policy area through the introduction of complexity appropriate methods and building capacity in them. This is not something that would otherwise have happened. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/news/cecan-case-study-defra-rural-development-programme-for-england |
| Description | RDPE Workshop 2 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Purpose: To use a systems mapping approach to help Defra, through a process of co-creation, map out the "Future Agriculture, Environment and Rural System." Specifically, policymakers and analysts from four key business areas (Animal and Plant Welfare, Rural Economies and Communities, Transition and Productivity & Risk and Resilience, and Environmental Land Management) came together, for the first time in this format, to engage in jointly mapping out their policy areas from a "whole-systems" perspective, considering the connections and inter-dependencies between each other's policies and the wider system (from both a policy design and evaluation standpoint) - putting their policies in a real-world and complex systems context. This workshop will be followed-up by further work to digitize those maps so that they can be refined and analysed, and used by Defra as a platform to further develop their ongoing policy work - in terms of both policy development and policy evaluation - across these areas in a more integrated and coherent way. Impact statement from Defra: "CECAN recently supported us to create a systems map to aid the design of a complex evaluation framework for a very large programme of policy changes. The CECAN team were invaluable in the project; helping us understand the value of this approach in our complex policy setting, how the maps are produced and can be used, and took us step by step through the project, aiding us to gather the policy insight needed so that the project was tailored to our programme needs. As a project manager I found the team a pleasure to work with, their experts were genuinely interested in our project and dedicated a huge amount of effort to making it a success. As part of the project they led all the planning and running of a large workshop with officials, which allowed my team to obtain valuable products for the evaluation and also allowed delegates to take away insights and ideas for how to better their own work areas. Without CECAN we would never have had the capacity to generate this valuable tool for evaluation and policy making and our projects will certainly be of a far higher quality due to their involvement." |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/index.php/index.php/case-studies/cecan-case-study-rural-development-programm... |
| Description | RGS-IBG Annual Conference, London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Candice Howarth presented 'Enabling more effective policy making around nexus issues: the role of the Centre for the Evaluation of Complexity Across the Nexus' at the RGS-IBG Annual Conference, London |
| Year(s) Of Engagement Activity | 2016 |
| Description | Re-launch of the CECAN Website following re-design project |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | A revised CECAN website that is more considered than the previous site, it looks better, works well and is being accessed by lots of people. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.cecan.ac.uk/ |
| Description | Relating societal expectation structures and opinion dynamics to actual policies and developments in facts and figures - How the discourses around New and Emerging Technologies (NEST) and Responsible Research and Innovation (RRI) shape the European agenda |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Petra gave a talk at this conference on behalf of CECAN - "Identification, location and temporal evolution of topics. Data and algorithm - comparison of approaches" |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://knowescape.org/budapest-workshop-identification-location-temporal-evolution-topics-august-29-... |
| Description | Richard Gunton (CECAN Fellow) and colleagues from CECAN presented Richard's work on the Pluralistic Evaluation Framework at a workshop at Defra on 12th June 2018 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | The pluralistic evaluation framework is a high-level approach to policy design, appraisal and evaluation that aims to be balanced, accountable and compatible with existing approaches such as those advocated in the UKNEA. It is based on a comprehensive theory of valuing as presented by Richard Gunton and colleagues in a 2017 publication (Trends in Ecology & Evolution 32:249-257). The pluralistic evaluation framework offers an alternative to the impersonal focus on ecosystem services or different kinds of "capital", instead focusing on stakeholders and their values. This is highly policy-relevant as the Government's 25-year Environment Plan seeks to roll out a natural capital approach to environmental management while the Clean Growth Strategy simultaneously focuses on financial capital and carbon - all under the pressure of other societal challenges.This workshop will provide an introduction to the pluralistic evaluation framework and the thinking behind it. Insights from the workshop will also feed in to further tool refinement. |
| Year(s) Of Engagement Activity | 2018 |
| Description | Simulating Knowledge Dynamics in Innovation Networks (SKIN) 2-slot session at SSC2016 - Professor Petra Ahrweiler |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Date: 19-23 September 2016 Title: Simulating Knowledge Dynamics in Innovation Networks (SKIN) Type of activity: 2-slot session at SSC2016 International - Number of people engaged with: 50 - Professor Petra Ahrweiler on behalf of CECAN |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://ssc2016.cnr.it/ |
| Description | Sustainability in Turbulent Times (SITT 2017) Event held in conjunction with the Nexus Network and CUSP. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Sustainability in Turbulent Times (SITT 2017) Event held in conjunction with the Nexus Network and CUSP. Approx audience of 200 UK national policy makers, researchers, analysts, sustainability experts. This one-day, agenda-setting conference will bring together researchers, policy makers, business and civil society leaders to take stock and consider ways ahead. Drawing on the activities, networks and findings of three flagship ESRC programmes: The Nexus Network; The Centre for the Understanding of Sustainable Prosperity (CUSP), and The Centre for the Evaluation of Complexity Across the Nexus (CECAN), the conference will debate what interdisciplinary research can contribute to the future sustainability of a post-Brexit Britain and an uncertain world. Leading researchers, policymakers and practitioners will explore the future of environmental policy, the relationship between inequality and sustainability; the role of expertise and emotion in policy making; and practical approaches to navigating complexity. Throughout, there will be a particular focus on connections at the food-water-energy nexus, drawing on recent ESRC-funded activities in this area. The conference will be relevant to anyone concerned with how research, policy and practice can help to tackle today's and tomorrow's biggest global challenges. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/sustainability-turbulent-times-16th-march-2017 |
| Description | Systems Mapping workshop (Renewable Heat Incentive (RHI) Case Study with BEIS), May 2018 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | CECAN has run a successful systems mapping workshop at BEIS with a range of stakeholders from industry and government, looking at biogas and biomethane production and usage, as part of our case study on the Renewable Heat Incentive (RHI). Pete Barbrook-Johnson, Alex Penn and Anna Kaxira helped develop a systems map of the biogas and biomethane production/use system. Subsequent work included working on verifying and refining the map and an associated report, which BEIS will use to: (i) inform theory and data collection in the ongoing evaluation of reformed RHI and (ii) inform future policy planning in relevant policy areas. https://www.cecan.ac.uk/case-studies/beis-and-renewable-heat-incentive-understanding-complex-applications |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/case-studies/beis-and-renewable-heat-incentive-understanding-complex-applica... |
| Description | The Pluralistic Evaluation Framework: A Workshop for People in Policy Impact Appraisal, Tuesday 3rd July 2018, University of York (Prof. Sue Hartley, CECAN/YESI, Dr Ian Christie, CECAN, Dr Richard Gunton, CECAN, Prof. John O'Neill, Manchester Univ.) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | When a problem is to be solved in a complex environment with a diversity of stakeholders, how can a policy be democratically justified as "good" use of public funds? How can we appraise policies in a more concrete way than resorting to vague general value judgements such as "enhanced" and "improved"? A comprehensive policy evaluation ought to analyse what good outcomes might mean to different stakeholders in a given context. The pluralistic evaluation framework is a high-level approach to policy design, appraisal and evaluation that aims to be balanced, accountable and compatible with existing approaches such as those advocated in the UKNEA. It is based on a comprehensive theory of valuing as presented by Richard Gunton and colleagues in a 2017 publication (Trends in Ecology & Evolution 32:249-257). The pluralistic evaluation framework offers an alternative to the impersonal focus on ecosystem services or different kinds of "capital", instead focusing on stakeholders and their values. This is highly policy-relevant as the Government's 25-year Environment Plan seeks to roll out a natural capital approach to environmental management while the Clean Growth Strategy simultaneously focuses on financial capital and carbon - all under the pressure of other societal challenges. This workshop will provide an introduction to the pluralistic evaluation framework and the thinking behind it. Insights from the workshop will also feed in to further tool refinement. By the end of the day, participants can expect to gain: Techniques, skills and a template for policy design, including familiarity with a general values framework for assessing any policy issue Improved skills for more transparent public communication on policy development Progress in their current areas of policy development and evaluation Ideas and opportunities for the interpretation and implementation of the 25-Year Plan and other high-level documents, operationalising their "values" emphasis. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.cecan.ac.uk/events/pluralistic-evaluation-framework-a-workshop-for-people-policy-impact-... |
| Description | UK Evaluation Society Annual Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | UK Evaluation Society Annual Conference - 27April, London Invited presentation: Evaluating Complexity |
| Year(s) Of Engagement Activity | 2016 |
| Description | Webinar Dr Nick Hart, George Washington University - 9th January 2017 - "Improving Environmental Evidence: Barriers and Facilitators of Evaluation Capacity at the US Environmental Protection Agency" |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | In this webinar, Nick will discuss recent research on policy evaluation and the use of environmental evidence within the U.S. Environmental Protection Agency (USEPA). Since the inception of the USEPA, considerable emphasis has been placed on the use of policy analysis tools that aim to prospectively inform environmental policy decisions, including cost-benefit analysis and risk assessment used for regulatory actions. However, compared to the amount of such ex ante analysis conducted at the USEPA before a decision is reached, relatively little evaluation of these same environmental policies is produced after implementation to inform future policy development or to modify existing policies. This original research applies accountability theory and organizational learning literature in order to identify and explain unique institutional factors that affect USEPA evaluation supply to inform future efforts. Through a series of mixed methods case studies, this research seeks to inform efforts aimed at improving the quality of environmental evidence within the USEPA-through evaluation and systematic reviews-in order to better inform decision-making and achieve desired environmental outcomes. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.cecan.ac.uk/events/improving-environmental-evidence-barriers-and-facilitators-of-evaluati... |
| Description | Workshop with DECC / BEIS |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Meeting with DECC/BEIS to discuss Dependency Modelling and Causal Tracing 14th July (Chris, Barbara, Helen and others) |
| Year(s) Of Engagement Activity | 2016 |
| Description | keynote 'Complexity, Mixed Methods and Time'. Moving Beyond the Linear Model: The Role of Mixed Methods Research in an Age of Complexity |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Emma Uprichard, University of Warwick, gave a keynote 'Complexity, Mixed Methods and Time'. Moving Beyond the Linear Model: The Role of Mixed Methods Research in an Age of Complexity, Durham University, UK. 3-5 August. |
| Year(s) Of Engagement Activity | 2016 |
| Description | meeting at the Royal Academy of Engineering on Systems approaches in engineering and government decision making, |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | NIgel chaired a meeting at the Royal Academy of Engineering on Systems approaches in engineering and government decision making, 1 July 2016 |
| Year(s) Of Engagement Activity | 2016 |