Decision-Making under Uncertainty: Developing a Deeper Conversation

Lead Research Organisation: University College London
Department Name: Division of PALS

Abstract

Recent events have made clear that many decisions must be made in uncertainty - that is without the luxury of the requisite time, information or understanding to allow the decision-maker to know if s/he is making the best choice at the moment it is made.
However, academic analyses of uncertainty and decision-making vary considerably between disciplines. They differ in their relevance to real world decisions.

The goal of this network proposal is to generate a funding proposition to allow Research Councils UK to form a permanent cross disciplinary Decision-Making Under Uncertainty Research and Evidence Hub (the 'Hub') made up from a wide range of academic experts and practitioners able to work in close proximity. The Hub's function wold be to co-ordinate best available expertise (or to commission it) and to offer government, industry and civil society partners tools to facilitate resilient and effective decision-making across a full range of real world uncertainty and practice contexts. Publishing relevant materials and manuals via a website, the Hub would meet the need for immediate, intermediate and long term consultation, research and training responses to practitioner requests.

The network proposed will lay the foundation for the Hub by delivering a draft guide to "Uncertainty Triage", and a "Code of Practice" for making resilient decisions under uncertainty. "Uncertainty Triage" will be designed to identify what aspects of a given real-world challenge are (a) likely to fall into traditional statistical/decision theoretic methods, (b) require a broader approach as mature probabilities are not available, and (c) those in which radical uncertainty dominates so that a case-by-case reflection is required. The Code of Practice will incorporate known hazards facing real-world decision making and will evolve to incorporate lessons derived from experience along the lines of airline pilot safety manuals.

We will implement the network through interaction between the PI (a social scientist and psychologist) and the CI (a mathematician and statistician) building on the expertise of a widening core group drawn from key practitioner collaborators and academics from very different disciplines, linked to a series of national and international centres of excellence.

There are nine steps over 24 months:
i. Engaging and integrating into the project a core group of practitioners from government and industry by undertaking with some of the academics a series of limited case studies describing in detail core features of some real world decision-making.
ii. A kick off conference addressed by experienced policy-makers and thinkers to introduce practitioners and academics to each other and the issues.
iii. Focusing the Core Group to explain their own frameworks for grappling with the cases collected above to people from outside their field.
iv. Organizing one-on-one or small group follow up meetings with practitioners to develop and deepen analysis of the specific challenges.
v. Broadening the core network group to those interested and willing. A website (and blog) will be created.
vi. Organizing an open conference to report progress and launch a call to apply to undertake feasibility projects. Up to £15,000 for 6-8 short projects of 6 months approx.
vii. Continue Core Network meetings (including a two-day retreat to deepen understanding) to draft and test the Uncertainty Triage and Code of Practice.
viii. Hold an open and highly interactive workshop conference (22 months) to discuss the feasibility projects and other emerging material (draft Triage and Code of Practice) to evaluate it from the practitioners' viewpoints.
ix. Produce Conclusions edited into long and short reports published on the website (and in book form) including a funding proposal to form the network into 'the Hub'.

The financial crisis, the Brexit decision and climate change create uncertainty. This network aims to help us deal with it.

Planned Impact

The aim of our network proposal is to put knowledge co-creation at the heart of our implementation. It means that we put impact now and in future at the heart of the proposal.

From Day 1 we will create interactions and deep conversations between academics and practitioners designed to create impact - to help practitioners see the value of different academic viewpoints and to help academics to develop new tools and approaches that more accurately describe, analyse or are useful in practice.

In fact, planning the proposal has already created initial conversations that have opened eyes among academic and practitioners starting with ourselves. They have made a difference which will be hugely amplified and leveraged in the formation of the network: there will be increased clarification and understanding between the PI and CI (as representatives of their several disciplines) and between practitioners (decision-makers in industry and government agencies) and academics drawn from a very wide range of disciplines across mathematical, physical, social and psychological science. In discussions to create the proposal we have already selected 27 academics (Case, p6) and 12 practitioners (Case, p5) as well as establishing relationships with 16 first class academic centres (below) and reaching another 40 or so experts elsewhere. The centres are hubs in their own often international networks and will be at the core of knowledge creation and onward dissemination. We have also spontaneously been contacted by others who know about the call. We will use these foundations both to construct a network and eventually to support the Hub.

A network website (which UCL have undertaken to maintain for at least 5 years even if the Hub proposal is eventually not funded) and mailing list will be at the centre of our network. Early on we will publish position papers and information and our prototype versions of the Uncertainty Triage and Decision-Making Code of Practice. The site will grow and be open for comment with an online form to allow requests for support. The further eventual aim is to produce both a book and academic articles and a report aimed at practitioners. We intend seeking sponsorship for the latter so that it can be produced along the lines of the LSE Munich Re programme report.
The hub (The RCUK Decision-Making Under Uncertainty Research & Evidence Hub) is what we expect the network to deliver clear feasibility-supported proposals about. It would aim to be a resource fir UK government, industry and civil society and an ongoing source of impact for UK research. Co-ordinating expertise and responding to requests from government and industry needing advice it aims to create a reputation that t can facilitate resilient and effective decision-making across a full range of real world uncertainty and practice contexts. If funded the hub would take over our network website and collate and publish relevant materials and manuals to meet the need for immediate, intermediate and long term consultation, research and training responses. The hub would be linked to the Centres we engage (below) and others and seek to work at three time horizons:

An immediate response (days/weeks) function would be expected to use (and further develop) the triage procedures and code of practice which we intend to be "living" tools that are adjusted with experience and feedback.
An intermediate response (weeks/months) would use more experts such as those within the 'core' and from the teams and centres of expertise in the wider network (see below). We envisage they would conduct inter-/multi- disciplinary academic workshops to explore the features of the problem with researchers invited to attend and observe or deliver short presentations on the insights that their research or discipline can offer.
The considered response (months/1-2 years) would be relevant to the commissioning of further research from teams or centres of expertise in the wider network.

Publications

10 25 50
 
Description The network has produced a Landscape and Research Agenda for Real Wolrd Decision-Making under Uncertainty document for UKRI and outlines the main findings at its 2018 conference.It is currently being prepared for a paper to be submitted to the Royal Society in collaboration with colelagues from thr Max Planck Institute for Mathematics in the Sciences but can be summarised as follows:

The CRUISSE network was set up by three UK research councils in "response to a demonstrable call" for support "from business and government" and funded by EPSRC for two years from January 2017. Made up of academics and policy makers with a wide range of backgrounds in physical, mathematical, social, political and psychological science , its task was to report on the current research landscape of real-world decision-making under uncertainty and suggest a future research agenda. To summarise the findings and recommendations:
1. Most significant decisions made in government, industry and the third sector do not have a clear, optimal solution at the time they have to be made. They are made in radical uncertainty.
2. Trying to find an optimal answer can be misleading so that relying on models that look to provide oner, which will often poorly represent the problem, can give rise to unintended consequences.
3. Most decision-making research is based around an Optimal Choice Framework (OCF) - using a probability-based approach to determine the best response. Formal OCF frameworks are useful for answering a large class of problems or exploring possible options, however there is often a clear imbalance between the demands of policy decision-makers and the decision support tools that academic research can provide.
4. This imbalance is due to the widespread silo approach to academic research where research questions are chosen to suit narrow academic agendas. This means there is a lack of institutions focussing on real world decision-making and that there is currently no multi-disciplinary research centre or academic group that can support government or business make these complex decisions outside of the OCF framework.
5. The seven small CRUISSE pilot projects [http://cruisse.ac.uk/completed-projects/projects/] funded during 2018 provided insights into how a new approach would work. They took real-world world relevant and academically exciting research problems, co-created between users and academics, and demonstrated that co-operation across multiple disciplines is a powerful approach to address these complex decision-making processes.
6. The CRUISSE report therefore proposes the formation of a new UK networked institute to build capacity in this area. This institute would: respond to requests for support across government, business and the third sector; develop a new research programme focussed on the underpinning science of decision-making under uncertainty; and build much needed capacity and capability in this area.
7. This would include: a better understanding of the roles of emotion, narratives, collaborative group processes in decision-making; quantification and uncertainty communication across socio-cultural boundaries; the role of expertise; how to learn from mistakes; and the selection of adaptive heuristics.
8. This is not an area where the problem of decision-making in complex environments can be approached through the lenses of different disciplines and then combined. It is clearly a cross Research Council issue that does not fall under the auspices of any individual Council. It is therefore the perfect example of a problem that needs to be approached using the new cross-Council structures.
9. If successful, the CRUISSE proposals would build on the UK's existing academic and consultancy expertise across a wide range of disciplines, give the UK a clear national advantage, and provide significant export potential.
Exploitation Route We have made a proposal to UKRI to establish a permanent intitute-centre-network to co-create research in this area and to address the new research questions we identified. We are in discussion with UKRI, Government Science Advisers and Business as to how to take this forward.
WE have organised a conference in collaboration with the Royal Society to bring these issues and out findings to public attention. See https://royalsociety.org/science-events-and-lectures/2020/04/radical-uncertainty/#
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Chemicals,Communities and Social Services/Policy,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Energy,Environment,Financial Services, and Management Consultancy,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Government, Democracy and Justice,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology,Retail,Security and Dip

URL http://cruisse.ac.uk/
 
Description During the grant period findings from the network activity or the 7 pilot projects weren used by the UK Cabinet Office, several government departments and by businesses attending the Global Solutions Summit in Berlin 2018 and 2019. More specifically they have been used by the START Network, several Pakistan NGO networks, by those affectied by extreme heat in Pakistan,by insurance companies,by Ove Arup and by PwC strategy. We are also in discussion with Chief Government Science advisers in Security, Transport, BEIS and others. A meeting to discuss the ideas and conclusions was to be held at the Royal Society (under their Hooke plus scheme) in April 2020. Over 200 people were enrolled inclusion g many from government service. But it was postponed by the pandemic. https://royalsociety.org/science-events-and-lectures/2020/04/radical-uncertainty/ It is now scheduled for 2022 Meanwhile, two papers applying the emerging thinking to monetary policy and the UK Government's Levelling Up agenda are publushed. • David Tuckett, Douglas Holmes, Alice Pearson and Graeme Chaplin. Monetary policy and the management of uncertainty: a narrative approach. Bank of England Staff Working paper No 870. June 2020. • Collier, Paul and Tuckett, David. (2021). Narratives as Mechanisms for Building Forward-Looking Common Purpose in Britain's City-Regions. Oxford Review of Economic Policy (Forthcoming) The ideas also played a major role in workshops organised by the Cabinet Office, UK Treasury, Ministry of Housing, Communities and Local Government, the Ministry of Transport, and the No 10 Policy Unit in January 2020, September 2020 and November 2020 in relation to the Government's Levelling Up agenda. They are under active consideration (March 2021). See Tim Besley, Paul Collier, Hilary Cottam, Colin Mayer, Sandra McNally, Philip McCann, David Tuckett and Tony Venables (2020) Levelling-Up the British Economy: A Workshop presenting recent research to the Cabinet Office, No 10 Policy Unit and HM Treasury, January 31st, 2020. Discussions are also ongoing with HMG's security advisers. In the last 18 months additional ;papers have been published, Erica Thomson won an UKRI Future Leaders Fellowship to pursue aspects of the work, and we gave evidence to the Cabinet Office consultation on National Resilience Policy A well-attended Royal Science plus meeting has took place on October 3-4, 2022. Details here: https://royalsociety.org/science-events-and-lectures/2022/10/radical-uncertainty/
Sector Aerospace, Defence and Marine,Communities and Social Services/Policy,Construction,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Energy,Environment,Financial Services, and Management Consultancy,Government, Democracy and Justice,Manufacturing, including Industrial Biotechology,Security and Diplomacy,Transport
Impact Types Cultural,Societal,Economic,Policy & public services

 
Description Contribution to Bank of England MPC and UK Government Policy Levelling Up the Regions
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Multi-stakeholder Communication in Flood Risk Management 
Organisation National Flood Forum
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution The aim of the project is to draw preliminary conclusions, based on a small number of UK case studies, about how to improve organisational and cultural behaviours in a multi-stakeholder environment with many different levels of understanding, from the first-hand practical knowledge of flooded residents to complex hydrological software used by flood modellers. How can these different communities talk constructively to each other about risk and uncertainty, to improve flood management and generate community resilience?
Collaborator Contribution NFF have consulted, are assisting the Research Fellow from the UNiversity of Exeter and await the outcome.
Impact The project has three types of value in addressing radical uncertainty in FRM. Firstly, it explicitly prioritised co-creation of this knowledge with community stakeholders. Given the sensitivity of the subject for many individuals who consented to participate, a collaborative approach was the only effective means for conducting such investigations. Secondly, it is evident from the research that radical uncertainty is not coherently considered in community FRM communication. The project is therefore pioneering in revealing much about how, potentially, this issue could be countered in the future: a matter of urgency given recent predictions on UK climate change impacts. Thirdly, the research shows how a transdisciplinary research process could support such a dialogue. Web search data highlighted the dominance of Optimum Choice Framework (OCF) responses to local FRM in the South West. Modelling of flood risks has formed the basis of Environment Agency flood risk maps and localised flood risk plans, while EA early warning systems rely on Met Office climate predictions. Cost-benefit analysis also drives government agency flood risk investment decisions. Although critical in highlighting potential and immediate flood risks, OCF approaches are unable to fully communicate radical uncertainty in FRM, particularly around long term climate change, sea level rise and land use. Moreover, current top down communication strategies based on OCF decision-making often miss vulnerable societal groups, necessitating new 'paradigmatic' thinking. In conclusion, our research demonstrates how a transdisciplinary agenda could help radically reshape our approach to FRM communication. Interviewees generally identified disconnection with conventional communication approaches, leaving vulnerable actors living effectively 'off the radar' from the current FRM governance system. While our study was necessarily limited in scope, a scaling up of findings would infer a sizeable proportion of the population are similarly disengaged: which is concerning given recent climate change predictions. Coping with radical uncertainty in FRM decision-making will, we argue, necessitate more qualitative, in-depth approaches to understanding this disconnection, using established empirical and theoretical tools such as narratives, role-play scenarios, experience-based anticipation, feedback learning and expert elicitation. Of particular interest to this area of study is the intuitive role of emotion and place-based attachment in FRM communication. Building the relationships of trust necessary for effective communication of risks under future radical uncertainty will require better understanding of individual stakeholder emotions (e.g. fear, happiness, distrust, anger) and also, as a result, the intrinsic values they place upon flood threatened property.
Start Year 2018
 
Description Open University Course no Decision-Making: Management of Uncertainty: Leadership, Decisions and Action. A description and access to booking on the course is here https://www.futurelearn.com/microcredentials/management-of-uncertainty-leadership-decisions-and-action 
Organisation Open University
Country United Kingdom 
Sector Academic/University 
PI Contribution A team member designed the course a d member bars f the team gave interviews and helped create materials.
Collaborator Contribution Management of Uncertainty: Leadership, Decisions and Action. A description and access to booking on the course is here https://www.futurelearn.com/microcredentials/management-of-uncertainty-leadership-decisions-and-action A brief description of content is below. This microcredential will support you in developing the skills, capabilities and perspectives needed in a world of radical uncertainties. You'll develop new ways of engaging with radical uncertainties, think critically about how uncertainties are distributed in societies and organisations, develop greater resilience in the face of radical uncertainty and unexpected events, and grow your own capacity for leadership and action. What topics will you cover? Exploring the different consequences of risk versus radical uncertainty Taking account of the impact of error culture and defensive decision making on managing uncertainty Avoiding the traps of confusing risk and radical uncertainty Exploring the construction of certainty and uncertainty in society and organisations Exploring the role of stories, imagination and emotion in the management of uncertainty Working with modelling, scenarios and big data whilst avoiding their pitfalls in the context of uncertainty Organising for radical uncertainty: collective mindfulness and organisational resilience Managing the unexpected Leading and influencing in the face of radical uncertainty Understanding and developing your own capabilities for tolerating uncertainty, coping with uncomfortable knowledge, and wise reasoning Developing socially responsible approaches to the management of uncertainty Making use of course ideas in your own context Planning your own personal development for leading and managing in the context of radical uncertainty and unexpected events Who is the course for? This microcredential would benefit anyone currently working, or looking to work in: Senior roles in business, the public sector or third sector Strategic analysis or consulting New product development Managing or commissioning large complex projects The course delivers transferable skills for any industry and is relevant across the board. This course has a global focus and the skills you'll learn will be relevant for learners from countries around the world. To join this microcredential you will ideally have: A Bachelors degree or an equivalent level qualification Experience of working in a role that exposes you to some of the significant uncertainties that organisations face A strong interest in improving your capabilities in leading, managing, deciding and acting in the face of major uncertainties By the end of the course, you'll be able to... Identify the ways in which as individuals, organisations and societies we construct certainty and avoid uncertainty. Apply multiple perspectives on uncertainty and tools for engaging with uncertainty whilst still being able to act. Reflect critically on the uses, limitations and dangers of big data, artificial intelligence, and models, in contexts of uncertainty. Demonstrate a critical and ethical awareness of how uncertainty is socially distributed in society and in organisations and how your actions as a manager/leader can impact this. Reflect on and develop your own capacity for leadership under uncertainty. Evaluate assumptions critically, in planning, forecasting, modelling, and strategizing to support more mindful organising and greater resilience in the face of uncertainty. Demonstrate the following skills: critical analysis; perspective-taking; personal resilience; engaging with uncomfortable knowledge; tolerance of ambiguity and ambivalence; ethical awareness. Guest contributors ( a mix of academic experts and senior managers) and leaders include (CRUISSE associates highlighted): • Prof. Richard Badham • Sarah Barr (START Network) • Prof. Kate Booth • Sanjeeb Chaudhuri (Multiple senior and board roles) • James Crotty Senior manager (Multiple senior and board roles) • Prof. Gerd Gigerenzer • Mervyn King (Baron King of LothBury. Former Governor of the Bank of England) • Prof. Elizabeth King • Laurie Knell (Management consultant) • Bruce Lee (Senior Vice President IT Infrastructure and Operations for Centene Corporation) • Maggie Miller (Multiple senior roles in her career) • Nicholas Pole (College of Policing) • Rory Sutherland (Vice Chair Ogilvy UK) • Dr Erica Thompson • Prof. David Tuckett • Carolyn Wilkins OBE (Chief Executive Oldham Council)
Impact Multidiusciplinary
Start Year 2021
 
Description Studying Uncertainty in Decision-Making Processes of Rapid Humanitarian Disaster Relief 
Organisation Start Network
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution START, is partnering the the CruIsse network, which aims to bring together academics from a range of disciplines (mathematical, physical, psychological, social and other sciences), to undertake a project to better understand how allocation and project decisions are taken, with a specific focus on the conceptualisation and impact of uncertainty. This analysis will be used to revise START's anticipation alert templates and to trial the delphi decision-making method on allocation decisions. We expect that this will reduce bias, improving the decision-making process and the outcomes of these decisions.
Collaborator Contribution This is a joint project. We have provided a Research Fellow to work in their organisation and do particuonata observation.
Impact At present we have an intrenal report. A presentation will be given at the CRUISSE Conference in November 2018.
Start Year 2017