Evaluating Cyber Security Evidence for Policy Advice: The Other Human Dimension

The quality of a state's capacity to respond to the challenges of cyber security is rapidly coming to be recognised as an important element of global competitiveness. This project seeks to understand the challenges faced by the UK's policy making community in interpreting, evaluating and understanding evidence about cyber security. Policy makers, sometimes with little relevant expertise and often in time-critical scenarios, are asked to assess evidence from a mix of sources including official threat intelligence, academic sources, and industry threat reports. Such a diverse evidence base is then used to make judgments on threat, risk, mitigation and consequences, and offer advice shaping the national regulatory landscape, foreign and domestic security policy, and a range of public and private sector initiatives. This element of the human dimension has significant relevance for the cyber security of the UK and is the main focus of this proposal.

Assessment of evidence is a particular problem for policy making in this context for three reasons:

First, some of the evidence is contradictory and/or potentially carries within it particular agendas or goals that may impede upon its rigour and reliability. The 'politicisation' of cyber security evidence is increasingly problematic as states sometimes privilege threat intelligence from sources located within their sovereign borders rather than based on the quality of the research they produce;

Secondly, it has proven to be extremely difficult to conclusively attribute cyber attacks and to quantify the cost of cyber insecurity. These challenges mean that evidence can only support policy makers' decisions and evaluation of cyber security risks, threats and consequences to an extent;

Finally, the landscape of cyber security is developing rapidly and spans many issue areas including national security, human rights, commercial concerns, and related infrastructure vulnerabilities. Consequently, policy makers must work to balance a range of sometimes conflicting interests that compete for attention and they must do so in a field with little precedent to draw upon.

When exploring the problems (and possible remedies) of the human dimension of cyber security, many focus on end users. While this is important, equally important is the human dimension of decision making and advice offered by civil servants who collectively influence policy level responses to cyber threats. This project focuses on policy makers in the UK, specifically those civil servants who provide short and long term policy advice, either in response to specific crisis incidents or in the context of longer term planning for capacity building. This cohort is of particular importance given:

- the unique set of technological, behavioural and policy challenges they currently face. They are a relatively small and disparate group, possessing varying levels of technical and behavioural experience;
- their responsibility and impact goes well beyond their own organisations to shape the national and international landscape; and finally,
- the lack of research to support this particular community, either in identifying specific challenges they face or in developing more effective mechanisms for doing so.

This leads to several questions: what evidence do UK policy makers rely upon in this context? What is the quality of that evidence? How effective are the judgements about threats, risks, mitigation and consequences based on that evidence? Understanding how UK policy makers select evidence, why they privilege one source over another, and how adept they are at recognising possible weaknesses or flaws in evidence is central to addressing these questions.

The project stands to benefit the following groups; academia (as outlined in the Academic Beneficiaries section), UK and global policy makers, and university teachers and students.

UK Policy Makers:
For UK policy makers, WP 4.3 provides direct input to various government entities through a number of policy briefings (up to five) with the participants of the policy crisis games and any of their colleagues who wish to attend. This kind of small group briefing will allow for maximum knowledge transfer as there will be a facilitated question and answer session. Identified HMG departments include the GCHQ, DCMS, DCLG, MoD and FCO. Briefings will be coordinated for delivery at convenient venues for maximum participation.

The project also delivers impact through professional development of the participants of the cyber policy crisis game. Rehearsing them for crisis response is not the focus of the game but it will be an outcome nonetheless. In addition, they will gain insight into their own work practices, path dependencies and preferences that will help them to develop a professional self-awareness and critical approach to their work. Relevant users with whom we have had discussions about the direct benefits to their staff include GCHQ/CESG, DCMS, DCLG, Foreign Commonwealth Office, CERT UK, the Cabinet Office, Ministry of Defence and the Centre for Cyber Assessment, the letters of support from some of whom are attached as confirmation.

Global Policy Makers:
In terms of global policy makers, the focus of WP 3.2 is to develop the findings of the project into a framework for assessment that can be used in an ongoing process of evaluating and supporting cyber policy advisors in the UK but also for the private sector (at board level) and as a capacity building program for those in other countries. Part of the UK's National Cyber Security Strategy is to build capacity in key countries. By equipping policy advisors in other countries to better assess threat, risk and consequences from their own and shared evidence bases, the UK can effectively enhance its own capacity to address the human factor in cyber security.

University teachers and students:
University teachers and students will benefit from the curriculum development and pedagogical sharing that is built into the project. The PI has developed a new multi-disciplinary Masters in Digital Technologies and Global Politics to be delivered at Cardiff University from September 2017. One of the core modules is Cyber Security: Strategy, Policy and Regulation which was written to fulfil aspects of the GCHQ Cyber Security accreditation requirements. The research findings from this project will inform the teaching of this module and given that many of the applicants will be considering working in policy positions, there is a valuable continuum of knowledge transfer through the project and onto the next generation of UK cyber security policy makers. The Co-PI is involved in teaching an Ethical Hacking and Cyber Security module at Coventry University, and will incorporate elements of the policy game into capture the flag (CTF) exercises already conducted to inform the student experience on non-technical issues around social acceptability, public policy, and law. The scenarios from the policy crisis game will be annotated and posted on the project website for others to use and adapt in their teaching.