Communicating and evaluating cyber risk and dependencies
Lead Research Organisation:
City, University of London
Abstract
Industrial computer-based control systems are crucial to society, they control the water we drink, the power we use, the cars we drive as well as railways and air transportation. These systems need to be trusted and trustworthy. They are often networked into complex and interconnected systems of systems and control and protect the UK national infrastructure.
An important aspect of infrastructures is their interactions and interdependencies: the functioning of one infrastructures service often depends on the functioning of another. As the infrastructure becomes layered and there are secondary services layered on top of these primary infrastructures and as the network becomes dynamic and controlled by computer networks and systems there is considerable potential for unforeseen interaction and dependencies.
As Industrial control systems become more networked, the previous strategy of making them secure by isolating them from the world becomes ineffective. In addition those who might harm the system either out of maliciousness or misplaced curiosity proliferate and their expertise increases, so the importance of security for the availability and integrity of services and systems is becoming ever more significant.
The research focuses on the importance of dependencies and interdependencies in this security context. These have been studied for a number of years and it is known that unforeseen interdependencies are a source of threat to systems and an important factor in our uncertainty of risk assessment, particularly risk due to cascade failures in which the rate and size of loss is amplified.
However there two faces to interdependencies, while we are concerned about how they might make attacking the system easier and a source of unforeseen behaviours, it is also central to providing tolerance to attack and failure. Redundancy, diversity, defence in depth are deliberately engineered into control systems to increase dependability and are an important mechanism for adaptation and overall resilience.
Any risk assessment of computer based control systems has to take into account uncertainty about the structure of the system. It is not just the uncertainty of when events might happen but uncertainty about the world, so-called epistemic uncertainty. For example, audits for the US DHS states that they find, on average, 11 unexpected connections between the SCADA system and the enterprise network for each audit
A key part of risk assessment is communication to stakeholders and society as appropriate. We will develop a security informed (or cyber-informed) enhancement to evaluating and communicating business and other risks from lack of control system integrity and availability based on a claims, arguments, evidence (CAE) framework. Our focus will be to include cyber informed dependency analysis within these assessments. The research to do this will follow an impact driven, threat-informed and vulnerability-focused strategy.
We will also develop probabilistic models that address explicitly the evolving relationship between an adversary and attacks on the one hand and of the consequences of a successful attack as well as the dependencies between the mitigations and barriers. We are particularly interested in modelling and evaluating defence in depth as a fundamental part of any resilient and trustworthy system yet estimating its effectiveness given uncertainties in the system structure and the attack space is enormously difficult. We will develop a modelling toolset based on existing tools we have developed within EU, Artemis and TSB projects that integrate stochastic and deterministic (e.g. of power flow). We will conduct case studies based on problems provided by our project partners Adelard (a specialist SME that evaluates ICS systems and components) and Alsthom.
An important aspect of infrastructures is their interactions and interdependencies: the functioning of one infrastructures service often depends on the functioning of another. As the infrastructure becomes layered and there are secondary services layered on top of these primary infrastructures and as the network becomes dynamic and controlled by computer networks and systems there is considerable potential for unforeseen interaction and dependencies.
As Industrial control systems become more networked, the previous strategy of making them secure by isolating them from the world becomes ineffective. In addition those who might harm the system either out of maliciousness or misplaced curiosity proliferate and their expertise increases, so the importance of security for the availability and integrity of services and systems is becoming ever more significant.
The research focuses on the importance of dependencies and interdependencies in this security context. These have been studied for a number of years and it is known that unforeseen interdependencies are a source of threat to systems and an important factor in our uncertainty of risk assessment, particularly risk due to cascade failures in which the rate and size of loss is amplified.
However there two faces to interdependencies, while we are concerned about how they might make attacking the system easier and a source of unforeseen behaviours, it is also central to providing tolerance to attack and failure. Redundancy, diversity, defence in depth are deliberately engineered into control systems to increase dependability and are an important mechanism for adaptation and overall resilience.
Any risk assessment of computer based control systems has to take into account uncertainty about the structure of the system. It is not just the uncertainty of when events might happen but uncertainty about the world, so-called epistemic uncertainty. For example, audits for the US DHS states that they find, on average, 11 unexpected connections between the SCADA system and the enterprise network for each audit
A key part of risk assessment is communication to stakeholders and society as appropriate. We will develop a security informed (or cyber-informed) enhancement to evaluating and communicating business and other risks from lack of control system integrity and availability based on a claims, arguments, evidence (CAE) framework. Our focus will be to include cyber informed dependency analysis within these assessments. The research to do this will follow an impact driven, threat-informed and vulnerability-focused strategy.
We will also develop probabilistic models that address explicitly the evolving relationship between an adversary and attacks on the one hand and of the consequences of a successful attack as well as the dependencies between the mitigations and barriers. We are particularly interested in modelling and evaluating defence in depth as a fundamental part of any resilient and trustworthy system yet estimating its effectiveness given uncertainties in the system structure and the attack space is enormously difficult. We will develop a modelling toolset based on existing tools we have developed within EU, Artemis and TSB projects that integrate stochastic and deterministic (e.g. of power flow). We will conduct case studies based on problems provided by our project partners Adelard (a specialist SME that evaluates ICS systems and components) and Alsthom.
Planned Impact
The relevance and increasing importance of trust in Industrial Control Systems will provide enormous scope for impact. The potential for wider industrial and societal impact is articulated in the call for proposals: the specific work on interdependencies is also recognised in the UK Infrastructure Plan.
Although this is a research proposal we have designed it to have broader impact: there are short term impact from engaging with our industrial partners, longer term, through tackling important underlying technical issues of modelling defence in depth.
The proposers are well connected with the networks on dependability and security and impact will be sought via these channels. For example through our membership of the ISA working group on cyber and industrial control system standards and the work of the Open Group on Dependability of open systems. As the international reviewer we have collaboration with a large Japanese dependability project DEOS project and its industrial partners. Bloomfield and Bishop work closely with CPNi on other projects and are familiar with the industry forums that CPNI facilitates. We will brief these group as appropriate, providing as they do important points of contact with industry and also stakeholders in shaping and providing feedback to the project.
The industrial collaborators will be involved in the development; some of the work is directly addressing their concerns. Thus, we expect to test applicability of the research results to industrial practice. Practitioners in Adelard and having to deal with risk assessment of large scale ICS systems and there will be opportunities and demand for using any near term results. Although Adelard is an SME it has a software product (ASCE) used worldwide (ASCE) and this could provide significant leveraged impact of the results.
Recognising the scarcity of detailed case studies we would release in the public domain:
1. A version of the case studies we are going to be developed in detail and use in the project to validate the theories and new assessment methods.
2. A light version of the software tools for building hybrid models of industrial systems, their ICS and probabilistic descriptions of the adversary, asset defences and attacks together with the solvers of the models.
As Assistant Editor in Chief of IEEE Security and Privacy, Bloomfield is well placed to find dissemination routes for the results of the work either in his column for the magazine or in specific articles.
Academic impact will not only be achieved through the usual publication and conference channels but also by working with other cyber-related research institutes. For example, the need to model and establish connectivity of systems and software could exploit links to the Automated Program Analysis and Verification Research Institute, as could the need to make judgments of vulnerabilities. The need to address socio-technical issues within dependency analysis provides a natural link to some of the work on organisational security and decision-making proposed in the Science of Security Institute.
Although this is a research proposal we have designed it to have broader impact: there are short term impact from engaging with our industrial partners, longer term, through tackling important underlying technical issues of modelling defence in depth.
The proposers are well connected with the networks on dependability and security and impact will be sought via these channels. For example through our membership of the ISA working group on cyber and industrial control system standards and the work of the Open Group on Dependability of open systems. As the international reviewer we have collaboration with a large Japanese dependability project DEOS project and its industrial partners. Bloomfield and Bishop work closely with CPNi on other projects and are familiar with the industry forums that CPNI facilitates. We will brief these group as appropriate, providing as they do important points of contact with industry and also stakeholders in shaping and providing feedback to the project.
The industrial collaborators will be involved in the development; some of the work is directly addressing their concerns. Thus, we expect to test applicability of the research results to industrial practice. Practitioners in Adelard and having to deal with risk assessment of large scale ICS systems and there will be opportunities and demand for using any near term results. Although Adelard is an SME it has a software product (ASCE) used worldwide (ASCE) and this could provide significant leveraged impact of the results.
Recognising the scarcity of detailed case studies we would release in the public domain:
1. A version of the case studies we are going to be developed in detail and use in the project to validate the theories and new assessment methods.
2. A light version of the software tools for building hybrid models of industrial systems, their ICS and probabilistic descriptions of the adversary, asset defences and attacks together with the solvers of the models.
As Assistant Editor in Chief of IEEE Security and Privacy, Bloomfield is well placed to find dissemination routes for the results of the work either in his column for the magazine or in specific articles.
Academic impact will not only be achieved through the usual publication and conference channels but also by working with other cyber-related research institutes. For example, the need to model and establish connectivity of systems and software could exploit links to the Automated Program Analysis and Verification Research Institute, as could the need to make judgments of vulnerabilities. The need to address socio-technical issues within dependency analysis provides a natural link to some of the work on organisational security and decision-making proposed in the Science of Security Institute.
Publications

Bloomfield R
(2017)
Computer Safety, Reliability, and Security

Bloomfield R
(2017)
Preliminary interdependency analysis: An approach to support critical-infrastructure risk-assessment
in Reliability Engineering & System Safety

Bloomfield, R.E.
(2017)
Security-informed safety: integrating security within the safety demonstration of a smart device

Li Z
(2023)
Automatic search intervals for the smoothing parameter in penalized splines.
in Statistics and computing

Netkachov O
(2016)
Critical Information Infrastructures Security

Netkachova K
(2015)
Computer Safety, Reliability, and Security

Netkachova K
(2017)
Is Chocolate Good for You-or, Is the Cloud Secure?
in Computer

Netkachova K
(2015)
Computer Safety, Reliability, and Security

Netkachova K
(2016)
Security-Informed Safety
in Computer
Description | We have developed an approach to decision supported based on Claims Arguments Evidence framework that is applicable to security and cyber related decisions. We have used it on a variety of examples - a form of action based research - with partners in avionics, aviation regulation security and critical infrastructure modeling. |
Exploitation Route | We are developing guidance material and considering the best way to disseminate - please see narrative section. We have submitted research proposals in the area of assurance of autonomous systems and are developing proposals in the are of infrastructure modelling. We have a web site claimsargumentsevidence.org to disseminate the work. The work has evolved in to an approach dubbed Assurance 2.0 and is the basis of a DARPA project on automated certification. It has also been used in research for dstl. Safety case Templates for Autonomous systems, full report at http://arxiv.org/abs/2102.02625 Assurance 2.0 Manifesto https://arxiv.org/abs/2004.10474 |
Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Energy,Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice,Security and Diplomacy,Transport |
URL | http://www.claimsargumentsevidence.org |
Description | The CAE Blocks framework - a way of structuring arguments will appear in the forthcoming (2018) IAEA Software Dependability Assessment guideline. Some findings are being used in the SME Adelard LLP on their projects on assessing security informed safety of industrial systems and in the development of codes of practice for security informed safety for the rail industry and in BSI PAS for connected autonomous vehicles. In addition there was the development of a regulatory cyber maturity model for aviation air traffic management (CAA). The PIA-FARA stochastic modelling has informed a follow on contract for NCSC on software tools. We have used the work on research funded by AAIP and DfT on autonomous systems: the Tigars project. Then work has led to Safety case Templates for Autonomous systems, full report at http://arxiv.org/abs/2102.02625, and a new approach to assurance dubbed Assurance 2.0 Manifesto https://arxiv.org/abs/2004.10474. Assurance 2.0 is the basis for a project within the Darpa ARCOS program on automated certification. Assurance 2.0 has been the basis of an industrial course for > 100 multidisciplinary practitioners in a safety critical industry. |
First Year Of Impact | 2018 |
Sector | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Energy,Transport |
Impact Types | Societal,Economic,Policy & public services |
Description | Guidance on CAE: Concepts, Blocks and Templates |
Geographic Reach | North America |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | Improved the recommended practice of how to structure and present assurance cases to regulators. |
Description | Information Security and Risk in an Assurance Case module |
Geographic Reach | National |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | Incorporation of our research results, in particular CAE Buiding Blocks supported by the CAE Blocks plugin into an Information Security and Risk in an Assurance Case module, has improved students understanding of the concepts of assurance cases, provided them with good knowledge and experience in structuring cases, as well as awareness of important issues that need to be addressed when creating and reviewing assurance cases. |
Description | Member of steering commitee and author of Royal Academy of Engineerign Report Cyber safety and resilience: strengthening the digital systems that support the modern economy |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
URL | https://www.raeng.org.uk/events/list-of-events/2018/march/thriving-in-an-interconnected-world |
Description | NSRA National Security Risk Assessment review |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
Description | Reviewer of National Infrastructure Commission Infrastructure and Digital Systems Resilience study - November 2017 |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Description | GCHQ DISCOVER |
Amount | £10,600 (GBP) |
Funding ID | ID: 61022662 |
Organisation | Government Communications Headquarters (GCHQ) |
Sector | Public |
Country | United Kingdom |
Start | 09/2016 |
End | 03/2017 |
Title | Helping hand and tool support for assurance case building blocks |
Description | A tool for structuring arguments in assurance cases and a new visual guidelines called "Helping hand" is provided to assist in applying the building blocks. The tool is integrated in the ASCE environment through the use of additional schemas and plugins. It is designed to support the methodology of Claims-Arguments-Evidence (CAE) Building Blocks that provides a series of archetypal CAE fragments to help structure cases more formally and systematically. It assists with the development and maintenance of structured assurance cases by providing facilities to manage CAE blocks and partially automate the generation of claim structures. |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2015 |
Provided To Others? | Yes |
Impact | The "Helping Hand" guideline and the tool have been used by our own research team to create assurance cases when using the ASCE environment. The tool integrated in ASCE has also been used in teaching of MSc students in Management of Information Security and Risk at City, University of London in 2015 and 2017. |
URL | http://openaccess.city.ac.uk/12968/ |
Title | PIA FARA simulation engine and models |
Description | A simulation engine developed for on-line deployment, which allows one to build hybrid models (probabilistic and deterministic) and solve them via Monte Carlo simulation. The engine has been used to conduct a number of studies with a model of power transmission network (NORDIC-32) and an ICT control and protection network compliant with IEC 61850. The behaviour of the modelled system was studied using various model of cyber-attacks. |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2015 |
Provided To Others? | Yes |
Impact | The tool allows for the impact of different attacks on cyber-physical systems to be studied in detail and thus, the attacks can be objectively ranked (most serious, less serious, etc.) and thus drive business decisions how to invest in improving cyber security. The work already attracted some interest from industry and we explore ways of maximising this impact. |
Description | Joint work on solving complex state-based semi-Markov models |
Organisation | Duke University |
Department | Electrical and Computer Engineering |
Country | United States |
Sector | Academic/University |
PI Contribution | Presented own work on modelling critical infrastructure resilience and a new state-based model of security and software reliability. In both cases I would solve these models via Monte Carlo simulation. |
Collaborator Contribution | Work on applying advanced techniques for solving the the models numerically (without simulation). Also hierarchical decomposition is being tried to the models. |
Impact | Two joint papers are in preparation. |
Start Year | 2017 |
Description | Models of effectiveness of Effect intrusion tolerant systems under of synchronised cyber attacks. |
Organisation | Johns Hopkins University |
Department | Department of Earth and Planetary Sciences |
Country | United States |
Sector | Academic/University |
PI Contribution | A model of reliability of redundant software under independent and synchronised cyber attacks. The model was developed and studied for a simple redundant system - 1-out-of-2, popular in industrial control systems. |
Collaborator Contribution | The partner has worked on intrusion-tolerant architectures based on Byzantine agreement protocols for different applications, including industrial control. Intrusion tolerance is achieved using mechanisms of periodic cleansing of replicas. The effectiveness of such solutions has been assessed under simplified assumptions of cyber-attacks. |
Impact | The partnership has just started and we expect joint publications. |
Start Year | 2017 |
Description | Presntation on own work on model-based Risk assessment in Critical Infrastructures |
Organisation | University of Naples |
Country | Italy |
Sector | Academic/University |
PI Contribution | I delivered a 2 hours seminar on own work at the University of Naples Frederico II. |
Collaborator Contribution | The Italian partner shared their work on monitoring critical infrastructures and invited me as a partner in a joint EU research proposal on Critical Infrasructure protection. |
Impact | The research proposal is currently worked on and is due to be submitted in August 2017. |
Start Year | 2016 |
Title | CAE Blocks plugin |
Description | CAE Blocks plugin is a software for structuring arguments in assurance cases, implemented on the Adelard ASCE platform. The plugin is designed to support the methodology of Claims-Arguments-Evidence (CAE) Building Blocks that provides a series of archetypal CAE fragments to help structure cases more formally and systematically. It assists with the development and maintenance of structured assurance cases by providing facilities to manage CAE blocks and partially automate the generation of claim structures. |
Type Of Technology | Software |
Year Produced | 2015 |
Impact | The software tool with the CAE Building Blocks methodology have already been used on a number of projects and tasks, including drafting of guidance for the IAEA on the assessment of dependability of nuclear I&C systems important for safety, drafting of templates for arguing about statistical testing as part of the EU Harmonics project, developing cases to address probabilistic modelling of critical infrastructure and particular how one addresses model doubt. We have also used the CAE Blocks plugin on a professional Masters level course at City University London on Information Security and Risk in an Assurance Case module. |
Description | 2nd Annual Conference of Research Institutes funded by EPSRC |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | 2nd Annual Conference of Research Institutes funded by EPSRC |
Year(s) Of Engagement Activity | 2016 |
Description | 2nd Annual Conference of Research Institutes funded by EPSRC |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | 2nd Annual Conference of Research Institutes funded by EPSRC, which included plenary sessions with reports from Institute Directors of the progress made by the Research Institutes, panel discussions and Exhibition of results by different research groups. |
Year(s) Of Engagement Activity | 2016 |
Description | A talk on " Probability Of Perfection - A Practicable Approach?" |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Policymakers/politicians |
Results and Impact | A speculative talk at a leading edge workshop |
Year(s) Of Engagement Activity | 2015 |
URL | http://fm.csl.sri.com/verisure/ |
Description | Invited talk at 5th China System Safety Workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | I was invited to China and gave a talk on "Communicating and reasoning about the safety and security of complex railway systems" and discussed our research. |
Year(s) Of Engagement Activity | 2016 |
Description | Joint Conference of the EPSRC Research Institutes |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | This is an annual forum for the Research Institutes funded by EPSRC to share their results among the participants in the institutes and to report to professional and policy makers. Although the event was mainly national, a significant number of overseas guests attended. The event offered a number of plenary talks and posters by the research groups. I presented a poster on the progress in CEDRICS. |
Year(s) Of Engagement Activity | 2015 |
Description | Keynote contribution by Robin E Bloomfield, Kate Netkachova, Peter Bishop - Confidence in a connected world: safe, secure, resilient and autonomous |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | The keynote speech was given as part of the SSS'17 symposium. It addressed the members and guests of the Safety-Critical Systems Club, operating in support of the wide safety community. |
Year(s) Of Engagement Activity | 2017 |
URL | https://scsc.org.uk/e438 |
Description | Opening keynote at RSSR 2016 : International Conference on Reliability, Safety and Security of Railway Systems |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | An International Conference on Reliability, Safety and Security of Railway Systems |
Year(s) Of Engagement Activity | 2016 |
Description | Presentation and discussion with FDA, Washington DC, Jan 2017 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Policymakers/politicians |
Results and Impact | Presented work on Assurance Cases (CAE Blocks) and discussed cyber policy work in the UK with FDA managers and scientists |
Year(s) Of Engagement Activity | 2017 |
Description | SAFECOMP 2015 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | This is an annual international conference dealing with safety, reliability and security of computer systems. |
Year(s) Of Engagement Activity | 2015 |
URL | http://safecomp2015.tudelft.nl/ |
Description | Structured Assurance Cases: a crash course |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | The tutorial was colocated with an international symposium on software reliability engineering (ISSRE 2015) and provided attendees with a practical understanding of the concepts of structured assurance cases, taught them how to create and review cases and raised awareness of current research directions. Participants expressed a high level of satisfaction with teaching methods and reported that they had acquired useful knowledge and skills for structuring and reviewing assurance cases. |
Year(s) Of Engagement Activity | 2015 |
Description | Talk on the argument strength - an engineering perspective |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | The talk was given to see how the justification of critical engineered systems that impact all out lives might be informed by the work of the philosophical argumentation community. |
Year(s) Of Engagement Activity | 2016 |
URL | http://homepages.ruhr-uni-bochum.de/defeasible-reasoning/Argument-Strength-2016.html |
Description | UK CRIC Workshop in Oxford, 2-3 February, 2016 |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | The new national initiative UK CRIC has been announced and the research directions for the next 10 years have been discussed. |
Year(s) Of Engagement Activity | 2016 |
URL | http://ukcric.co.uk/ |