Bayesian Analysis of Competing Cyber Hypotheses

Lead Research Organisation: University of Liverpool
Department Name: Electrical Engineering and Electronics

Abstract

Cyber security is recognised as important at the highest levels of international government. President Obama has said that "the Cyber threat is one of the most serious economic and national security challenges [the US] face as a nation". Even the £650M in additional funding that accompanied the UK's Cyber Security Strategy is dwarfed by the >£10B estimated annual cost of cyber-crime to the UK economy. Additionally, we see links to "transnational organised crime" (cyber-crime is lucrative and widespread) as well as "Terrorism" (state-sponsored cyber-warfare is increasing) and "Ideologies and beliefs" (anti-establishment hacktivists, eg Anonymous, are also resorting to cyber-attack to express their views).

Companies such as HP help organisations who are subjected to cyber attacks to protect their assets and information from such attacks. These cyber defence companies achieve this using a combination of hardware and software augmented with human effort. Allocating human effort to activity is critical since inappropriate allocation can result in human time being wasted or attacks going unchallenged. Time pressure, the presence of ambiguous information and the high stakes involved can then degrade the human judgement associated with this allocation process.

Psychologists understand that such pressures degrade human decision making and similar issues have been found to exist in other domains. Indeed, Pearl Harbour and the Cuban Missile Crisis were each the result of failures in the intelligence process that can be traced back to human analysis errors educating decision making.

Motivated by such experiences, in the 1970s, the CIA developed a technique, "Analysis of Competing Hypotheses" which encourages analysts and decision makers to avoid the pitfalls that can be associated with intelligence analysis. This technique involves consideration of multiple candidate explanations for what is being observed. The hypotheses are then assessed (and iteratively refined) using the observations to discriminate between likely and unlikely hypotheses. While the technique has proven its utility, for it to work effectively, it is important that the hypotheses considered include the "possible" not just the "probable" explanations. Unfortunately, "possible" and "probable" aren't precisely defined in this context.

However, a recent advance in the statistics literature, "Sequential Monte Carlo Samplers", exhibits many of the same features as Analysis of Competing Hypotheses. Sequential Monte Carlo samplers are typically applied in contexts where a computer (not a person) generates the hypotheses and assesses them. However, just like Analysis of Competing Hypotheses, they consider a population of hypotheses, assessed against data and then iteratively used to spawn a new population of hypotheses. Crucially, the analogous concept to the notion of "possible" and "probable" hypotheses is both well defined and well understood.

We propose to adapt Sequential Monte Carlo samplers to become part of Analysis of Competing Hypotheses. We further propose to apply and demonstrate a tool embodying the technique in an operational cyber security context.

If successful, this project would develop techniques that would ensure that decisions made in operational cyber security settings were well motivated. Where those decisions relate to the allocation of human analyst resources to activities, this would improve the efficiency of cyber security operations. The technology will position the UK at the forefront of the state-of-the-art in this high priority application domain.

Planned Impact

HP Labs are a partner in this proposal with the explicit purpose of maximising the project's impact. HP Labs is the long term corporate research entity for HP and has a team dedicated to innovation in security. That team include experts in the UK and US who are specifically interested in improving decision making in operational contexts relevant to cyber security.

HP is a major global player in security and offers a wide range of products and services to enterprises and governments internationally. These offerings include extensive human analyses and decision making inside SOCs. HP are keen to explore the potential to use technology to mitigate biases encountered in such decision making and improve the effectiveness of such analysts. Indeed the US Department of Homeland Security (DHS) is funding HP Labs (Princeton) to conduct research as part of a project funded via Dartmouth University and with George-Mason University looking at issues that are closely related to this proposal.

HP provides products for use in other people's SOCs, as well as providing SOCs as a service to its customers: HP runs SOCs for organisations that range from SMEs through corporate enterprises to government organisations. This positions HP as a route through to a global market related to SOCs specifically and cyber security in general.

The primary impact for this project will therefore be the use of tools, developed and then prototyped in his project, that can improve the effectiveness of SOCs which HP either provides as a service or supports via the provision of security products.

By developing such prototype tools in this project, the security of companies associated with such SOCs will be improved; this project will not only impact HP directly, but HP's customers indirectly. In so doing, this project will improve the cyber security of organisations across the UK and right across the wider global marketplace.

Publications

10 25 50
 
Description We have identified an issue related to how the question asked in an intelligence analysis context is not always the question answered. More importantly, we have developed a method for converting an answer to the wrong question into an answer to the right question. This reduces the cognitive load on the person answering the question without degrading the performance of the wider system. We have also begun to develop a software framework that will make it possible to test such analysis techniques in a fully open manner (making use of a historical "mystery").
Exploitation Route The advances made are generically applicable to any setting where difficult decisions have to be made in the context of incomplete, ambiguous and uncertain data. This is particularly pertinent in defence and security, but also in contexts (such as manufacture) where big decisions need to be made in the absence of the knowledge of how things will pan out.
Sectors Aerospace

Defence and Marine

Financial Services

and Management Consultancy

Government

Democracy and Justice

Manufacturing

including Industrial Biotechology

Pharmaceuticals and Medical Biotechnology

Security and Diplomacy

 
Description Above-Water Maritime Research
Amount £121,566 (GBP)
Organisation Defence Science & Technology Laboratory (DSTL) 
Sector Public
Country United Kingdom
Start 01/2015 
End 07/2018
 
Description Distributed Hypothesis Generation and Evaluation'
Amount £100,000 (GBP)
Funding ID 2476782 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 12/2020 
End 11/2024
 
Description ICASE award with Unilever
Amount £80,000 (GBP)
Funding ID ICASE Voucher #18000200 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2018 
End 09/2022