COMMANDO-HUMANS: COMputational Modelling and Automatic Non-intrusive Detection Of HUMan behAviour based iNSecurity

This project addresses mainly the Human Factors challenge of the joint Singapore-UK call, and it has an interdisciplinary team with expertise in cyber security, cognitive psychology, and human-computer interface (HCI). It aims at producing direct evidence that human behaviour related insecurity can be detected automatically by applying human cognitive models to model and simulate humans involved in security systems. A key outcome of the project will be a working software system that can be used for this purpose by researchers and practitioners. The project will focus on human user authentication systems as a representative use case and will produce new knowledge on the role of human behaviours in such systems and security systems in general. Both the software framework and new knowledge on human behaviours can also help address other challenges of the call (e.g., detection of intruders/extremists requires knowledge on how they behave; protection of user privacy require knowledge on how human users handle personal data; policy makers need to understand behaviours of their organisations' employees and human attackers targeting their organisations to make more informed decisions).

It has been well known that human factors are a very important aspect of cyber security, as recognised by governments all over the world e.g., in the UK Cyber Security Strategy (2011), in Singapore's National Cyber Security Masterplan 2018 (2013), and in the US Federal Cybersecurity Research and Development Strategic Plan (2011). Human related insecurity is often related to intended or unintentional (maybe subconscious) insecure human behaviours. To conduct research on human behaviours (in cyber security, HCI, psychology and other related fields), researchers normally depend on involvement of real human users via surveys, interviews, simulated scenarios, observations of real cases, interactive games, or other specially designed user studies. Such approaches are often time-consuming and costly, and suffer from other issues like limited and/or biased samples, questionable ecological validity, difficulties in reproducing results, and impossibility of running some studies due to ethical/privacy/legal concerns.

This project aims at developing the first (to the best our knowledge) general-purpose computational framework and supporting software tools that will enable automatic detection of human behaviour related insecurity at the HCI level without the need to involve real human users. The framework will be built on computational models of human cognitive processes, HCIs, human behaviour related attacks and (in)security measures. The framework will be non-intrusive: instead of evaluating the running system itself, the framework will evaluate an abstract executable model of the system and humans involved. Removing real human users from the process allows faster and more objective inspection of potential insecurity of a given security system. The automated process can still be combined with traditional user studies to make better use of limited resources in automatically detecting potential insecurity problems deserving further manual analysis.

The framework and software tools developed will be of great value for cyber security researchers, security system designers/developers and security industry to deliver securer systems to end users. As a natural byproduct, they will also allow easier evaluation of usability of security and non-security related computer systems with an HCI. As we mentioned above in this summary, people having concerns on other challenges of the call can benefit from the project's outcomes as well.

In this project we will focus mainly on HCI-level ("micro") human behaviours, but possible extensions to higher-level ("macro") behaviours (e.g., how human users adapt their behaviours over time via rehearsals and learning) will be looked at as well to pave the way for our future research.

The "Academic Beneficiaries" field of the Je-S form explains the expected academic impact in detail, so here we focus on economic and societal impact.

While the project is targeting mainly researchers, we will make the software framework accessible to non-researchers as well so it can help security system designers and developers, and security industry in general to check human behaviour related insecurity problems at the HCI level in the design stage of their security products and services. Even when user studies are still needed to evaluate their products and services' performance, the software framework can help identify key areas they need to pay more attention to and thus making a better use of the limited resources. This, on one hand, can help enhance the research capacity, knowledge and skills and efficiency of security industry to deliver securer security products and services, and on the other hand can improve the overall experience and quality of life of end users by reducing security incidents that can be avoided before such products and services are introduced into the real world. If it is possible to collect more realistic (and anonymous) information about human users using a deployed security product or service, the vendor/provider can also identify more potential insecurity problems that exist for a particular group of users only and find ways to serve them better.

We also expect that the software framework developed will help organisations' policy makers and IT managers to get more information about behaviours of their employee's and human attackers targeting their organisations, and the usability-security trade-off of their security systems (deployed and those under consideration for purchase), which will allow them to make more informed decisions on things like what security systems to use, how to use them, what security policies should be enforced, and if any training or educational programmes are needed for their staff and customers. We understand policy makers and IT managers will have more interests in macro human behaviours and more systems beyond human user authentication, so they can be potential users of the planned extensions of our research in future.

Like most IT systems, there are two types of end users of security products and services: 1) non-security service providers using such products and services developed by other companies to serve their customers (e.g., banks); 2) end human users who are actually using the products and services. In addition to indirectly benefiting from the software framework we will develop, both groups of end users can actually use the software framework to conduct independent evaluation of security products and services they use, which can help increase transparency of the security industry and eventually benefit security industry by giving more credits to better products and services. This may also foster a new service on independent security and usability evaluation of IT systems (e.g., like what Virus Bulletin Ltd is currently doing on anti-malware products). We will exploit the possible commercialisation of the software framework developed towards this direction.

As can be expected, our proposed research on human behaviours at the HCI level will create new knowledge on how human users and attackers behave and interact with computer systems. Such knowledge is not only useful for researchers, but equally so for practitioners and end users. This is particularly important for security education and training purposes, e.g., in designing and implementing cyber security awareness campaigns for the general public. The focused cyber security systems, human user authentication systems, are also a very good use case here as passwords are widely used in security education and training.

It deserves mentioning that the human and HCI modelling parts of our software framework are independent of security, so can be used for evaluating usability of any IT systems.