Why Johnny doesn't write secure software? Secure software development by the masses

Lead Research Organisation: University of Bristol
Department Name: Computer Science

Abstract

Do you use mobile or web apps or have Internet of Things devices on your person, in your home or workplace? Have you thought about who developed the software that drives these apps and devices, what was their understanding of cyber security, how did they make design decisions that impact the cyber security of the resulting software, and what factors influenced their behaviour and design choices? Or perhaps you are one of the masses exploiting app development platforms and easy-to-program hardware devices such as Arduino and Raspberry Pi to develop applications and deploy them for personal use or distribute them to millions of people around the world? How do you make cyber security decisions when you write software? Do you consciously think about the security implications of your design choices, or are there other factors that are more critical? What will help you achieve your goals from the software that you are developing while ensuring that it is not vulnerable to attacks by malicious actors?

This project aims to develop a deep foundational understanding of these issues. We recognise that developing software is no longer the preserve for the select few with deep technical skills, training, and knowledge. A wide range of people from diverse backgrounds are increasingly developing software for mobile and web apps and for programmable consumer devices. This diversity of developers is at the heart of many innovations in the digital economy. The software they produce can be, and is, deployed across systems embedded in many aspects of human activity, and is used by a global user base. However, little is currently understood about the security behaviours and decision-making processes of 'the masses' engaged in software development.

We refer to these masses by the pseudonym 'Johnny' - based on a seminal work by Whitten and Tygar where they highlighted the challenges faced by Johnny, the prototypical user of encryption. In this project we aim to tackle the challenges faced by Johnny in a contemporary setting beyond encryption. We focus on the Johnnys with diverse backgrounds, know-how and cyber security expertise who can, and are, developing software used, potentially, by millions worldwide.

Drawing on a research team of experts in cyber security, software engineering, and psychology, our aim in this project is to conduct empirically-grounded research to better understand the security implications of Johnny's behaviours and practices and develop effective support for secure software development by Johnny. We propose to achieve this by uncovering and characterising the security vulnerabilities that Johnny tends to introduce, by analysing how and why these vulnerabilities are introduced, and by identifying and evaluating a range of interventions to improve Johnny's security behaviours during software development. We will do this in collaboration with eminent international research partners, drawn from leading research and practitioner organisations around the world. This project will be the first to study the inter-relationship between the cognitive and social processes that shape Johnny's cyber security decisions, their impact on the security of the resultant software and the novel interventions that may steer Johnny towards more effective cyber security decisions during software development.

Planned Impact

The ultimate beneficiaries of the research are citizens and the digital economy at large. They will benefit from fundamental improvements in security of the software that pervades their daily lives - due to the foundational advances in security behaviours of the masses developing such software and novel interventions to support them in effective security practices.

Developers, i.e., Johnny, will benefit from fundamental insights into factors influencing their security behaviours, tools to automatically detect and flag their security assumptions and interventions to support them in effective security behaviours and design choices.

Large industry organisations and SMEs will benefit from insights into human factors and assumptions that impact software security in contemporary settings where developers have a diversity of knowledge and expertise in both software development and security. They will benefit from the insights relating behaviours, vulnerabilities and interventions to inform their own practices within their software development teams - whether in-house or out-sourced. They will also benefit from new tools and techniques as well as evolution of existing APIs and development environment to more effectively support security design choices. They will also benefit from innovative interventions to support secure software development by the masses, hence stimulating economic growth and development of a strong industry sector in this area.

Policy and government organisations will benefit from a deeper understanding of how Johnny's behaviour impacts software security and hence security of the wider environment and the kind of interventions that can lead to improved security cultures in the masses developing software.

Last, but not least, UK science will be a major beneficiary by being at the forefront of tackling fundamental security challenges that are not only critical to the UK society and economy but that of the world at large.

Publications

10 25 50
publication icon
Van Der Linden D (2019) The Effect of Software Warranties on Cybersecurity in ACM SIGSOFT Software Engineering Notes

publication icon
Rauf I (2021) The Case for Adaptive Security Interventions in ACM Transactions on Software Engineering and Methodology

publication icon
Van Der Linden D (2022) The Impact of Surface Features on Choice of (in)Secure Answers by Stackoverflow Readers in IEEE Transactions on Software Engineering

publication icon
Tahaei M (2022) Charting App Developers' Journey Through Privacy Regulation Features in Ad Networks in Proceedings on Privacy Enhancing Technologies

publication icon
Van Der Linden D (2018) Safe cryptography for all

 
Description Developing software is no longer the domain of the select few with deep technical skills, training and knowledge. A wide range of people from diverse backgrounds are developing software for smart phones, websites and IoT devices used by millions of people. Johnny is our pseudonym for such developers (following Whitten and Tygar's pseudonym for typical users). Currently, little is understood about the security behaviours and decision-making processes of such developers engaging in software development. The overall aim of this EPSRC-funded project is to develop an empirically-grounded theory of secure software development by the masses. Our focus is on understanding:

1. what typical classes of security vulnerabilities arise from their mistakes,
2. why these mistakes occur, and
3. how we may mitigate these issues and promote secure behaviours.

To achieve this, we designed a number of studies. First, working with a number of researchers from different disciplines, we designed a study meant to understand developers' reasoning and decision-making across the different kinds of software development tasks they typically come across--why these mistakes occur. This includes not only having to deal with source-code and potential vulnerabilities, but also with less technical aspects, such as considering social aspects of whom to trust when seeking testers for software, or when asking for help, or the decisions that come from monetisation, such as the potential impact of advertisement libraries on security, or the longer-term implications of typical clauses in software licensing agreements. We designed a task-based study where we had 44 mobile app developers engage with these kinds of tasks, prioritising their solutions, and reasoning about their choices: *why* they think they make particular decisions. We found that developers really only frequently consider security when directly facing code (e.g., fixing vulnerabilities), but in many software development activities choices perceived to be secure may only be an illusion, with rationales indicating little to no security considerations.

We also undertook an analysis of over 2400 Stack Overflow posts where developers struggle with using cryptography libraries and identified 16 underlying usability issues. We analysed these further in the light of usability principles proposed in literature to identify 4 *usability smells* where the principles are not being observed.

Following up on these findings, we designed further experimental psychology studies on how cognitive biases may play into developers' decision-making on trusting particular people or resources (such as code fragments on Stack Overflow), which so far indicate that developers places trusts in people (and the resources they provide) based on *their* perception of those people, which may not be an accurate view of reality at all. This provides further in-depth understanding of why these mistakes occur, especially in software development tasks where developers are not directly engaged in writing code. We further collaborated with cognitive scientists and software engineers to design a psychometric instrument to elicit developers' attitudes towards handling of personal data in their software, to allow for quantifiable measurements of the extent to which they care about minimizing the data they capture of users, placing users in control of their own data, and using such personal data for monetization.

Further studies have investigated the impact of particular software libraries, e.g., Ad libraries on developers' decision-making wrt security and privacy as well as studying the role of security champions in supporting the development of security best practices. We have also undertaken research on development of adaptive security interventions to support developers with security tasks as they undertake their day-to-day development activities as well as new mechanisms to support choice of security and cryptography features to meet requirements they are trying to address.

The key findings from this project as described above are already of interest to developers themselves, as well as policy makers intending to support developers in writing more secure software. Understanding that secure software development is about more than just writing secure code, and stimulating a critical reflective attitude towards the choices developers make and what potential impacts these may have on the security of their software should be an important aspect of mitigating these aspects and promoting secure behaviour.
Exploitation Route Our findings can form the basis of identifying where suitable support may be targeted to help developers in producing more secure apps that are used by a variety of users around the world. The interventions may be of a technical nature or in the form of guidelines or other mechanisms such as peer support. Such interventions are a subject of on-going investigation within the project.
Sectors Digital/Communication/Information Technologies (including Software)

URL https://www.writingsecuresoftware.org
 
Description The workshops organised on the topics of impact of software warranties on cyber security and need for large-scale empirical research on software security have led to a number of discussions with industry and policy/practice organisations. The project continues to provide input into the Developer-centred Security stream in the Research Institute for Sociotechnical Cyber Security. The project also contributed an invited overview article on Developer-Centred Security in the Encyclopedia of Cryptography, Security and Privacy edited by S. Jajodia, P. Samarati, M. Yung and published by Springer. The various talks and seminars have also engaged a range of researchers and practitioners on this topic with requests for input into guidelines for software developers and API developers. The project's outputs have appeared in major international conference proceedings and journals including IEEE/ACM International Conference on Software Engineering, IEEE Transactions on Software Engineering, ACM Transactions on Software Engineering Methodology, Empirical Software Engineering, International Conference on Software Engineering and USENIX Symposium on Usable Security and Privacy.
First Year Of Impact 2019
Sector Digital/Communication/Information Technologies (including Software)
Impact Types Policy & public services

 
Description CSPM: the usability of Configuring the Security and Privacy Mechanisms of Smart Home Devices
Amount £13,904 (GBP)
Organisation University of Bristol 
Sector Academic/University
Country United Kingdom
Start 01/2022 
End 03/2022
 
Description DiScriBe: Digital Security by Design Social Science Hub+
Amount £3,570,745 (GBP)
Funding ID ES/V003666/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 09/2020 
End 08/2024
 
Description REPHRAIN: Research centre on Privacy, Harm Reduction and Adversarial Influence online
Amount £6,972,599 (GBP)
Funding ID EP/V011189/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2020 
End 09/2023
 
Description 'Talking Design' workshops on expert practices 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Professor Marian Petre ran 'Talking Design' workshops on expert practices in software design at Mozilla Foundation (Mountain View, CA, USA in May 2018; Toronto, Canada in January 2018)
Year(s) Of Engagement Activity 2018
 
Description Keynote at SICSA Cyber Security Student Conference: Why Johnny doesn't write Secure Software? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact This was a keynote talk discussing the challenges faced by software developers with regards to security and insights gained from the research so far in the project.
Year(s) Of Engagement Activity 2020
 
Description Keynote: 13th European Conference on Software Architecture, Paris, France 2019. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact A keynote speech was given at the European Conference on Software Architecture. The talk discussed the challenges of developing secure software architectures for large connected environments.
Year(s) Of Engagement Activity 2019
URL https://ecsa2019.univ-lille.fr/program/keynotes
 
Description Presentation at workshop for female cyber professionals 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Irum Rauf participated in 'Queue for the Loo: female entrepreneurism in cyber' at Canary Wharf , London on 5th December, 2018 and introduced The Johnny project at informal networking event held as part of the workshop. The workshop had female cyber professionals. The event was organised by TechUK.
Year(s) Of Engagement Activity 2018
 
Description Presentation: Secure software development by the masses 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Irum Rauf presented "Secure software development by the masses" at Institute of Coding's Cyber security workshop on 28th Feb, 2019 held at The Open University, MK. The audience was largely SMEs and wider community.
Year(s) Of Engagement Activity 2019
 
Description Secure Software Development talk delivered at 2022 NCSC ACE-CSR conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Secure Software Development talk delivered at 2022 NCSC ACE-CSR conference by Partha Das Chowdhury (Research Associate).

A key research output has been a systematic understanding of the challenges that developers face and the behaviour they display. This led to the notion of collaboration deficit in secure software development. With insights from the Johnny project combined with a scrutiny of the IDEs we came out with recommendations of actionable support that we can provide to developers. The wider community we believe would benefit from these insights and build upon them.
Year(s) Of Engagement Activity 2022
 
Description Seminar at Swansea University: Why Johnny doesn't write Secure Software? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact This was a research seminar at Swansea University describing the insights from the work undertaken in the project.
Year(s) Of Engagement Activity 2021
 
Description Seminar at TOBB University Ankara Turkey: Why Johnny doesn't write secure software? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This was a research seminar at University of Cambridge to deliver insights from the project and planned future research directions.
Year(s) Of Engagement Activity 2022
 
Description Seminar at University of Cambridge: Why Johnny doesn't write Secure Software? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact This was a research seminar at University of Cambridge to deliver insights from the project and planned future research directions.
Year(s) Of Engagement Activity 2021
 
Description Seminar at University of York: Why Johnny doesn't write secure software? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Seminar at the University of York Computer Science Seminar Series.
Year(s) Of Engagement Activity 2020
 
Description Talk at the 1st UK PhD Winter School on Cyber Security, Newcastle, UK 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact A talk titled "Why Johnny doesn't write secure software" was delivered at the Winter School. The talk provided attending PGR students with background to the problem of secure software development and discussed the findings arising to date from the project.
Year(s) Of Engagement Activity 2020
URL https://sites.google.com/view/phd-cyber-winterschool2020/home
 
Description Workshop on large-scale empirical research on software security 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This workshop, supported by the NCSC, brought together researchers within UK, Europe and the US to discuss the requirements for an international collaborative effort and infrastructure to support large-scale empirical research on software security. The collective goal for the day was to develop a shared understanding of the challenges faced by research on software code analysis for cybersecurity and outline a roadmap for an international testbed infrastructure for large-scale experimental research on software security. The workshop was jointly organised by Professor Hridesh Rajan, a Fulbright Scholar from Iowa State University whom we hosted at the University of Bristol, and Professor Awais Rashid.
Year(s) Of Engagement Activity 2018
 
Description Workshop on the Effect of Software Warranties on Cyber Security 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact This workshop, sponsored by the NCSC, focused on bringing software developers and legal professionals together to understand the shared challenges they face in promoting the development of secure software on the one hand, and software at all, on the other hand. The workshop also featured a keynote by Professor Annie Anton from Georgia Tech as well as an invited talk by Robert Carolina from RHUL. There was also a panel involving government and industry figures as well as roundtable discussions. A summary of the discussions and insights from the workshop has appeared in a report in ACM SIGSOFT Software Engineering Notes:

Dirk van der Linden, Awais Rashid: The Effect of Software Warranties on Cybersecurity. ACM SIGSOFT Software Engineering Notes 43(4): 31-35 (2018)
Year(s) Of Engagement Activity 2018