Why Johnny doesn't write secure software? Secure software development by the masses
Lead Research Organisation:
University of Bristol
Department Name: Computer Science
Abstract
Do you use mobile or web apps or have Internet of Things devices on your person, in your home or workplace? Have you thought about who developed the software that drives these apps and devices, what was their understanding of cyber security, how did they make design decisions that impact the cyber security of the resulting software, and what factors influenced their behaviour and design choices? Or perhaps you are one of the masses exploiting app development platforms and easy-to-program hardware devices such as Arduino and Raspberry Pi to develop applications and deploy them for personal use or distribute them to millions of people around the world? How do you make cyber security decisions when you write software? Do you consciously think about the security implications of your design choices, or are there other factors that are more critical? What will help you achieve your goals from the software that you are developing while ensuring that it is not vulnerable to attacks by malicious actors?
This project aims to develop a deep foundational understanding of these issues. We recognise that developing software is no longer the preserve for the select few with deep technical skills, training, and knowledge. A wide range of people from diverse backgrounds are increasingly developing software for mobile and web apps and for programmable consumer devices. This diversity of developers is at the heart of many innovations in the digital economy. The software they produce can be, and is, deployed across systems embedded in many aspects of human activity, and is used by a global user base. However, little is currently understood about the security behaviours and decision-making processes of 'the masses' engaged in software development.
We refer to these masses by the pseudonym 'Johnny' - based on a seminal work by Whitten and Tygar where they highlighted the challenges faced by Johnny, the prototypical user of encryption. In this project we aim to tackle the challenges faced by Johnny in a contemporary setting beyond encryption. We focus on the Johnnys with diverse backgrounds, know-how and cyber security expertise who can, and are, developing software used, potentially, by millions worldwide.
Drawing on a research team of experts in cyber security, software engineering, and psychology, our aim in this project is to conduct empirically-grounded research to better understand the security implications of Johnny's behaviours and practices and develop effective support for secure software development by Johnny. We propose to achieve this by uncovering and characterising the security vulnerabilities that Johnny tends to introduce, by analysing how and why these vulnerabilities are introduced, and by identifying and evaluating a range of interventions to improve Johnny's security behaviours during software development. We will do this in collaboration with eminent international research partners, drawn from leading research and practitioner organisations around the world. This project will be the first to study the inter-relationship between the cognitive and social processes that shape Johnny's cyber security decisions, their impact on the security of the resultant software and the novel interventions that may steer Johnny towards more effective cyber security decisions during software development.
This project aims to develop a deep foundational understanding of these issues. We recognise that developing software is no longer the preserve for the select few with deep technical skills, training, and knowledge. A wide range of people from diverse backgrounds are increasingly developing software for mobile and web apps and for programmable consumer devices. This diversity of developers is at the heart of many innovations in the digital economy. The software they produce can be, and is, deployed across systems embedded in many aspects of human activity, and is used by a global user base. However, little is currently understood about the security behaviours and decision-making processes of 'the masses' engaged in software development.
We refer to these masses by the pseudonym 'Johnny' - based on a seminal work by Whitten and Tygar where they highlighted the challenges faced by Johnny, the prototypical user of encryption. In this project we aim to tackle the challenges faced by Johnny in a contemporary setting beyond encryption. We focus on the Johnnys with diverse backgrounds, know-how and cyber security expertise who can, and are, developing software used, potentially, by millions worldwide.
Drawing on a research team of experts in cyber security, software engineering, and psychology, our aim in this project is to conduct empirically-grounded research to better understand the security implications of Johnny's behaviours and practices and develop effective support for secure software development by Johnny. We propose to achieve this by uncovering and characterising the security vulnerabilities that Johnny tends to introduce, by analysing how and why these vulnerabilities are introduced, and by identifying and evaluating a range of interventions to improve Johnny's security behaviours during software development. We will do this in collaboration with eminent international research partners, drawn from leading research and practitioner organisations around the world. This project will be the first to study the inter-relationship between the cognitive and social processes that shape Johnny's cyber security decisions, their impact on the security of the resultant software and the novel interventions that may steer Johnny towards more effective cyber security decisions during software development.
Planned Impact
The ultimate beneficiaries of the research are citizens and the digital economy at large. They will benefit from fundamental improvements in security of the software that pervades their daily lives - due to the foundational advances in security behaviours of the masses developing such software and novel interventions to support them in effective security practices.
Developers, i.e., Johnny, will benefit from fundamental insights into factors influencing their security behaviours, tools to automatically detect and flag their security assumptions and interventions to support them in effective security behaviours and design choices.
Large industry organisations and SMEs will benefit from insights into human factors and assumptions that impact software security in contemporary settings where developers have a diversity of knowledge and expertise in both software development and security. They will benefit from the insights relating behaviours, vulnerabilities and interventions to inform their own practices within their software development teams - whether in-house or out-sourced. They will also benefit from new tools and techniques as well as evolution of existing APIs and development environment to more effectively support security design choices. They will also benefit from innovative interventions to support secure software development by the masses, hence stimulating economic growth and development of a strong industry sector in this area.
Policy and government organisations will benefit from a deeper understanding of how Johnny's behaviour impacts software security and hence security of the wider environment and the kind of interventions that can lead to improved security cultures in the masses developing software.
Last, but not least, UK science will be a major beneficiary by being at the forefront of tackling fundamental security challenges that are not only critical to the UK society and economy but that of the world at large.
Developers, i.e., Johnny, will benefit from fundamental insights into factors influencing their security behaviours, tools to automatically detect and flag their security assumptions and interventions to support them in effective security behaviours and design choices.
Large industry organisations and SMEs will benefit from insights into human factors and assumptions that impact software security in contemporary settings where developers have a diversity of knowledge and expertise in both software development and security. They will benefit from the insights relating behaviours, vulnerabilities and interventions to inform their own practices within their software development teams - whether in-house or out-sourced. They will also benefit from new tools and techniques as well as evolution of existing APIs and development environment to more effectively support security design choices. They will also benefit from innovative interventions to support secure software development by the masses, hence stimulating economic growth and development of a strong industry sector in this area.
Policy and government organisations will benefit from a deeper understanding of how Johnny's behaviour impacts software security and hence security of the wider environment and the kind of interventions that can lead to improved security cultures in the masses developing software.
Last, but not least, UK science will be a major beneficiary by being at the forefront of tackling fundamental security challenges that are not only critical to the UK society and economy but that of the world at large.
Organisations
Publications
Chowdhury P
(2021)
Developers Are Neither Enemies Nor Users: They Are Collaborators
Iwaya L
(2023)
Privacy Engineering in the Wild: Understanding the Practitioners' Mindset, Organizational Aspects, and Current Practices
in IEEE Transactions on Software Engineering
Iwaya LH
(2023)
On the privacy of mental health apps: An empirical investigation and its implications for app development.
in Empirical software engineering
Patnaik N.
(2019)
Usability smells: An analysis of developers' struggle with crypto libraries
in Proceedings of the 15th Symposium on Usable Privacy and Security, SOUPS 2019
Rauf I
(2021)
The Case for Adaptive Security Interventions
in ACM Transactions on Software Engineering and Methodology
Rauf I
(2023)
The case for adaptive security interventions
Description | Developing software is no longer the domain of the select few with deep technical skills, training and knowledge. A wide range of people from diverse backgrounds are developing software for smart phones, websites and IoT devices used by millions of people. Johnny is our pseudonym for such developers (following Whitten and Tygar's pseudonym for typical users). Currently, little is understood about the security behaviours and decision-making processes of such developers engaging in software development. The overall aim of this EPSRC-funded project is to develop an empirically-grounded theory of secure software development by the masses. Our focus is on understanding: 1. what typical classes of security vulnerabilities arise from their mistakes, 2. why these mistakes occur, and 3. how we may mitigate these issues and promote secure behaviours. To achieve this, we designed a number of studies. First, working with a number of researchers from different disciplines, we designed a study meant to understand developers' reasoning and decision-making across the different kinds of software development tasks they typically come across--why these mistakes occur. This includes not only having to deal with source-code and potential vulnerabilities, but also with less technical aspects, such as considering social aspects of whom to trust when seeking testers for software, or when asking for help, or the decisions that come from monetisation, such as the potential impact of advertisement libraries on security, or the longer-term implications of typical clauses in software licensing agreements. We designed a task-based study where we had 44 mobile app developers engage with these kinds of tasks, prioritising their solutions, and reasoning about their choices: *why* they think they make particular decisions. We found that developers really only frequently consider security when directly facing code (e.g., fixing vulnerabilities), but in many software development activities choices perceived to be secure may only be an illusion, with rationales indicating little to no security considerations. We also undertook an analysis of over 2400 Stack Overflow posts where developers struggle with using cryptography libraries and identified 16 underlying usability issues. We analysed these further in the light of usability principles proposed in literature to identify 4 *usability smells* where the principles are not being observed. Following up on these findings, we designed further experimental psychology studies on how cognitive biases may play into developers' decision-making on trusting particular people or resources (such as code fragments on Stack Overflow), which so far indicate that developers places trusts in people (and the resources they provide) based on *their* perception of those people, which may not be an accurate view of reality at all. This provides further in-depth understanding of why these mistakes occur, especially in software development tasks where developers are not directly engaged in writing code. We further collaborated with cognitive scientists and software engineers to design a psychometric instrument to elicit developers' attitudes towards handling of personal data in their software, to allow for quantifiable measurements of the extent to which they care about minimizing the data they capture of users, placing users in control of their own data, and using such personal data for monetization. Further studies have investigated the impact of particular software libraries, e.g., Ad libraries on developers' decision-making wrt security and privacy as well as studying the role of security champions in supporting the development of security best practices. We have also undertaken research on development of adaptive security interventions to support developers with security tasks as they undertake their day-to-day development activities as well as new mechanisms to support choice of security and cryptography features to meet requirements they are trying to address. The key findings from this project as described above are already of interest to developers themselves, as well as policy makers intending to support developers in writing more secure software. Understanding that secure software development is about more than just writing secure code, and stimulating a critical reflective attitude towards the choices developers make and what potential impacts these may have on the security of their software should be an important aspect of mitigating these aspects and promoting secure behaviour. |
Exploitation Route | Our findings can form the basis of identifying where suitable support may be targeted to help developers in producing more secure apps that are used by a variety of users around the world. The interventions may be of a technical nature or in the form of guidelines or other mechanisms such as peer support. Such interventions are a subject of on-going investigation within the project. |
Sectors | Digital/Communication/Information Technologies (including Software) |
URL | https://www.writingsecuresoftware.org |
Description | The workshops organised on the topics of impact of software warranties on cyber security and need for large-scale empirical research on software security have led to a number of discussions with industry and policy/practice organisations. The project continues to provide input into the Developer-centred Security stream in the Research Institute for Sociotechnical Cyber Security. The project also contributed an invited overview article on Developer-Centred Security in the Encyclopedia of Cryptography, Security and Privacy edited by S. Jajodia, P. Samarati, M. Yung and published by Springer. The various talks and seminars have also engaged a range of researchers and practitioners on this topic with requests for input into guidelines for software developers and API developers. The project's outputs have appeared in major international conference proceedings and journals including IEEE/ACM International Conference on Software Engineering, IEEE Transactions on Software Engineering, ACM Transactions on Software Engineering Methodology, Empirical Software Engineering, International Conference on Software Engineering and USENIX Symposium on Usable Security and Privacy. The project's insights have formed the basis for a sister programme of research on developer-centred privacy being pursued within the REPHRAIN National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online as well as the EPSRC Equitable Privacy project. |
First Year Of Impact | 2019 |
Sector | Digital/Communication/Information Technologies (including Software) |
Impact Types | Policy & public services |
Description | CSPM: the usability of Configuring the Security and Privacy Mechanisms of Smart Home Devices |
Amount | £13,904 (GBP) |
Organisation | University of Bristol |
Sector | Academic/University |
Country | United Kingdom |
Start | 01/2022 |
End | 03/2022 |
Description | DiScriBe: Digital Security by Design Social Science Hub+ |
Amount | £3,570,745 (GBP) |
Funding ID | ES/V003666/1 |
Organisation | Economic and Social Research Council |
Sector | Public |
Country | United Kingdom |
Start | 08/2020 |
End | 08/2024 |
Description | Equitable privacy |
Amount | £1,016,817 (GBP) |
Funding ID | EP/W025361/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 08/2022 |
End | 02/2026 |
Description | REPHRAIN: Research centre on Privacy, Harm Reduction and Adversarial Influence online |
Amount | £6,972,599 (GBP) |
Funding ID | EP/V011189/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 09/2020 |
End | 09/2024 |
Description | 'Talking Design' workshops on expert practices |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Professor Marian Petre ran 'Talking Design' workshops on expert practices in software design at Mozilla Foundation (Mountain View, CA, USA in May 2018; Toronto, Canada in January 2018) |
Year(s) Of Engagement Activity | 2018 |
Description | Dagstuhl Seminar 23181, Empirical Evaluation of Secure Development Processes (May 01 - May 05, 2023) |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Participation (by invitation) in Dagstuhl Seminar 23181, Empirical Evaluation of Secure Development Processes. The primary goal of this Dagstuhl Seminar was to lay the foundation for future interdisciplinary collaboration on how security researchers, software engineering researchers, and practitioners think about secure software engineering, and which challenges they perceive, particularly with respect to empirical assessment and evidence. . |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.dagstuhl.de/en/seminars/seminar-calendar/seminar-details/23181 |
Description | Keynote at SICSA Cyber Security Student Conference: Why Johnny doesn't write Secure Software? |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | This was a keynote talk discussing the challenges faced by software developers with regards to security and insights gained from the research so far in the project. |
Year(s) Of Engagement Activity | 2020 |
Description | Keynote: 13th European Conference on Software Architecture, Paris, France 2019. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | A keynote speech was given at the European Conference on Software Architecture. The talk discussed the challenges of developing secure software architectures for large connected environments. |
Year(s) Of Engagement Activity | 2019 |
URL | https://ecsa2019.univ-lille.fr/program/keynotes |
Description | Presentation at workshop for female cyber professionals |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | Irum Rauf participated in 'Queue for the Loo: female entrepreneurism in cyber' at Canary Wharf , London on 5th December, 2018 and introduced The Johnny project at informal networking event held as part of the workshop. The workshop had female cyber professionals. The event was organised by TechUK. |
Year(s) Of Engagement Activity | 2018 |
Description | Presentation: Secure software development by the masses |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | Irum Rauf presented "Secure software development by the masses" at Institute of Coding's Cyber security workshop on 28th Feb, 2019 held at The Open University, MK. The audience was largely SMEs and wider community. |
Year(s) Of Engagement Activity | 2019 |
Description | Secure Software Development talk delivered at 2022 NCSC ACE-CSR conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | Secure Software Development talk delivered at 2022 NCSC ACE-CSR conference by Partha Das Chowdhury (Research Associate). A key research output has been a systematic understanding of the challenges that developers face and the behaviour they display. This led to the notion of collaboration deficit in secure software development. With insights from the Johnny project combined with a scrutiny of the IDEs we came out with recommendations of actionable support that we can provide to developers. The wider community we believe would benefit from these insights and build upon them. |
Year(s) Of Engagement Activity | 2022 |
Description | Seminar at Swansea University: Why Johnny doesn't write Secure Software? |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | This was a research seminar at Swansea University describing the insights from the work undertaken in the project. |
Year(s) Of Engagement Activity | 2021 |
Description | Seminar at TOBB University Ankara Turkey: Why Johnny doesn't write secure software? |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | This was a research seminar at University of Cambridge to deliver insights from the project and planned future research directions. |
Year(s) Of Engagement Activity | 2022 |
Description | Seminar at University of Cambridge: Why Johnny doesn't write Secure Software? |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | This was a research seminar at University of Cambridge to deliver insights from the project and planned future research directions. |
Year(s) Of Engagement Activity | 2021 |
Description | Seminar at University of York: Why Johnny doesn't write secure software? |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | Seminar at the University of York Computer Science Seminar Series. |
Year(s) Of Engagement Activity | 2020 |
Description | Talk at the 1st UK PhD Winter School on Cyber Security, Newcastle, UK 2020 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | A talk titled "Why Johnny doesn't write secure software" was delivered at the Winter School. The talk provided attending PGR students with background to the problem of secure software development and discussed the findings arising to date from the project. |
Year(s) Of Engagement Activity | 2020 |
URL | https://sites.google.com/view/phd-cyber-winterschool2020/home |
Description | Workshop on large-scale empirical research on software security |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | This workshop, supported by the NCSC, brought together researchers within UK, Europe and the US to discuss the requirements for an international collaborative effort and infrastructure to support large-scale empirical research on software security. The collective goal for the day was to develop a shared understanding of the challenges faced by research on software code analysis for cybersecurity and outline a roadmap for an international testbed infrastructure for large-scale experimental research on software security. The workshop was jointly organised by Professor Hridesh Rajan, a Fulbright Scholar from Iowa State University whom we hosted at the University of Bristol, and Professor Awais Rashid. |
Year(s) Of Engagement Activity | 2018 |
Description | Workshop on the Effect of Software Warranties on Cyber Security |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | This workshop, sponsored by the NCSC, focused on bringing software developers and legal professionals together to understand the shared challenges they face in promoting the development of secure software on the one hand, and software at all, on the other hand. The workshop also featured a keynote by Professor Annie Anton from Georgia Tech as well as an invited talk by Robert Carolina from RHUL. There was also a panel involving government and industry figures as well as roundtable discussions. A summary of the discussions and insights from the workshop has appeared in a report in ACM SIGSOFT Software Engineering Notes: Dirk van der Linden, Awais Rashid: The Effect of Software Warranties on Cybersecurity. ACM SIGSOFT Software Engineering Notes 43(4): 31-35 (2018) |
Year(s) Of Engagement Activity | 2018 |