FAULT ANALYSES IN INDUSTRY AND ACADEMIC RESEARCH

Lead Research Organisation: Brunel University London
Department Name: Computer Science

Abstract

Faults in software cost the UK IT industry hundreds of millions of pounds each year; a common aim of software engineering research in academic terms is to address industry problems. The Fault Analyses in Industry and Academic Research Network (FIAR-NET) brings closer ties between industry and academic research in the area of software fault analysis. A range of pressing issues exists and it is to the mutual benefit of the two communities that they work more closely than they have in the past. This is the underpinning motivation and reason for proposing FIAR-NET.

Two key themes run through the network. Firstly, challenges of fault case-studies using industrial systems. While recent progress has been made in this area, more industrial involvement has the potential to enhance and add high value to these foundations. The theme will embrace the practicalities of predicting faults, fault data collection/assimilation (including ensuring that the data is reliable), fault data interpretation and faults in an 'agile' context, a model for developing software which is prevalent in UK IT industry. The second theme is an exploration of approaches to re-engineering code so that is in some way better and thereby mitigate and minimise faults; finally through better testing strategies (both pre- and post-release).

FIAR-NET will operate through regular workshops, events and a range of on-line resources and initiatives including a set of on-site "road-shows". An initial network of core academics and industrial partners including SMEs and large organisations will be established, maintained and then extended. A key aim is to stimulate innovative, leading research collaboration and dissemination of subsequent results. The proposed network team at Brunel builds on a track record of expertise in fault analyses and empirical software engineering, widely recognised as world-leading. This reputation is underpinned by a wide variety of past and present industrial and academic collaborations which FIAR-NET will build on in the three years of its duration and thereafter.

Both Counsell (PI) and Hierons (a core FIAR-NET member) have led successful research networks in the past and all investigators on the project have wide experience of industrial collaborations and bring that experience to the network. Finally, FIAR-NET falls within one of the EPSRC's strategic aims and portfolio plan to 'grow' research into fault-finding and fixing.

Planned Impact

ADVANCEMENT OF KNOWLEDGE

Advancement of knowledge facilitated by FIAR-NET can be seen from four viewpoints. Firstly, the areas that FIAR-NET explores are open research areas. For example, the optimal set of characteristics of code and combinations of code features for predicting faults has yet to be established. Equally, the link between faults and refactoring and whether refactoring actually improves the quality of code has received very little empirical attention. Progress in each of these areas has been hampered by the lack of data from proprietary systems and so involvement of industry is vital to understanding and then solving some of these problems. The proposed network brings together an initial set of six industry partners, the BCS and four academic institutions, all with specific interests in these types of area. Secondly, the results can be adopted, adapted and pursued by industrial and academic collaborations through the project deliverables; specifically, through the workshop agendas and road-shows, both of which provide a forum for research to be further explored. The set of dissemination activities and potential for extending the network over the project duration also promote knowledge advancement. Thirdly, the research agenda will inform new knowledge about, a) collection and analysis of fault data, b) testing, the problems, differences and emphases of commercial test environments (Agile or otherwise) and c) how systems can be re-engineered to minimise faults. The project thus provides many opportunities for better allocation and prioritisation of resources by developers and project managers.

PEOPLE IMPACT

From a people perspective, the project promotes a set of skills through the investigation of research problems shared between industry and academic groups. These skills can be adopted by practitioners who wish to apply the same principles to their software and fault bases. A range of people will also benefit from the research. Software architects, managers and developers at the industrial partners will benefit directly through results and lessons generated from the network. The network will allow academic researchers in the areas that FIAR-NET covers to benefit from a shared understanding and the transfer of knowledge that academic and industrial collaboration fosters.

ECONOMIC IMPACT

From an economic viewpoint, faults in software will always occur and there will always be some cost associated with faults and fixing them when they do occur. However, there are significant potential cost savings to be gained by the UK IT industry if research can produce demonstrably successful methods of understanding faults and tackling issues surrounding faults. Firstly, resources can be pre-allocated and directed where they are most needed on a proactive rather than a reactive basis, the former of which often causes disruption to other projects. Secondly, explanations of the causes of faults can improve and streamline the processes through which software is developed. Finally, the ability to understand system behaviour in a more timely fashion ultimately frees up time for developers to work on improving/perfecting the quality of their code. The proposed research has huge potential for unlocking the opportunity that a more timely awareness and understanding of system pressures create.

SOCIETAL IMPACT

From a societal viewpoint, the research is in alignment with a key aim of any company of developing robust code and high quality products. Customers of the industrial partners will benefit in terms of the software that they 'deliver'. The value of the research also appeals to the wider societal community. In fact, the research affects anyone who uses software since improvement in the quality of code through minimisation of faults is a fundamental aim of any software developer.

Publications

10 25 50
 
Description Although the situation is improving all the time, Industry and academia still have
varying goals and views on direction of research. Finding a middle ground and
education of each party on the conflicts/challenges in each is still very limited.

Industry don't have the time to spend carrying out activities that academia does
and academics are largely shielded from the realities of coding and the pressures
that involves.

Many of the difficulties are embedded in each other's working lives,
Exploitation Route More involvement by industry in academic activities. Schemes/initiatives and sponsorships/secondments
are the way forward.

Academics should spend more time in industry to appreciate the problems that they face. .
Sectors Digital/Communication/Information Technologies (including Software)

 
Description We have established, until recently, a relationship with BAE systems (submarines and warships) and some of the outputs of FIAR-NET were presented at their Software Engineering Committee. The Head of the Group at BAE also used the empirical nature of the work we do to write papers of his own on evidence-based software engineering. Clearly, avoidance of faults is something that safety-critical systems are keen to understand. Our work was used to spread good practice in the area to other members of the SE group at BAE systems. The work with BAE was also considered for an impact case study in the latest REF (but not used). The same Head of the division Dr Ken Wallace also came to Brunel for our annual research day and was a judge at our awards ceremony as well as being a contributor to the annual Made in Brunel event where students display their research to industry over the course of an evening and prizes awarded. A grant is currently being prepared to work on Software Resilience with multiple companies as a direct lead from the fault-based events from FIAR-NET. This will be applied in areas of cyber and diverse areas such as driverless cars; put another way, the work of FIAR-NET has generated an awareness that techniques for finding and fixing are being applied in more and more areas. Many of the network participants are working with me on ongoing research topics and FIAR-NET opened up these opportunities. It informed a TSE paper just published on developer perceptions of program repair.
First Year Of Impact 2019
Sector Aerospace, Defence and Marine
Impact Types Economic

 
Description Changed the way developers developed through company policy
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact One of our collaborators through the Network issued new standards and changes across the UK in Software Engineering practice.
 
Description Brunel WAT scheme (Incentive Scheme)
Amount £2,000 (GBP)
Organisation Brunel University London 
Sector Academic/University
Country United Kingdom
Start 06/2017 
End 09/2018
 
Description Brunel WAT scheme (Incentive Scheme)
Amount £2,000 (GBP)
Organisation Brunel University London 
Sector Academic/University
Country United Kingdom
Start 06/2017 
End 09/2018
 
Description Concrete collaborations with BaeSystems and outcome of Visiting Industry Research Fellow 
Organisation BAE Systems
Country United Kingdom 
Sector Academic/University 
PI Contribution The FIAR-NET project has allowed a more formal footing with BAE Systems to be established and will result in a Fellowship submission to the EPSRC. We are in the process of inviting Dr Ken Wallace to become a Visiting Industry Fellow at Brunel.
Collaborator Contribution Invited talks and contributions to the University's Made In Brunel.
Impact Visiting Industry Fellowship in process.
Start Year 2014
 
Description Meeting at Keele for grant proposal group 
Organisation Keele University
Country United Kingdom 
Sector Academic/University 
PI Contribution Keele UCL Newcastle Durham Brunel
Collaborator Contribution Joint meeting to discuss future funding bid.
Impact Working documents before proposal is formally put together
Start Year 2017
 
Description First Workshop 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Ran a Workshop as part of FIARNET.
Here is the url of the event.

https://www.fiarnet.org/events/
Year(s) Of Engagement Activity 2016
URL https://www.fiarnet.org/events/
 
Description ICSE Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact ICSE workshop in part collaboration with FIAR-NET,
Year(s) Of Engagement Activity 2017
URL http://www.agilegroup.eu/wetsom2017/
 
Description Microsoft 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Explored and asserted collaboration with Microsoft, Seattle for the purposes of a research grant on fault
analysis.
Year(s) Of Engagement Activity 2017
 
Description Workshop as part of ISSRE 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Workshop as part of the Network Grant. Attended by IBM and other SMEs.
Year(s) Of Engagement Activity 2017
URL https://www.fiarnet.org/events/fiars-workshop/
 
Description Workshop as part of Network in London 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A workshop to forge links with industry in fault analyses.
Year(s) Of Engagement Activity 2018
URL https://www.fiarnet.org/events/
 
Description Workshop as part of XP 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Organised a workshop for industry/academic paper presentations.

The workshops was attended by practitioners and academics and featured a keynote industry speaker.
Year(s) Of Engagement Activity 2018