FAULT ANALYSES IN INDUSTRY AND ACADEMIC RESEARCH
Lead Research Organisation:
Brunel University London
Department Name: Computer Science
Abstract
Faults in software cost the UK IT industry hundreds of millions of pounds each year; a common aim of software engineering research in academic terms is to address industry problems. The Fault Analyses in Industry and Academic Research Network (FIAR-NET) brings closer ties between industry and academic research in the area of software fault analysis. A range of pressing issues exists and it is to the mutual benefit of the two communities that they work more closely than they have in the past. This is the underpinning motivation and reason for proposing FIAR-NET.
Two key themes run through the network. Firstly, challenges of fault case-studies using industrial systems. While recent progress has been made in this area, more industrial involvement has the potential to enhance and add high value to these foundations. The theme will embrace the practicalities of predicting faults, fault data collection/assimilation (including ensuring that the data is reliable), fault data interpretation and faults in an 'agile' context, a model for developing software which is prevalent in UK IT industry. The second theme is an exploration of approaches to re-engineering code so that is in some way better and thereby mitigate and minimise faults; finally through better testing strategies (both pre- and post-release).
FIAR-NET will operate through regular workshops, events and a range of on-line resources and initiatives including a set of on-site "road-shows". An initial network of core academics and industrial partners including SMEs and large organisations will be established, maintained and then extended. A key aim is to stimulate innovative, leading research collaboration and dissemination of subsequent results. The proposed network team at Brunel builds on a track record of expertise in fault analyses and empirical software engineering, widely recognised as world-leading. This reputation is underpinned by a wide variety of past and present industrial and academic collaborations which FIAR-NET will build on in the three years of its duration and thereafter.
Both Counsell (PI) and Hierons (a core FIAR-NET member) have led successful research networks in the past and all investigators on the project have wide experience of industrial collaborations and bring that experience to the network. Finally, FIAR-NET falls within one of the EPSRC's strategic aims and portfolio plan to 'grow' research into fault-finding and fixing.
Two key themes run through the network. Firstly, challenges of fault case-studies using industrial systems. While recent progress has been made in this area, more industrial involvement has the potential to enhance and add high value to these foundations. The theme will embrace the practicalities of predicting faults, fault data collection/assimilation (including ensuring that the data is reliable), fault data interpretation and faults in an 'agile' context, a model for developing software which is prevalent in UK IT industry. The second theme is an exploration of approaches to re-engineering code so that is in some way better and thereby mitigate and minimise faults; finally through better testing strategies (both pre- and post-release).
FIAR-NET will operate through regular workshops, events and a range of on-line resources and initiatives including a set of on-site "road-shows". An initial network of core academics and industrial partners including SMEs and large organisations will be established, maintained and then extended. A key aim is to stimulate innovative, leading research collaboration and dissemination of subsequent results. The proposed network team at Brunel builds on a track record of expertise in fault analyses and empirical software engineering, widely recognised as world-leading. This reputation is underpinned by a wide variety of past and present industrial and academic collaborations which FIAR-NET will build on in the three years of its duration and thereafter.
Both Counsell (PI) and Hierons (a core FIAR-NET member) have led successful research networks in the past and all investigators on the project have wide experience of industrial collaborations and bring that experience to the network. Finally, FIAR-NET falls within one of the EPSRC's strategic aims and portfolio plan to 'grow' research into fault-finding and fixing.
Planned Impact
ADVANCEMENT OF KNOWLEDGE
Advancement of knowledge facilitated by FIAR-NET can be seen from four viewpoints. Firstly, the areas that FIAR-NET explores are open research areas. For example, the optimal set of characteristics of code and combinations of code features for predicting faults has yet to be established. Equally, the link between faults and refactoring and whether refactoring actually improves the quality of code has received very little empirical attention. Progress in each of these areas has been hampered by the lack of data from proprietary systems and so involvement of industry is vital to understanding and then solving some of these problems. The proposed network brings together an initial set of six industry partners, the BCS and four academic institutions, all with specific interests in these types of area. Secondly, the results can be adopted, adapted and pursued by industrial and academic collaborations through the project deliverables; specifically, through the workshop agendas and road-shows, both of which provide a forum for research to be further explored. The set of dissemination activities and potential for extending the network over the project duration also promote knowledge advancement. Thirdly, the research agenda will inform new knowledge about, a) collection and analysis of fault data, b) testing, the problems, differences and emphases of commercial test environments (Agile or otherwise) and c) how systems can be re-engineered to minimise faults. The project thus provides many opportunities for better allocation and prioritisation of resources by developers and project managers.
PEOPLE IMPACT
From a people perspective, the project promotes a set of skills through the investigation of research problems shared between industry and academic groups. These skills can be adopted by practitioners who wish to apply the same principles to their software and fault bases. A range of people will also benefit from the research. Software architects, managers and developers at the industrial partners will benefit directly through results and lessons generated from the network. The network will allow academic researchers in the areas that FIAR-NET covers to benefit from a shared understanding and the transfer of knowledge that academic and industrial collaboration fosters.
ECONOMIC IMPACT
From an economic viewpoint, faults in software will always occur and there will always be some cost associated with faults and fixing them when they do occur. However, there are significant potential cost savings to be gained by the UK IT industry if research can produce demonstrably successful methods of understanding faults and tackling issues surrounding faults. Firstly, resources can be pre-allocated and directed where they are most needed on a proactive rather than a reactive basis, the former of which often causes disruption to other projects. Secondly, explanations of the causes of faults can improve and streamline the processes through which software is developed. Finally, the ability to understand system behaviour in a more timely fashion ultimately frees up time for developers to work on improving/perfecting the quality of their code. The proposed research has huge potential for unlocking the opportunity that a more timely awareness and understanding of system pressures create.
SOCIETAL IMPACT
From a societal viewpoint, the research is in alignment with a key aim of any company of developing robust code and high quality products. Customers of the industrial partners will benefit in terms of the software that they 'deliver'. The value of the research also appeals to the wider societal community. In fact, the research affects anyone who uses software since improvement in the quality of code through minimisation of faults is a fundamental aim of any software developer.
Advancement of knowledge facilitated by FIAR-NET can be seen from four viewpoints. Firstly, the areas that FIAR-NET explores are open research areas. For example, the optimal set of characteristics of code and combinations of code features for predicting faults has yet to be established. Equally, the link between faults and refactoring and whether refactoring actually improves the quality of code has received very little empirical attention. Progress in each of these areas has been hampered by the lack of data from proprietary systems and so involvement of industry is vital to understanding and then solving some of these problems. The proposed network brings together an initial set of six industry partners, the BCS and four academic institutions, all with specific interests in these types of area. Secondly, the results can be adopted, adapted and pursued by industrial and academic collaborations through the project deliverables; specifically, through the workshop agendas and road-shows, both of which provide a forum for research to be further explored. The set of dissemination activities and potential for extending the network over the project duration also promote knowledge advancement. Thirdly, the research agenda will inform new knowledge about, a) collection and analysis of fault data, b) testing, the problems, differences and emphases of commercial test environments (Agile or otherwise) and c) how systems can be re-engineered to minimise faults. The project thus provides many opportunities for better allocation and prioritisation of resources by developers and project managers.
PEOPLE IMPACT
From a people perspective, the project promotes a set of skills through the investigation of research problems shared between industry and academic groups. These skills can be adopted by practitioners who wish to apply the same principles to their software and fault bases. A range of people will also benefit from the research. Software architects, managers and developers at the industrial partners will benefit directly through results and lessons generated from the network. The network will allow academic researchers in the areas that FIAR-NET covers to benefit from a shared understanding and the transfer of knowledge that academic and industrial collaboration fosters.
ECONOMIC IMPACT
From an economic viewpoint, faults in software will always occur and there will always be some cost associated with faults and fixing them when they do occur. However, there are significant potential cost savings to be gained by the UK IT industry if research can produce demonstrably successful methods of understanding faults and tackling issues surrounding faults. Firstly, resources can be pre-allocated and directed where they are most needed on a proactive rather than a reactive basis, the former of which often causes disruption to other projects. Secondly, explanations of the causes of faults can improve and streamline the processes through which software is developed. Finally, the ability to understand system behaviour in a more timely fashion ultimately frees up time for developers to work on improving/perfecting the quality of their code. The proposed research has huge potential for unlocking the opportunity that a more timely awareness and understanding of system pressures create.
SOCIETAL IMPACT
From a societal viewpoint, the research is in alignment with a key aim of any company of developing robust code and high quality products. Customers of the industrial partners will benefit in terms of the software that they 'deliver'. The value of the research also appeals to the wider societal community. In fact, the research affects anyone who uses software since improvement in the quality of code through minimisation of faults is a fundamental aim of any software developer.
People |
ORCID iD |
Steve Counsell (Principal Investigator) | |
Tracy Hall (Co-Investigator) |
Publications
Ajienka N
(2017)
An empirical study on the interplay between semantic coupling and co-change of software classes
in Empirical Software Engineering
Counsell S
(2018)
Re-visiting a Test Taxonomy with Refactoring and Defect-fix Data
Counsell S
(2017)
Assert Use and Defectiveness in Industrial Code
Destefanis G
(2017)
On the randomness and seasonality of affective metrics for software development
Ebert C
(2016)
Cyclomatic Complexity
in IEEE Software
Laplante P
(2017)
Future Software Engineering Perspectives
in IEEE Software
Description | Although the situation is improving all the time, Industry and academia still have varying goals and views on direction of research. Finding a middle ground and education of each party on the conflicts/challenges in each is still very limited. Industry don't have the time to spend carrying out activities that academia does and academics are largely shielded from the realities of coding and the pressures that involves. Many of the difficulties are embedded in each other's working lives, |
Exploitation Route | More involvement by industry in academic activities. Schemes/initiatives and sponsorships/secondments are the way forward. Academics should spend more time in industry to appreciate the problems that they face. . |
Sectors | Digital/Communication/Information Technologies (including Software) |
Description | We have established, until recently, a relationship with BAE systems (submarines and warships) and some of the outputs of FIAR-NET were presented at their Software Engineering Committee. The Head of the Group at BAE also used the empirical nature of the work we do to write papers of his own on evidence-based software engineering. Clearly, avoidance of faults is something that safety-critical systems are keen to understand. Our work was used to spread good practice in the area to other members of the SE group at BAE systems. The work with BAE was also considered for an impact case study in the latest REF (but not used). The same Head of the division Dr Ken Wallace also came to Brunel for our annual research day and was a judge at our awards ceremony as well as being a contributor to the annual Made in Brunel event where students display their research to industry over the course of an evening and prizes awarded. A grant is currently being prepared to work on Software Resilience with multiple companies as a direct lead from the fault-based events from FIAR-NET. This will be applied in areas of cyber and diverse areas such as driverless cars; put another way, the work of FIAR-NET has generated an awareness that techniques for finding and fixing are being applied in more and more areas. Many of the network participants are working with me on ongoing research topics and FIAR-NET opened up these opportunities. It informed a TSE paper just published on developer perceptions of program repair. |
First Year Of Impact | 2019 |
Sector | Aerospace, Defence and Marine |
Impact Types | Economic |
Description | Changed the way developers developed through company policy |
Geographic Reach | National |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | One of our collaborators through the Network issued new standards and changes across the UK in Software Engineering practice. |
Description | Brunel WAT scheme (Incentive Scheme) |
Amount | £2,000 (GBP) |
Organisation | Brunel University London |
Sector | Academic/University |
Country | United Kingdom |
Start | 05/2017 |
End | 09/2018 |
Description | Brunel WAT scheme (Incentive Scheme) |
Amount | £2,000 (GBP) |
Organisation | Brunel University London |
Sector | Academic/University |
Country | United Kingdom |
Start | 05/2017 |
End | 09/2018 |
Description | Concrete collaborations with BaeSystems and outcome of Visiting Industry Research Fellow |
Organisation | BAE Systems |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | The FIAR-NET project has allowed a more formal footing with BAE Systems to be established and will result in a Fellowship submission to the EPSRC. We are in the process of inviting Dr Ken Wallace to become a Visiting Industry Fellow at Brunel. |
Collaborator Contribution | Invited talks and contributions to the University's Made In Brunel. |
Impact | Visiting Industry Fellowship in process. |
Start Year | 2014 |
Description | Meeting at Keele for grant proposal group |
Organisation | Keele University |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Keele UCL Newcastle Durham Brunel |
Collaborator Contribution | Joint meeting to discuss future funding bid. |
Impact | Working documents before proposal is formally put together |
Start Year | 2017 |
Description | First Workshop |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Industry/Business |
Results and Impact | Ran a Workshop as part of FIARNET. Here is the url of the event. https://www.fiarnet.org/events/ |
Year(s) Of Engagement Activity | 2016 |
URL | https://www.fiarnet.org/events/ |
Description | ICSE Workshop |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | ICSE workshop in part collaboration with FIAR-NET, |
Year(s) Of Engagement Activity | 2017 |
URL | http://www.agilegroup.eu/wetsom2017/ |
Description | Microsoft |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Explored and asserted collaboration with Microsoft, Seattle for the purposes of a research grant on fault analysis. |
Year(s) Of Engagement Activity | 2017 |
Description | Workshop as part of ISSRE |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Workshop as part of the Network Grant. Attended by IBM and other SMEs. |
Year(s) Of Engagement Activity | 2017 |
URL | https://www.fiarnet.org/events/fiars-workshop/ |
Description | Workshop as part of Network in London |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | A workshop to forge links with industry in fault analyses. |
Year(s) Of Engagement Activity | 2018 |
URL | https://www.fiarnet.org/events/ |
Description | Workshop as part of XP |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Organised a workshop for industry/academic paper presentations. The workshops was attended by practitioners and academics and featured a keynote industry speaker. |
Year(s) Of Engagement Activity | 2018 |