Stable Prediction of Defect-Inducing Software Changes (SPDISC)

Lead Research Organisation: University of Birmingham
Department Name: School of Computer Science

Abstract

Context: software systems have become ever larger and more complex. This inevitably leads to software defects, whose debugging is estimated to cost the global economy 312 billion USD annually. Reducing the number of software defects is a challenging problem, and is particularly important considering the strong pressure towards rapid delivery. Such pressure impedes different parts of the software source code to all receive equally large amount of inspection and testing effort.

With that in mind, machine learning approaches have been proposed for predicting defect-inducing changes in the source code as soon as these changes finish being implemented. Such approaches could enable software engineers to target special testing and inspection attention towards parts of the source code most likely to induce defects, reducing the risk of committing defective changes.

Problem: the predictive performance of existing approaches is unstable, because the underlying defect generating process being modelled may vary over time (i.e., there may be concept drift). This means that practitioners cannot be confident about the prediction ability of existing approaches -- at any given point in time, predictive models may be performing very well or failing dramatically.

Aim and vision: SPDISC aims at creating more stable models for predicting defect-inducing changes, through the development of a novel machine learning approach for automatically adapting to concept drift. When integrated with software versioning systems, the models will provide early, reliable and automated defect-inducing change alerts throughout the lifetime of software projects.

Impact: SPDISC will enable a transformation in the way software developers review and commit their changes. By creating stable models to make software developers aware of defect-inducing changes as soon as these are implemented, it will allow targeted inspection and testing attention towards defect-inducing code throughout the lifetime of software projects. This will reduce the debugging cost and ultimately lead to better software quality.

Proposed approach: an online learning algorithm will be developed to process incoming data as they become available, enabling fast reaction to concept drift. Concept drift will be detected using methods designed to cope with class imbalance, which typically occurs in prediction of defect-inducing software changes. Class imbalance refers to the issue of having a much smaller number of defect-inducing changes than the number of safe changes. The proposed approach will also make use of data from different projects (i.e., transfer learning between domains) to speed up adaptation to concept drift.

Novelty: SPDISC is the first proposal to look into the stability of predictive performance over time in the context of defect-inducing software changes. Most previous work ignored the fact that predictions are required over time, being oblivious of the instability of predictive performance in this problem. To deal with instability, SPDISC will develop the first online transfer learning approach for predicting defect-inducing software changes.

Ambitiousness: online transfer learning between domains with concept drift is not only a very new area of research in software engineering, but also in machine learning. Very few approaches exist for that, and none of them can deal with class-imbalanced problems. Therefore, SPDISC will not only advance software engineering by enabling a transformation in the way software developers review and commit their changes, but also advance the area of machine learning itself.

Timeliness: given the current size and complexity of software systems, the increased number of life-critical applications, and the high competitiveness of the software industry, approaches for improving software quality and reducing the cost of producing and maintaining software are currently of utmost importance.

Planned Impact

SPDISC's beneficiaries are the software industry, software users and related scientific communities.

1) Software Industry
The software industry is SPDISC's main beneficiary. The UK software industry is estimated to be worth more than 9bn GBP, and is the second largest market by value in the EU. Globally, the software industry's estimated value is over 407bn USD. And yet, the global cost of debugging software is estimated to be 312 billion USD annually, representing an enormous loss of revenue. SPDISC will lead to an impact on the economy by reducing debugging cost and increasing software quality.

In particular, SPDISC will empower software developers with early, reliable and automated alerts of defect-inducing software changes throughout the lifetime of software projects. It will enable a transformation in the way software changes are reviewed and committed in software development companies who use software versioning and bug-tracking systems. Defect-inducing changes will be automatically pinpointed for attention right after their implementation, allowing easy and wise allocation of the limited testing and inspection resources. This is specially desirable in companies leaning towards a more agile software development process.

As the software changes will be fresh in the developers' minds when defect alerts are triggered, their inspection will be much cheaper than later debugging cost. In addition, changes typically have few lines of code, further facilitating inspection. Therefore, SPDISC's approach will reduce the risk of committing changes that will lead to defects, reducing debugging cost and increasing software quality. The lower debugging cost will translate into cheaper software cost, as finding and fixing defects typically takes 50% of a software developer's time.

From a project management perspective, as each software change is inherently associated to a single developer, the assignment of developers to inspect defect-inducing changes will be straightforward. With SPDISC, the task of deciding which parts of the source code should receive increased attention and by whom can be delegated to the software developers themselves, freeing project managers to other tasks.

Both large enterprises and SMEs can benefit from SPDISC, as its approach automatically adapts to different environments. I anticipate that software development tools based on SPDISC will be commercialised in the future. One of SPDISC's industrial partners has already expressed interest in doing that. This will assist SMEs in benefitting from SPDISC, increasing their competitiveness and driving faster and more balanced economic growth. This will in turn lead to an impact on society by increasing wealth and employment.

2) Software Users
The more cost-effective software development enabled by SPDISC will consequently bring benefits to software users, who can be private users, users of public services, or other enterprises. Cheaper cost will facilitate access of private users and public services to software. Higher quality will improve quality of life through better and safer software experience. This is key to a world of smart cities, which are greatly controlled by software. It is also important to life-critical software applications, which could pose serious threats if defective. Cheaper and higher quality software will increase the competitiveness of other enterprises who depend on software, driving faster economic growth. Extensions of SPDISC's approach can also potentially help to solve other data analytics problems than defect prediction.

3) Scientific Communities
SPDISC will create a tighter bond between software engineering and machine learning through its new machine learning approach for software engineering. These two areas will benefit from this research. There will also be some impact on mathematical sciences, as part of SPDISC's foundation lies in this area. More details are in the academic beneficiaries summary.

Related Projects

Project Reference Relationship Related To Start End Award Value
EP/R006660/1 03/01/2018 03/09/2018 £100,542
EP/R006660/2 Transfer EP/R006660/1 04/09/2018 01/11/2019 £47,775
 
Description Context: software systems have become ever larger and more complex. This inevitably leads to software defects, whose debugging is estimated to cost the global economy 312 billion USD annually. Reducing the number of software defects is a challenging problem, and is particularly important considering the strong pressure towards rapid delivery. Such pressure impedes different parts of the software source code to all receive equally large amount of inspection and testing effort.

With that in mind, machine learning approaches have been proposed for predicting defect-inducing changes in the source code as soon as these changes finish being implemented. Such approaches could enable software engineers to target special testing and inspection attention towards parts of the source code most likely to induce defects, reducing the risk of committing defective changes.

Problem: the predictive performance of existing approaches is unstable, because the underlying defect generating process being modelled may vary over time (i.e., there may be concept drift). This means that practitioners cannot be confident about the prediction ability of existing approaches -- at any given point in time, predictive models may be performing very well or failing dramatically.

Key findings: we provided a detailed understanding of the characteristics of concept drift in prediction of defect-inducing software changes, enabling new approaches to be proposed to overcome the problem posed above. We also performed the first detailed investigation of the benefit of using data from different projects to improve predictive performance in realistic online learning scenarios faced by prediction of defect-inducing software changes. Three different approaches to make use of such data have been proposed to solve the problem posed above. Two of these approaches perform particularly well, helping the predictive performance of prediction models to be more consistently high, dealing the key problem proposed to be addressed in this project. Improvements in predictive performance are of up to 40% during periods of likely concept drifts, and up to 53.9% during the initial stage of the projects. A case study with industry has also been performed, showing that these approaches are not only helpful for open source, but also proprietary projects. Such approaches can be much more reliably adopted by practitioners than previous approaches. When adopted in practice, they have the potential to help significantly reduce the number of software defects.

Future work: this project has opened up the path for research in a number of different areas, including further research on how to deal with different types of concept drift, how to automatically tune hyperparameters that control machine learning approaches in realistic scenarios, and how to better use data from different domains to improve predictions in a given domain. Some papers have already been submitted on these topics, and a further case study with industry is being performed. Tools could potentially be developed to make the approaches proposed in this project available for practitioners.
Exploitation Route A software tool can be developed for practitioners to be able to adopt our proposed approach within their software development environments. This could potentially be done through one of the industrial partners of the project, or through the creation of a spin out.

Related approaches developed by this grant for data stream learning under multiple types of concept drift and with transfer learning could also potentially be applied to different real world problems.
Sectors Digital/Communication/Information Technologies (including Software)

 
Description We have shown that our approaches for predicting defect-inducing software changes can provide more consistently high predictive performance in open source projects. A case study of our approaches for predicting defect-inducing software changes has been performed with a company using their proprietary data, showing that these approaches can also improve predictive performance in such scenarios. This shows that such approaches can potentially be adopted by practitioners to help preventing defects in software code both in open source and proprietary data.
Sector Digital/Communication/Information Technologies (including Software)
 
Description IASESE School
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact I gave a tutorial entitled "Data Science for Software Engineering: Important Considerations and Typical Setbacks" at the 15th International Advanced School on Empirical Software Engineering (IASESE 2018). The tutorial discussed how to apply data science for software engineering, including problems such as software defect prediction investigated in this grant. The tutorial raised the audience's awareness of important considerations to make when applying data science for software engineering, and typical setbacks resulting from ignoring such considerations. It provided attendees with knowledge on how to make more informed decisions when applying data science to software engineering, increasing their skill level in this area. The tutorial was attended by around 35 researchers, students and practitioners.
 
Title A12 Effect Size 
Description Implementation of A12 effect size, facilitating other researchers' use of this measure of effect size in their experimental analyses. 
Type Of Material Improvements to research infrastructure 
Year Produced 2019 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt this measure of effect size in their experimental studies. So far, 3 downloads of the tool have been performed. 
URL https://zenodo.org/record/3353573
 
Title CommitGuru - Chinese 
Description This is an extension of the Commit Guru tool to enable collecting just-in-time software defect prediction data from Chinese git repositories. It was used in the following paper to collect proprietary data in the following paper: TABASSUM, S.; MINKU, L.L.; FENG, D.; CABRAL, G.; SONG, L. . "An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect Prediction", 2020 International Conference on Software Engineering (ICSE), 2020. 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers will be able to use this tool to collect just-in-time software defect prediction data from Chinese git repositories. This tool has been used to perform a just-in-time software defect prediction case study with a Chinese company. 
URL https://zenodo.org/record/3684635
 
Title ICSE 2020 
Description Novel methods to make use of cross-project data for creating models for predicting defect-inducing software changes in realistic environments has been proposed. The methods were published at: TABASSUM, S.; MINKU, L.L.; FENG, D.; CABRAL, G.; SONG, L. . "An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect Prediction", 2020 International Conference on Software Engineering (ICSE), 2020 (accepted). 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their software defect prediction studies, being able to perform more realistic studies of prediction of defect-inducing software changes. However, the methodology was recently accepted for publication. Therefore, it is early to quantify its impact in practice. 
 
Title IJCNN 2019 Melanie: Multi-Source Transfer Learning for Non-Stationary Environments 
Description A novel method to transfer knowledge between domains in data stream learning. The method was published at: DU, H.; MINKU, L. L.; ZHOU, H. . "Multi-Source Transfer Learning for Non-Stationary Environments", Proceedings of the International Joint Conference on Neural Networks (IJCNN), July 2019. The source code of the implementation is available publicly at Github: https://github.com/nino2222/Melanie 
Type Of Material Improvements to research infrastructure 
Year Produced 2019 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their transfer learning studies in data stream mining. 
URL https://github.com/nino2222/Melanie
 
Title TNNLS 2020 A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space 
Description A novel method to deal with multiple types of concept drift in data stream learning based on diveristy and clustering in the model space mechanisms. The method was published at: CHIU, C. W.; MINKU, L. L. . "A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space", IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020. The source code of the implementation is available publicly at GitHub (https://github.com/michaelchiucw/CDCMS) and Zenodo (https://zenodo.org/record/4294789#.YDFup8-mOgU). 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their data stream learning studies. 
URL https://zenodo.org/record/4294789#.YDFup8-mOgU
 
Title ICSE 2020 algorithm 
Description A novel algorithm for prediction of defect-inducing software changes using cross-project data has been proposed and implemented. The algorithm was published at: TABASSUM, S.; MINKU, L.L.; FENG, D.; CABRAL, G.; SONG, L. . "An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect Prediction", 2020 International Conference on Software Engineering (ICSE), 2020. The algorithm is able to operate in realistic scenarios that take the chronology of the data into account, and achieves better predictive performance than other algorithms proposed in the literature. 
Type Of Material Computer model/algorithm 
Year Produced 2020 
Provided To Others? Yes  
Impact A case study with a company has been performed, and the results were positive. The company has now provided more data for an additional case study with this algorithm. 
 
Title ICSE 2020 data collection tool 
Description This is an extension of the Commit Guru tool to enable collecting just-in-time software defect prediction data from Chinese git repositories. It was used in the following paper to collect proprietary data in the following paper: TABASSUM, S.; MINKU, L.L.; FENG, D.; CABRAL, G.; SONG, L. . "An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect Prediction", 2020 International Conference on Software Engineering (ICSE), 2020. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers will be able to use this tool to collect just-in-time software defect prediction data from Chinese git repositories. This tool has been used to perform a just-in-time software defect prediction case study with a Chinese company. 
URL https://zenodo.org/record/3684635
 
Title IJCNN 2019 Melanie algorithm 
Description A novel algorithm to transfer knowledge between domains in data stream learning. The method was published at: DU, H.; MINKU, L. L.; ZHOU, H. . "Multi-Source Transfer Learning for Non-Stationary Environments", Proceedings of the International Joint Conference on Neural Networks (IJCNN), July 2019. The source code of the implementation is available publicly at Github: https://github.com/nino2222/Melanie 
Type Of Material Computer model/algorithm 
Year Produced 2019 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their transfer learning studies in data stream mining. 
URL https://github.com/nino2222/Melanie
 
Title MARLINE:Multi-Source Mapping Transfer Learning forNon-Stationary Environments 
Description This release include grid searches' results and Supplementary Material for {MARLINE}: Multi-Source Mapping TransferLearning for Non-Stationary Environments. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers will be able to replicate the results in our paper: DU, H.; MINKU, L.; ZHOU, H. . "MARLINE: Multi-Source Mapping Transfer Learning for Non-Stationary Environments", 20th IEEE International Conference on Data Mining (ICDM), 10 pages, November 2020. 
URL https://zenodo.org/record/4040990
 
Title TNNLS 2020 algorithm 
Description A novel algorithm to deal with multiple types of concept drift in data stream learning based on diveristy and clustering in the model space mechanisms. The method was published at: CHIU, C. W.; MINKU, L. L. . "A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space", IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020. The source code of the implementation is available publicly at GitHub (https://github.com/michaelchiucw/CDCMS) and Zenodo (https://zenodo.org/record/4294789#.YDFup8-mOgU). 
Type Of Material Computer model/algorithm 
Year Produced 2020 
Provided To Others? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their data stream learning studies. 
URL https://zenodo.org/record/4294789#.YDFxfs-mOgV
 
Title michaelchiucw/CDCMS: CDCMS-TNNLS2020 
Description This version of CDCMS is used in [1]. [1] CHIU, C. W.; MINKU, L. L. . "A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space", IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020 (accepted). 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Datasets generated for the study conducted in the paper CHIU, C. W.; MINKU, L. L. . "A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space", IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020 
URL https://zenodo.org/record/4294789
 
Description Are 20% of Files Responsible for 80% of Defects? 
Organisation University of Sheffield
Department Department of Computer Science
Country United Kingdom 
Sector Academic/University 
PI Contribution I contributed with knowledge on software defect prediction in discussions about the research topic, and helped with: the formulation of the research questions, the analysis of the results and their potential impact on software defect prediction studies, writing parts of the paper, and discussing the presentation prepared for delivery at the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) 2018.
Collaborator Contribution My partner developed the approach to investigate whether 20% of files are responsible for 80% of defects, discussed the research topic, formulated research questions, designed and ran experiments, analysed the results, wrote a large portion of the paper, prepared and delivered a presentation at the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) 2018.
Impact WALKINSHAW, N.; MINKU, L. . "Are 20% of Files Responsible for 80% of Defects?", Proceedings of the 9th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), p. 2.1:2.10, October 2018. Collaboration involving the disciplines of data analytics and software engineering.
Start Year 2018
 
Description Data mining + optimisation for software engineering 
Organisation North Carolina State University
Country United States 
Sector Academic/University 
PI Contribution I have organised a NII Shonan Meeting to discuss this research area. I contributed with knowledge on data mining, optimisation and their combination to solve software engineering problems such as software defect prediction. I participated in discussions and wrote part of the paper published on the topic at the 2018 International Conference on Mining Software Repositories (https://arxiv.org/abs/1801.10241) and helped to revise the paper. I also helped to conduct a systematic literature review and wrote part of the extended version of the paper to a journal. Finally, I helped with the preparation of a keynote on the topic, delivered by Prof. Tim Menzies at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2018.
Collaborator Contribution All partners provided similar, but complementary contributions.
Impact NAIR, V.; AGRAWAL, A.; CHEN, J.; FU, W.; MATHEW, G.; MENZIES, L.; MINKU, L; WAGNER, M.; YU, Z. . "Data-Driven Search-Based Software Engineering", International Conference on Mining Software Repositories (MSR), p. 341--352, May 2018. AGRAWAL, A.; MENZIES, T.; MINKU, L.L.; WAGNER, M.; YU, Z. ."Better Software Analytics via "DUO":Data Mining Algorithms Using/Used-by Optimizers", Empirical Software Engineering Journal, 2020 (in press). Tim Menzies was invited to give a keynote about our work on Data-Driven Search-Based Software Engineering at ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2018. Tim Menzies is a collaborator of the SPDISC project. The keynote was prepared by the team who wrote the paper on the topic (https://arxiv.org/abs/1801.10241), and was delivered by Tim Menzies. This is a collaboration between the areas of artificial intelligence (including machine learning and optimisation) and software engineering.
Start Year 2018
 
Description Dealing with Real and Virtual Concept Drifts 
Organisation Federal University of Pernambuco
Country Brazil 
Sector Academic/University 
PI Contribution Proposing the research problem, guiding the proposal of the machine learning approach to solve the problem, guiding the design of experiments to evaluate the approach, guiding the evaluation of the approach, guiding and revising the writing of the paper.
Collaborator Contribution Discussing the proposed approach and experimental design, implementing the approach, running experiments, analysing the results, and writing first draft of the paper.
Impact OLIVEIRA, G. H. F. M.; MINKU, L. L.; OLIVEIRA, A. L. I. . "GMM-VRD: A Gaussian Mixture Model for Dealing With Virtual and Real Concept Drifts", Proceedings of the International Joint Conference on Neural Networks (IJCNN), July 2019.
Start Year 2018
 
Description Github software changes dataset collection 
Organisation Concordia University
Country Canada 
Sector Academic/University 
PI Contribution Myself and my team contributed with the proposal of the research topic, formulation of research questions, development of new approach to predict defects in software changes, design of experiments, experimental runs, analysis of results, paper writing, paper response preparation and paper revision.
Collaborator Contribution My partner contributed with the collection of Github data to evaluate the proposed approach.
Impact The following paper was accepted for publication: Cabral, G.; Minku, L.; Shibab, E.; Mujahid, S. Class Imbalance Evolution and Verification Latency in Just-in-Time Software Defect Prediction. International Conference on Software Engineering (ICSE 2019).
Start Year 2018
 
Description Transfer learning in non-stationary environments 
Organisation University of Leicester
Country United Kingdom 
Sector Academic/University 
PI Contribution Proposing research problem, guiding the proposal of the machine learning approach to solve the problem, guiding the design of experiments to evaluate the approach, guiding the evaluation of the approach, guiding and revising the writing of the paper.
Collaborator Contribution Discussing the proposed approach and experimental design, implementing the approach, running experiments, analysing the results, and writing first draft of the paper.
Impact DU, H.; MINKU, L.; ZHOU, H. . "MARLINE: Multi-Source Mapping Transfer Learning for Non-Stationary Environments", 20th IEEE International Conference on Data Mining (ICDM), 10 pages, November 2020. DU, H.; MINKU, L. L.; ZHOU, H. . "Multi-Source Transfer Learning for Non-Stationary Environments", Proceedings of the International Joint Conference on Neural Networks (IJCNN), 8 pages, July 2019.
Start Year 2018
 
Title IJCNN 2019 Melanie: Multi-Source Transfer Learning for Non-Stationary Environments 
Description A novel method to transfer knowledge between domains in data stream learning. The method was published at: DU, H.; MINKU, L. L.; ZHOU, H. . "Multi-Source Transfer Learning for Non-Stationary Environments", Proceedings of the International Joint Conference on Neural Networks (IJCNN), July 2019. The source code of the implementation is available publicly at Github: https://github.com/nino2222/Melanie 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact Other researchers and practitioners will be able to adopt the same methodology in their transfer learning studies in data stream mining. 
 
Title Software - ICSE 2020 data collection 
Description This is an extension of the Commit Guru tool to enable collecting just-in-time software defect prediction data from Chinese git repositories. It was used in the following paper to collect proprietary data in the following paper: TABASSUM, S.; MINKU, L.L.; FENG, D.; CABRAL, G.; SONG, L. . "An Investigation of Cross-Project Learning in Online Just-In-Time Software Defect Prediction", 2020 International Conference on Software Engineering (ICSE), 2020. 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Other researchers will be able to use this tool to collect just-in-time software defect prediction data from Chinese git repositories. This tool has been used to perform a just-in-time software defect prediction case study with a Chinese company. 
 
Title michaelchiucw/CDCMS: CDCMS-TNNLS2020 
Description This version of CDCMS is used in [1]. [1] CHIU, C. W.; MINKU, L. L. . "A Diversity Framework for Dealing with Multiple Types of Concept Drift Based on Clustering in the Model Space", IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2020 (accepted). 
Type Of Technology Software 
Year Produced 2020 
Impact Other researchers and practitioners will be able to adopt the same methodology in their data stream learning studies. 
URL https://zenodo.org/record/4294789
 
Description Artificial Intelligence: What Is It And How Can It Help Us? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact This talk was part of a science festival at the North and North-East regions of Brazil. It discussed what is artificial intelligence and how it can help us on various different tasks, including tasks investigated in this grant. The talk was broadcasted live and is available on YouTube. It currently has 407 views, 75 likes and 0 dislikes. The talk was intended at increasing awareness about artificial intelligence and encouraging pupils to join this field once they reach university level.
Year(s) Of Engagement Activity 2020
URL https://youtu.be/VUiySDwKha4
 
Description How will machine learning / AI change the way IT professionals work? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This was a Panel on "How will machine learning / AI change the way IT professionals work" at the 2020 International Conference on the Quality of Information and Communications Technology. It aimed at sparking discussions and raising awareness of how machine learning and artificial intelligence can benefit software practitioners. The discussion led to participants building up or changing their views in terms of the future of IT in view of machine learning and artificial intelligence.
Year(s) Of Engagement Activity 2020
URL https://2020.quatic.org/
 
Description IEEE Software Column for Practitioners -- Highlights from ICSE 2019 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact This is a column for practitioners published at IEEE Software. The intention of my contribution was to increase practitioners' awareness of the existence of intelligent automated approaches for software testing.
Year(s) Of Engagement Activity 2019
URL https://ieeexplore.ieee.org/document/8802626
 
Description The Whole is Greater than The Sum of the Parts: On the Value of Machine Learning Ensemble Methods 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact This was a keynote at the BEAR PGR Conference in Birmingham. It aimed at discussing machine learning approaches and their uses to postgraduate students who may not be from the area of computer science. Some of the machine learning approaches discussed in the talk have been influenced by this grant. The talk sparked questions, discussions and led to a student requesting detailed information on how to apply machine learning for her work.
Year(s) Of Engagement Activity 2020