SENSitivE data: Sensitivity Analysis for Software Engineering Data

This short travel grant is to enable Prof Shepperd to visit Prof MacDonell's research labs in Auckland.In recent years there has been a welcome move to realign software engineering as an evidence-based practice. However, the underlying empirical studies are dependent upon the quality of their data sets, yet this is a surprisingly neglected topic. Our aim during the visit is to find a means to demonstrate the reliability, or freedom from noise-effects, of data-driven software engineering research results.We propose a new method for assessing the vulnerability to noise of an empirical research result, based on sensitivity analysis and a laddering technique, to determine the tipping point of published empirical studies.If successful, this will provide the software engineering research community with a general purpose method of assessing the potential impact of data quality upon their study conclusions.This research is extremely important as it proposes a solution to the problem of not knowing how vulnerable our research is to potentially problematic data sets. Given the stated aim of this research is to influence practice, this will have considerable societal impact.

Bad decisions and erroneous research results are the inevitable consequence of poor data. It is hard to put a precise figure on the cost of poor data other than to note that the annual value to the UK economy of the software industry is in excess of 50bn (e- skills UK, 2008) and a recent survey found the revenue loss due to poor data of 17% (Experian QAS, 2008). Thus we believe the potential impact of research aimed at improving software engineering practices is enormous. However, in the short term our work is aimed at researchers and enabling researchers to work more effectively so the impact upon society and the software industry (of necessity) will be indirect.