RE-COST: REducing the Cost of Oracles for Software Testing

Lead Research Organisation: University of Sheffield
Department Name: Computer Science

Abstract

Testing involves examining the behaviour of a system in order to discover potential faults. The problem of determining the desired correct behaviour for a given input is called the Oracle Problem. Since manual testing is expensive and time consuming there has been a great deal of work on automation and part automation of Software Testing. Unfortunately, it is often impossible to fully automate the process of determining whether the system behaves correctly. This must be performed by a human, and the cost of the effort expended is referred to as the Human Oracle Cost.RE-COST will develop Search-Based Optimisation techniques to attack the Human Oracle Cost problem quantitatively and qualitatively. The quantitative approach will develop methods and algorithms to both reduce the number of test cases and the evaluation effort per test case. The qualitative approach will develop methods and algorithms that will reduce test case cognition time.The RE-COST project seeks to transform the way that researchers and practitioners think about the problem of Software Test Data Generation. This has the potential to provide a breakthrough in Software Testing, dramatically increasing real world industrial uptake of automated techniques for Software Test Data Generation.

Publications

10 25 50
 
Description 1) Readability of automatically-generated test inputs is key when it comes to users of automatic test input generation tools in practice.

2) The readability of automatically-generated string test inputs can be improved, lowering the time needed for humans to check the corresponding outputs to those inputs, in particular through:

a) strings sourced from the Internet.

b) through combining meta-heuristic algorithms with statistical natural language models

3) That the costs of generating test data can be reduced by combining symbolic execution with search based test data generation

4) That the success rate of test data generation for Java programs may be improved when Memetic algorithms are used.
Exploitation Route The research can be used by companies that develop software testing tools. The test data generated by those tools may be made more realistic and human-readable, thus reducing costs of testing for the tool users.
Sectors Digital/Communication/Information Technologies (including Software)