Higher accuracy and lower run time: efficient mutation analysis using non-redundant mutation operators

Mutation analysis is a powerful but computationally expensive method to measure the effectiveness of a testing or debugging technique. The high cost is due, in part, to redundant mutants generated by commonly used mutation operators. A mutant is said to be redundant if its outcome can be predicted based on the outcome of other mutants. The execution of those redundant mutants is unnecessary and wastes CPU resources. Moreover, the inclusion of redundant mutants may lead to a skewed mutant detection rate and therefore misrepresent the effectiveness of the assessed testing or debugging technique. This paper extends previous work and makes the following contributions. First, it defines and provides non-redundant versions of the conditional operator replacement, unary operator insertion, and relational operator replacement mutation operators. Second, it reports on a conducted empirical study using 10 real-world programmes that comprise a total of 410000 lines of code. The empirical study used developer-written and generated test suites. The results show how prevalent redundant mutants are and how their elimination improves the efficiency and accuracy of mutation analysis. In summary, the total mutation analysis run time decreased by more than 20% by removing redundant mutants, and the inclusion of redundant mutants led to an overestimated mutation score for all analysed test suites. Copyright (C) 2014 John Wiley & Sons, Ltd.