MULTI: Multi-objective effort-aware just-in-time software defect prediction

Context: Just-in-time software defect prediction (JIT-SDP) aims to conduct defect prediction on code changes, which have finer granularity. A recent study by Yang et al. has shown that there exist some unsupervised methods, which are comparative to supervised methods in effort-aware JIT-SDP. Objective: However, we still believe that supervised methods should have better prediction performance since they effectively utilize the gathered defect prediction datasets. Therefore we want to design a new supervised method for JIT-SDP with better performance. Method: In this article, we propose a multi-objective optimization based supervised method MULTI to build JIT-SDP models. In particular, we formalize JIT-SDP as a multi-objective optimization problem. One objective is designed to maximize the number of identified buggy changes and another object is designed to minimize the efforts in software quality assurance activities. There exists an obvious conflict between these two objectives. MULTI uses logistic regression to build the models and uses NSGA-II to generate a set of non-dominated solutions, which each solution denotes the coefficient vector for the logistic regression. Results: We design and conduct a large-scale empirical studies to compare MULTI with 43 state-of-the-art supervised and unsupervised methods under the three commonly used performance evaluation scenarios: cross-validation, cross-project-validation, and timewise-cross-validation. Based on six open-source projects with 227,417 changes in total, our experimental results show that MULTI can perform significantly better than all of the state-of-the-art methods when considering ACC and P-OPT performance metrics. Conclusion: By using multi-objective optimization, MULTI can perform significantly better than the state-of-the-art supervised and unsupervised methods in the three performance evaluation scenarios. The results confirm that supervised methods are still promising in effort-aware JIT-SDP. (c) 2017 Elsevier B.V. All rights reserved.