期刊
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING
卷 46, 期 3, 页码 302-320出版社
IEEE COMPUTER SOC
DOI: 10.1109/TSE.2018.2852744
关键词
Testing; Subspace constraints; Computational efficiency; Fault detection; Analytical models; Software systems; Combinatorial testing; random testing; adaptive random testing
资金
- National Key Research and Development Plan [2018YFB1003800]
- Program B for Outstanding PhD Candidate of Nanjing University [201701B028]
- DAASE EPSRC [EP/J017515/1]
- EPSRC Fellowship [EP/P023991/1]
- EPSRC [EP/P023991/1, EP/J017515/1] Funding Source: UKRI
We present an empirical comparison of three test generation techniques, namely, Combinatorial Testing (CT), Random Testing (RT) and Adaptive Random Testing (ART), under different test scenarios. This is the first study in the literature to account for the (more realistic) testing setting in which the tester may not have complete information about the parameters and constraints that pertain to the system, and to account for the challenge posed by faults (in terms of failure rate). Our study was conducted on nine real-world programs under a total of 1683 test scenarios (combinations of available parameter and constraint information and failure rate). The results show significant differences in the techniques' fault detection ability when faults are hard to detect (failure rates are relatively low). CT performs best overall; no worse than any other in 98 percent of scenarios studied. ART enhances RT, and is comparable to CT in 96 percent of scenarios, but its computational cost can be up to 3.5 times higher than CT when the program is highly constrained. Additionally, when constraint information is unavailable for a highly-constrained program, a large random test suite is as effective as CT or ART, yet its computational cost of test generation is significantly lower than that of other techniques.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据