MT-ART: A Test Case Generation Method Based on Adaptive Random Testing and Metamorphic Relation

Most of metamorphic testing (MT) research works focused on the generation and application of metamorphic relations (MRs). There is no clear conclusion about the relationship between test case generation methods and performance of MT. In this article, we introduce a novel method based on adaptive random testing (ART) and MR for MT test case generation. It proposes a family of algorithms for MT test cases generation, named as MT based ART (MT-ART). Three distances are measured to generate the next MT test case. In order to verify the performance of this method, series of experiments on four programs with different numbers of inputs are introduced. The results show that MT-ART performs better than other ART algorithms not only in test effectiveness, but also in test efficiency and test coverage. Based on this article, the following conclusions can be drawn: first, considering the effectiveness of MRs and test cases in MT may lead to better results. In this way, most of the existing research can be improved by this method. This is the most important contribute of our research. Second, not only the source test cases, but also the follow-up test cases can improve the performance of MT. Therefore, they should be considered together during the process of the next test case generation. Third, the average distance performs better than the max distance and the minimum distance in metamorphic test case selection.

Automated summary

This article introduces a novel method based on adaptive random testing and metamorphic relations for metamorphic testing test case generation, which outperforms other algorithms in test effectiveness, test efficiency, and test coverage. The study concludes that considering the effectiveness of metamorphic relations and test cases can lead to better results, both source test cases and follow-up test cases should be considered together, and average distance performs better in test case selection for metamorphic testing.