Regression Fuzzing for Deep Learning Systems

Deep learning (DL) Systems have been widely used in various domains. Similar to traditional software, DL system evolution may also incur regression faults. To find the regression faults between versions of a DL system, we propose a novel regression fuzzing technique called DRFuzz, which facilitates generating inputs that trigger diverse regression faults and have high fidelity. To enhance the diversity of the found regression faults, DRFuzz proposes a diversity-oriented test criterion to explore as many faulty behaviors as possible. Then, DRFuzz incorporates the GAN model to guarantee the fidelity of generated test inputs. We conduct an extensive study on four subjects in four regression scenarios of DL systems. The experimental results demonstrate the superiority of DRFuzz over the two compared state-of-the-art approaches, with an average improvement of 1,177% and 539% in terms of the number of detected regression faults.

Automated summary

In this paper, a novel regression fuzzing technique called DRFuzz is proposed to find regression faults between versions of DL systems. DRFuzz generates inputs that trigger diverse regression faults with high fidelity, and incorporates a diversity-oriented test criterion and the GAN model to enhance the diversity and fidelity of the found regression faults. Experimental results show that DRFuzz outperforms two state-of-the-art approaches in terms of the number of detected regression faults, with an average improvement of 1,177% and 539%.