Classifying crowdsourced mobile test reports with image features: An empirical study

Crowdsourced testing has become a popular mobile application testing method, and it is capable of simulating real usage scenarios and detecting various bugs with a large workforce. However, inspecting and classifying the overwhelming number of crowdsourced test reports has become a time-consuming yet inevitable task. To alleviate such tasks, in the past decades, software engineering researchers have proposed many automatic test report classification techniques. However, these techniques may become less effective for crowdsourced mobile application testing, where test reports often consist of insufficient text descriptions and rich screenshots and are fundamentally different from those of traditional desktop software. To bridge the gap, we firstly fuse features extracted from text descriptions and screenshots to classify crowdsourced test reports. Then, we empirically investigate the effectiveness of our feature fusion approach under six classification algorithms, namely Naive Bayes (NB), k-Nearest Neighbors (kNN), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF) and Convolutional Neural Network (CNN). The experimental results on six widely used applications show that (1) SVM with fused features can outperform others in classifying crowdsourced test reports, and (2) image features can improve the test report classification performance. (c) 2021 Elsevier Inc. All rights reserved.

Automated summary

Crowdsourced testing is a popular method for testing mobile applications. It can simulate real usage scenarios and detect various bugs with a large workforce. However, the inspection and classification of crowdsourced test reports can be time-consuming. To address this issue, researchers have proposed techniques for automatically classifying test reports. In this study, we fuse features from text descriptions and screenshots to classify crowdsourced test reports and evaluate the effectiveness of our approach using six classification algorithms. The results show that SVM with fused features performs the best in classifying crowdsourced test reports, and image features improve the classification performance.