Copy-paste with self-adaptation: A self-adaptive adjustment method based on copy-paste augmentation

Data augmentation diversifies the information in the dataset. For class imbalance, the copy-paste augmentation generates new class information to alleviate the impact of this problem. However, these methods rely excessively on human intuition. Over-fitting or under-fitting can occur while adding the class information, which is inappropriate. The authors propose a self-adaptive data augmentation: the copy-paste with self-adaptation (CPA) algorithm, which improves the phenomenon of over-fitting and under-fitting. For the CPA, the evaluation results of a model are taken as an important adjustment basis. The evaluation results are combined with the information of class imbalance to generate a set of class weights. Different number of class information will be replenished according to class weights. Finally, the generated images will be inserted into the training dataset and the model will start formal training. The experimental results show that CPA can alleviate class imbalance. For TT100 K dataset, YOLOv3 is trained with the optimised dataset and its AP is increased by 2% for VOC2007 dataset, the mAP of RetinaNet on optimised dataset is 78.46, which is 1.2% higher than original dataset. For COCO2017 dataset, SSD300 is trained with the optimised dataset and its AP is increased by 1.3%.

Automated summary

Data augmentation helps diversify the information in the dataset, and copy-paste augmentation generates new class information to mitigate class imbalance. The authors propose a self-adaptive data augmentation algorithm called CPA, which addresses the issues of over-fitting and under-fitting. CPA generates class weights based on model evaluation results and class imbalance information, replenishes different amounts of class information accordingly, and incorporates the generated images into the training dataset. Experimental results show that CPA can alleviate class imbalance.