Novel regularization method for the class imbalance problem

In neural network models, obtaining a high-quality dataset is critical because they are generally reliant on training data. A common problem that occurs is class imbalance, in which models tend to be biased to the majority class when the training data is not balanced. To overcome this problem, we propose a novel regularization method that provides a penalty to the loss function, using two facets of the distribution of the model's output p((y) over cap vertical bar x): (1) skewed mean and (2) variance divergence between p((y) over cap vertical bar x is an element of D+) and p((y) over cap vertical bar x is an element of D_). The experimental results demonstrate that our methods consistently improve the performance on imbalanced datasets. Moreover, the combination of two regularization methods provides a substantial performance improvement on five sentence classification datasets and also an image classification dataset; notably, state-of-the-art performances are achieved on the WikiQA and SelQA datasets.

Automated summary

In neural network models, obtaining high-quality datasets is crucial. To address the problem of class imbalance, a novel regularization method is proposed, which significantly improves the performance on multiple datasets, achieving state-of-the-art results.