Journal
JOURNAL OF BIG DATA
Volume 8, Issue 1, Pages -Publisher
SPRINGERNATURE
DOI: 10.1186/s40537-021-00492-0
Keywords
Data Augmentation; Natural Language Processing; Overfitting; Big Data; NLP; Text Data
Categories
Funding
- NSF [IIS-2027890]
Ask authors/readers for more resources
In this paper, we discuss how Data Augmentation can aid in the development of Natural Language Processing, covering topics such as the motivation and specific frameworks of Data Augmentation, promising ideas yet to be tested, tools for implementation, and interesting topics around Data Augmentation.
Natural Language Processing (NLP) is one of the most captivating applications of Deep Learning. In this survey, we consider how the Data Augmentation training strategy can aid in its development. We begin with the major motifs of Data Augmentation summarized into strengthening local decision boundaries, brute force training, causality and counterfactual examples, and the distinction between meaning and form. We follow these motifs with a concrete list of augmentation frameworks that have been developed for text data. Deep Learning generally struggles with the measurement of generalization and characterization of overfitting. We highlight studies that cover how augmentations can construct test sets for generalization. NLP is at an early stage in applying Data Augmentation compared to Computer Vision. We highlight the key differences and promising ideas that have yet to be tested in NLP. For the sake of practical implementation, we describe tools that facilitate Data Augmentation such as the use of consistency regularization, controllers, and offline and online augmentation pipelines, to preview a few. Finally, we discuss interesting topics around Data Augmentation in NLP such as task-specific augmentations, the use of prior knowledge in self-supervised learning versus Data Augmentation, intersections with transfer and multi-task learning, and ideas for AI-GAs (AI-Generating Algorithms). We hope this paper inspires further research interest in Text Data Augmentation.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available