Designing a Better Data Representation for Deep Neural Networks and Text Classification

Traditional machine learning requires data to be described by attributes prior to applying a learning algorithm. In text classification tasks, many feature engineering methodologies have been proposed to extract meaningful features; however, no best practice approach has emerged. Traditional methods of feature engineering have inherent limitations due to loss of information and the limits of human design. An alternative is to use deep learning to automatically learn features from raw text data. One promising deep learning approach is to use convolutional neural networks. These networks can learn abstract text concepts from character representations and be trained to perform discriminate tasks, such as classification. In this paper, we propose a new approach to encoding text for use with convolutional neural networks that greatly reduces memory requirements and training time for learning from character-level text representations. Additionally, this approach scales well with alphabet size allowing us to preserve more information from the original text, potentially enhancing classification performance. By training tweet sentiment classifiers, we demonstrate that our approach uses less computational resources, allows faster training for networks and achieves similar, or better performance compared to the previous method of character encoding.