Improving Usual Naive Bayes Classifier Performances with Neural Naive Bayes based Models

Naive Bayes is a popular probabilistic model appreciated for its simplicity and interpretability. However, the usual form of the related classifier suffers from two significant problems. First, as caring about the observations' law, it cannot consider complex features. Moreover, it considers the conditional independence of the observations given the hidden variable. This paper introduces the original Neural Naive Bayes, modeling the classifier's parameters induced from the Naive Bayes with neural network functions. This method allows for correcting the first default. We also introduce new Neural Pooled Markov Chain models, alleviating the conditional independence assumption. We empirically study the benefits of these models for Sentiment Analysis, dividing the error rate of the usual classifier by 4:5 on the IMDB dataset with the FastText embedding, and achieving an equivalent F-1 as RoBERTa on TweetEval emotion dataset, while being more than a thousand times faster for inference.

Automated summary

Naive Bayes is simple and interpretable, but has problems with complex features and conditional independence assumption. By introducing Neural Naive Bayes and Neural Pooled Markov Chain models, these issues can be addressed, leading to significant improvements in sentiment analysis tasks.