Applying data mining and machine learning techniques for sentiment shifter identification

Sentiment shifters, as a set of words and expressions that can affect text polarity, play a fundamental role in opinion mining. However, the limited ability of current automated opinion mining systems in handling shifters is a major challenge. This paper presents three novel and efficient methods for identifying sentiment shifters in reviews in order to improve the overall accuracy of opinion mining systems: two data mining based algorithms and a machine learning based algorithm. The data mining algorithms do not need shifter tagged datasets. They use weighted association rule mining (WARM) for finding frequent patterns representing sentiment shifters from a domain-specific and a general corpus. These patterns include different kinds of shifter words such as shifter verbs and quantifiers and are able to handle both local and long-distance shifters. The items in WARM for the two designed methods are in the form of dependency relations and SRL arguments of sentences, respectively. Secondly, we implemented a supervised machine learning system based on semantic features of sentences for shifter identification and polarity classification. This method obviously needs shifter tagged dataset for shifter identification. We tested our proposed algorithms on polarity classification task for 2 domains: a specific domain (drug reviews) and a general domain. Experiments demonstrate that (1) the extracted shifters improve the performance of the polarity classification, (2) the proposed data mining methods outperform other implemented methods in shifter identification, and (3) the proposed semantic based machine learning method has the best efficiency among all implemented methods in polarity classification.