YUAN Lei. Study on sentiment text classification based on improved CHI feature selection[J]. 2017, 36(5): 47-51. DOI: 10.13873/j.1000-9787(2017)05-0047-05.
In order to improve the accuracy of sentiment text classification
different preprocessing methods of the sentiment of English text is studied
and an improved algorithm of Chi-square statistic( CHI) feature extraction is put forward. CHI is one of the most efficient feature selection methods
but there are two weaknesses
negative correlation phenomenon and tend to choose low-frequency feature words. In order to overcome these two shortcomings
on the basis of taking into account factors of word frequency
concentration information and dispersion information
considering the length of the text is not balanced and the distribution of feature words
word frequency is normalized
CHI feature extraction algorithm is proposed. Using classical naive Bayes and support vector machine( SVM) classification algorithms experiments is carried out on balanced corpus
imbalanced corpus and mixed-length corpus
and experimental results show that the new method improves accuracy of sentiment text classification.
Chi-square Statistics Feature Selection Based on Term Frequency and Distribution for Text Categorization [J] . Chuanxin Jin,Tinghuai Ma,Rongtao Hou,Meili Tang,Yuan Tian,Abdullah Al-Dhelaan,Mznah Al-Rodhaan. IETE Journal of Research . 2015 (4)