Geek asks and answers

Good mood analysis algorithm

I tried the naive classifier of the bays, and it works very poorly. SVM works a little better, but still awful. Most of the articles that I read about SVM and naive bays with some variations (n-gram, POS, etc.), But they all give results close to 50% (the authors of the articles say 80% and high, but I don’t I can get accurate accurate according to real data).

Are there any more powerful methods besides lexixal analys? SVM and Bayes believe that words are independent. This approach is called the word bag. What if we assume these words are related?

For example: use the apriory algorithm to find that if the sentences contain "bad and terrible", then the probability of 70% of this sentence is negative. We can also use the distance between words, etc.

Is this a good idea or am I reinventing the wheel?

+7
source share
4 answers

You are misleading a couple of concepts. Neither Naive Bayes nor SVM are attached to the word approach. Neither the SVM nor the BOW approach has an assumption of independence between the terms.

Here are some things you can try:

  • include punctuation in your word bags; special! and? can be useful for mood analysis, while many function extractors designed to classify documents throw them.
  • the same for stop words: words like “I” and “mine” can point to subjective text.
  • build a two-stage classifier; first determine if any opinion is expressed, whether it is positive or negative
  • ​​SVM .
+6

Sentimnetal- python. 3

  • &
+2

Sentiment Analysis - . . , SemEval. Twitter. , , 2016 ( , ): http://alt.qcri.org/semeval2016/task4/data/uploads/semeval2016_task4_report.pdf

, ( ).

0

More articles:


All Articles