NLTK: sentiment analysis: result one value
Question: So sorry for posting this, as the answer probably is in either this: [NLTK
sentiment analysis is only returning one
value](http://stackoverflow.com/questions/15106032/nltk-sentiment-analysis-is-
only-returning-one-value)

or this post: [Python NLTK not sentiment calculate
correct](http://stackoverflow.com/questions/19622538/python-nltk-not-
sentiment-calculate-correct)

but I don't get how to apply it to my code.

I'm a huge newbie at Python and NLTK and I hate that I have to bother you with
a huge block of code, so sorry once again.

With the code I use, I always get 'pos' as a result. I've tried doing the
classification by leaving the positive features out of the training set. Then
the return is always 'neutral'.

Can anybody tell me what I'm doing wrong? Thank you so much in advance! And
don't mind the random test sentence I used, it was just something that came up
while I was trying to figure out what was wrong.

    
    
    import re, math, collections, itertools
    import nltk
    import nltk.classify.util, nltk.metrics
    from nltk.classify import NaiveBayesClassifier
    from nltk.metrics import BigramAssocMeasures
    from nltk.probability import FreqDist, ConditionalFreqDist  
    from nltk.util import ngrams
    from nltk.tokenize import word_tokenize
    from sklearn.feature_extraction.text import TfidfVectorizer
    from nltk.stem.porter import *
    from nltk.stem.snowball import SnowballStemmer
    
    stemmer = SnowballStemmer("english", ignore_stopwords = True)
    
    pos_tweets = ['I love bananas','I like pears','I eat oranges']
    neg_tweets = ['I hate lettuce','I do not like tomatoes','I hate apples']
    neutral_tweets = ['I buy chicken','I am boiling eggs','I am chopping vegetables']
    
    def uni(doc):
        x = []
        y = []
        for tweet in doc:
            x.append(word_tokenize(tweet))
        for element in x:
            for word in element:
                if len(word)>2:
                    word = word.lower()
                    word = stemmer.stem(word)
                    y.append(word)
        return y
    
    def word_feats_uni(doc):
         return dict([(word, True) for word in uni(doc)])
    
    def tokenizer_ngrams(document):
        all_tokens = []
        filtered_tokens = []
        for (sentence) in document:
            all_tokens.append(word_tokenize(sentence))
        return all_tokens
    
    def get_bi (document):
        x = tokenizer_ngrams(document)
        c = []
        for sentence in x:
            c.extend([bigram for bigram in nltk.bigrams(sentence)])
        return c
    
    def get_tri(document):
        x = tokenizer_ngrams(document)
        c = []
        for sentence in x:
            c.extend([bigram for bigram in nltk.bigrams(sentence)])
        return c
    
    def word_feats_bi(doc): 
        return dict([(word, True) for word in get_bi(doc)])
    
    def word_feats_tri(doc):
        return dict([(word, True) for word in get_tri(doc)])
    
    def word_feats_test(doc):
        feats_test = {}
        feats_test.update(word_feats_uni(doc))
        feats_test.update(word_feats_bi(doc))
        feats_test.update(word_feats_tri(doc))
        return feats_test
    
    pos_feats = [(word_feats_uni(pos_tweets),'pos')] + [(word_feats_bi(pos_tweets),'pos')] + [(word_feats_tri(pos_tweets),'pos')]
    
    neg_feats = [(word_feats_uni(neg_tweets),'neg')] + [(word_feats_bi(neg_tweets),'neg')] + [(word_feats_tri(neg_tweets),'neg')]
    
    neutral_feats = [(word_feats_uni(neutral_tweets),'neutral')] + [(word_feats_bi(neutral_tweets),'neutral')] + [(word_feats_tri(neutral_tweets),'neutral')]
    
    trainfeats = pos_feats + neg_feats + neutral_feats
    
    classifier = NaiveBayesClassifier.train(trainfeats)
    
    print (classifier.classify(word_feats_test('I am chopping vegetables and boiling eggs')))
    


Answer: The solution is very simple. Your `word_feats_test` will return an empty
dictionary for the sentence `'I am chopping vegetables and boiling eggs'`;
thus the classifier is biased towards `pos` in case of no features.

I wrapped your sentence in a list:

    
    
    print(classifier.classify(word_feats_test(
          ['I am chopping vegetables and boiling eggs'])))
    

and `neutral` is printed.

You ought to use the exact same function to calculate the features for all 3:
the training set, testing set and classification.

