Naive Bayes for SA in Scikit Learn - how does it work
Collecting your data

From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of words (the paragraph) to a number which represents the rating. I will assume that you also have labels for some of the comments which we will call your training set.

I do not have access to your data but I have a similar dummy example which uses comments and tries to classify them as being a review of a hotel or a car rental. I think this example would suit this purpose nicely.

Pre-process the data

There's many techniques we can use for this purpose. For example some popula rones are bag-of-words, tf-idf, and n-grams. These techniques will take a comment and vectorize it into a set of features. The features which we will use are learned from the training set and then will be the same for new comments coming in. We will use bag-of-words for this example.

Bag-of-words

This method has a dictionary of words which we identify from our training set. We take all the unique words after stemming. We then count the number of times that each word shows up in the comment, that is the vector that represents a given instance. For example if our dictionary looks like


  ['car', 'book', 'test', 'wow']


And the comment was 


  Wow, this car is AMAZING!!!!!


The resulting vector would be


  [1, 0, 0, 1]


We will repeat this vectorization process for each comment. We will then be left with a matrix where the rows represents each comment and the columns represent the frequency at which these words appear in the comment. We will call this matrix $X$ it is our dataset. Each of these instances will also have a label we will call that vector $Y$. We want to map each row in $X$ to it's label $Y$.

Assume this is the data you have for 40 instances in a training dataset for the cars/hotel comments. With it's corresponding label on the right. Label 0 is for cars and label 1 is for hotels.



The columns of the matrix $X$ are


  ['car', 'passenger', 'seat', 'drive', 'power', 'highway', 'purchase',
  'hotel', 'room', 'night', 'staff', 'water', 'location']


Split the data

We will split the data in order to test the accuracy of our model

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.3)




Apply Naive Bayes

from sklearn.naive_bayes import GaussianNB

clf = GaussianNB()
clf = clf.fit(X_train, y_train)
clf.score(X_test, y_test)


This gives a result of 


  1.0


Perfect classification!



How does it work?

Gaussian Naive Bayes assumes that each feature is described by a Gaussian distribution (you can pick other distributions which may be better suited for your data, such as multinomial). It will then assume that each feature is entirely independent of one another. Thus we can say that the probability of 

$P(Y=0|X) = P(X_0|Y=0)P(X_0) * ... * P(X_n|Y=0)P(X_n)$

$P(Y=1|X) = P(X_0|Y=1)P(X_0) * ... * P(X_n|Y=1)P(X_n)$

for your $n$ features. The the one that is the largest is going to be our label. If $P(Y=1|X)$ is larger then we will say that the comment was that of a car. So to do this we need to train the parameters of our probability distributions $P(X_i|Y)$. To calculate this term we need to know the parameters of our distribution, in our case we are using a Gaussian distribution, thus we need the mean and the variance $\mathcal{N}(\mu, \theta)$.

We will build a dictionary of values for each label that contains the mean and variance for each feature. This is the training stage!

Homemade Naive Bayes

import csv
import random
import math
import pandas as pd
import numpy as np

class NaiveBayes(object):

    def __init__(self):
        self.groupClass = None
        self.stats = None

    def calculateGaussian(self, x, mean, std):    
        exponent = np.exp(-1*(np.power(x-mean,2)/(2*np.power(std,2))))
        std[std==0] = 0.00001
        return (1 / (np.sqrt(2*math.pi) * std)) * exponent

    def predict(self, x):
        probs = np.ones((len(x), len(self.stats)))

        for ix, instance in enumerate(x):
            for label_ix, label in enumerate(self.stats):
                probs[ix, int(label)] = probs[ix, int(label)] * \
                                      np.prod(self.calculateGaussian(instance, self.stats[label][0], self.stats[label][1]))
        return np.argmax(probs, 1)

    def score(self, x, y):
        pred = self.predict(x)
        return np.sum(1-np.abs(y - pred))/len(x)

    def train(self, x, y):
        self.splitClasses(x, y)
        self.getStats()
        pass

    def splitClasses(self, x, y):
        groupClass = {}

        for instance, label in zip(x, y):
            if not label in groupClass:
                groupClass.update({label: [instance]})
            else:
                groupClass[label].append(instance)
        self.groupClass = groupClass

    def getStats(self):
        stats = {}

        for label in self.groupClass:
            mean = np.mean(np.asarray(self.groupClass[label]), 0)
            std = np.std(np.asarray(self.groupClass[label]), 0)
            stats.update({label: [mean, std]})
        self.stats = stats

clf = NaiveBayes()
clf.train(X_train, y_train)
clf.score(X_test, y_test)


Once again we get an accuracy of 1.0 on the testing set!