to generate the values that go into the columns of our matrixwe have to tokenize our training datafrom sklearn feature_extraction text import countvectorizer from sklearn feature_extraction text import tfidftransformer from sklearn naive_bayes import multinomialnb count_vect countvectorizer(training_matrix count_vect fit_transform(training_data datathe training_matrix has dimension of ( this means that corresponds to the dataset while corresponds to the number of columns that make up the unique set of words in all posts we instantiate the countvectorizer class and pass the training_data data to the fit_transform method of the count_vect object the result is stored in training_matrix the training_matrix holds all the unique words and their respective frequencies to mitigate the problem of basing prediction on frequency count alonewe will import the tfidftransformer that helps to smooth out the inaccuracies in our datamatrix_transformer tfidftransformer(tfidf_data matrix_transformer fit_transform(training_matrixprint(tfidf_data[ : todense()tfidf_data[ : todense(only shows truncated list of three rows by , columns matrix the values seen are the term frequency--inverse document frequency that reduce the inaccuracy resulting from using frequency countmodel multinomialnb(fit(tfidf_datatraining_data targetmultinomialnb is variant of the naive bayes model we pass the rationalized data matrixtfidf_data and categoriestraining_data targetto its fit method prediction to test whether our model has learned enough to predict the category that an unknown post is likely to belong towe have the following sample datatest_data ["my god is good""arm chip set will rival intel"test_counts count_vect transform(test_datanew_tfidf matrix_transformer transform(test_counts