# Import modules

import numpy as np
import pandas as pd

"""
Explanation: YouTube Spam Collection Data Set (Part 1)
Source: UCI Machine Learning Repository
Original Source: YouTube Spam Collection v. 1

Alberto, T.C., Lochter J.V., Almeida, T.A. Filtragem Automática de Spam nos Comentários do YouTube. Anais do XII Encontro Nacional de Inteligência Artificial e Computacional (ENIAC'15), Natal, RN, Brazil, 2015. (preprint)
Alberto, T.C., Lochter J.V., Almeida, T.A. TubeSpam: Comment Spam Filtering on YouTube. Proceedings of the 14th IEEE International Conference on Machine Learning and Applications (ICMLA'15), 1-6, Miami, FL, USA, December, 2015. (preprint)

Contents

1 Data Set Description
2 Approach
3 Solution
3a Import modules
3b Read the data set
3c Data cleanup
3d Split the data
3e Transform the data
3f Build the model
3g Run predictions
3h Score the prediction
3i Analyze the results
3j Other learnings
4 Summary

<a id='section1'></a>
1. Data Set Description
From the description accompanying the data set, "the samples were extracted from the comments section of five videos that were among the 10 most viewed on YouTube during the collection period."
The data is available in five distinct data sets, and the data is classified as 1 for "spam" and 0 for "ham"
<a id='section2'></a>
2. Approach
Since the data set is split across five data sets, we will take two passes at the data. 
In this notebook, we will only consider the Psy data set. We will use this as a way to wrap our hands around the problem. We will not do any model tuning in this round.
Our second pass will involve merging all five data sets and then running the classification on the combined data set. In this round, we will also tune the model and the vectorizer to eke out some improvements. The notebook for this can be accessed here.
<a id='section3'></a>
3. Solution
<a id='section3a'></a>
Import initial set of modules
End of explanation
"""


# Read the data set; print the first few rows
psy = pd.read_csv('data\\Youtube01-Psy.csv')
psy.head()

"""
Explanation: <a id='section3b'></a>
Read in the data from the first CSV alone
End of explanation
"""


# Check for missing values
psy.info()

# Looks like there are no missing values. Let's proceed.
# Of the five columns, the only relevant columns for spam/ham classification are the CONTENT and CLASS columns.
# We will use just these two columns. But first, let's check the distribution of spam and ham 

psy.CLASS.value_counts()

# There is an equal distribution. Given that this is a small data set, this is probably good, 
# because the algorithm has enough items it can learn from
# Now, let us set up our X and y
X = psy.CONTENT
y = psy.CLASS

"""
Explanation: <a id='section3c'></a>
Data cleanup
End of explanation
"""


# Let us now split the data set into train and test sets
# We will use an 80/20 split
test_size = 0.2

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=test_size)

"""
Explanation: <a id='section3d'></a>
Split the data
End of explanation
"""


# Set up a vectorizer, and create a Document-Term matrix
from sklearn.feature_extraction.text import CountVectorizer

vect = CountVectorizer()
X_train_dtm = vect.fit_transform(X_train)

# Check the layout of the Document-Term matrix
X_train_dtm

"""
Explanation: <a id='section3e'></a>
Transform the data
End of explanation
"""


# We will build a Naive Bayes model for the Psy data set
from sklearn.naive_bayes import MultinomialNB
nb = MultinomialNB()

# Fit the training data (the DTM, to be precise)
nb.fit(X_train_dtm, y_train)

"""
Explanation: <a id='section3f'></a>
Build the model
End of explanation
"""


# Transform the test data to a DTM and predict
X_test_dtm = vect.transform(X_test)
y_pred_nb = nb.predict(X_test_dtm)

"""
Explanation: <a id='section3g'></a>
Run the prediction
End of explanation
"""


# Let us check the accuracy score
# It needs to better than 50%, which was the baseline
from sklearn.metrics import accuracy_score, confusion_matrix, roc_auc_score
accuracy_score(y_test, y_pred_nb)

# The accuracy score was 97%, which is indeed good. 
# Let us check the confusion matrix to get a sense of the prediction distribution
confusion_matrix(y_test, y_pred_nb)

# The model predicted 68 out of 70 instances correctly
# We had one false positive and one false negative

# What was the false positive comment? (That is, ham marked as spam)
X_test[y_pred_nb > y_test]

# And what was the false negative comment? (That is, a spam comment that went undetected)
X_test[y_pred_nb < y_test]

"""
Explanation: <a id='section3h'></a>
Score the prediction
End of explanation
"""


roc_auc_score(y_test, nb.predict_proba(X_test_dtm)[:, 1])

"""
Explanation: Both these cases are interesting. If one were to hazard a guess, the reason for the false positive could have been that the algorithm was unable to make out the words "PEACE" and "LOVE" because of the way they appeared in the comment.
The false negative comment could have been due to the way the comment was worded. We will check out the top spam and ham words, but before that, let us check the area under the ROC curve, which I anticipate would be pretty high.
End of explanation
"""


# Get the tokens from the vectorization process earlier
X_train_tokens = vect.get_feature_names()

# Here is the feature count from the Naive Bayes model
# The first line represents class 0, i.e. ham, and the second line represents spam
nb.feature_count_

ham_token_count = nb.feature_count_[0, :]
ham_token_count

spam_token_count = nb.feature_count_[1, :]
spam_token_count

# We will create a data frame of all tokens, and their corresponding ham and spam scores
tokens = pd.DataFrame({'token': X_train_tokens, 'ham': ham_token_count, 'spam': 
    spam_token_count}).set_index('token')

# Here are 10 random values drawn from the data frame
# Note that these are absolute occurrences
tokens.sample(10, random_state=50)

# Take the word "getting" for example. Was it a ham word or a spam word?
# If we concluded that it was equal-chance spam and ham, that would be incorrect
# That is because we do not know the proportion of words marked as ham and spam
# In this step, we shall standardize this to come up with a truly comparable score
class_count = nb.class_count_

class_count

tokens.ham /= class_count[0]
tokens.spam /= class_count[1]

tokens.sample(10, random_state=50)

# This gives us a sense for the relative spamminess and hamminess of a word
# In this case, 'getting' is slightly more spammy

# Now that we know this, we can compute which words tends to be the most spammy
# And which tend to be the least spammy
# Please note that since some words have a ham ratio of 0
# We can avoid dividing by 0 by adding the same small value to both the numerator and denominator
words_with_spam_score = ((tokens.spam + 1e-6)/(tokens.ham + 1e-6)).sort_values(ascending=False)

"""
Explanation: And it is! Now, let us check out the top spam and ham keywords.
<a id='section3i'></a>
Analysis
End of explanation
"""


# Top 50 most spam words
words_with_spam_score[:50].keys()

# Top 50 most ham words
words_with_spam_score[-50:].keys()

"""
Explanation: <a id='section3j'></a>
Learnings
End of explanation
"""


