df[5].unique()

"""
Explanation: Conclusion
In this dataset, Fails take roughly 1%. 
If it is a representative sample of the real data, then running machine learning on the whole set will just not make sense. Any classifier that just predict "Success" for every line will attain 99% accuracy. 
This means that I need to collect data for "Fail" cases and randomly sample data for "Success" in roughly equal amounts and then look at machine learning (classifier) for such sets to see the true accuracy of the algorithm.
Next Steps
I want to examine this dataset to see if there are any obvious correlations and to understand the data I have in my columns.
authentication type
End of explanation
"""


df[6].unique()

"""
Explanation: logon type
End of explanation
"""


df[7].unique()

df.groupby([5,8]).count()

df.groupby([6,8]).count()

df.groupby([7,8]).count()

df.groupby([6,7]).count()

"""
Explanation: authentication orientation
End of explanation
"""


print len(df[3].unique()), len(df[4].unique())

"""
Explanation: This is a simple way to see if there are any labels in columns 5-7 that predict the outcome. (answer: not really as the count for most events that can be interpreted this way is too low). Also I am try to see if there are any interesting correlations between labels.
End of explanation
"""


df["source_user"], df["source_domain"] = zip(*df[1].str.split('@').tolist())

df["source_user"]=df["source_user"].str.rstrip('$')

df["destination_user"], df["destination_domain"] = zip(*df[2].str.split('@').tolist())
df["destination_user"]=df["destination_user"].str.rstrip('$')

df['same_user']=(df['destination_user']==df['source_user'])
df['same_domain']=(df['destination_domain']==df['source_domain'])

df['same_user'].value_counts()

df['same_domain'].value_counts()

df['source_domain'].unique()

df['destination_domain'].unique()

df['source_user'].unique()

df['destination_user'].unique()

"""
Explanation: Potentially too many variables to be used in analysis
End of explanation
"""


good=df[~df.source_user.str.startswith("U")] 
good=good.source_user[~good.source_user.str.startswith('C')]
good.unique()

good=df[~np.logical_or(df.destination_user.str.startswith("U"), df.destination_user.str.startswith("C"))] 
#good=good.destination_user[~good.destination_user.str.contains('C')]
good.destination_user.unique()

"""
Explanation: Potentially too many variable. I now want to explore what users I have in addition to C-numbers and U-numbers. (C=computer and U=user?)
End of explanation
"""


dd=df['destination_domain'].str.startswith('C')
print min(df['destination_domain'][dd].str.slice(1).astype(int)), max(df['destination_domain'][dd].str.slice(1).astype(int))
dd=df[~df.destination_domain.str.startswith('C')]
print dd.destination_domain.unique()

sd=df['source_domain'].str.startswith('C')
print min(df['source_domain'][sd].str.slice(1).astype(int)), max(df['source_domain'][sd].str.slice(1).astype(int))
sd=df[~df.source_domain.str.startswith('C')]
print sd.source_domain.unique()

"""
Explanation: Idea: one can expand this column into into 6 categories: C-users, U-users, 'ANONYMOUS LOGON', 'LOCAL_SERVICE', 'SYSTEM', 'NETWORK SERVICE'
End of explanation
"""


df['source_user_comp_same']=(df[3]==df['source_user'])
df['destination_user_comp_same']=(df['destination_user']==df[4])
df['same_comp']=(df[3]==df[4])
df['source_domain_comp_same']=(df[3]==df['source_domain'])
df['destination_domain_comp_same']=(df['destination_domain']==df[4])

df['source_user_comp_same'].value_counts()

df['destination_user_comp_same'].value_counts()

df['same_comp'].value_counts()

df['source_domain_comp_same'].value_counts()

df['destination_domain_comp_same'].value_counts()

"""
Explanation: Conclusion
This dataset contains columns of categorical data (aside from time). To work with this data, each label should be converted to its own column with values 1 (True) if the label applies and 0 (False) otherwise. Some columns (5-7) contain ~10 labels, where as other columns contain ten thousands of labels. I will ignore the 2nd class of labels on the first pass. Instead I will consider when these labels coincide. This way I will prevent my set of features from exploding. Also the 2nd class of labels most likely comes from some ordering of computers and users in the lab. Considering if one wants to authenticate to the same computer or to a different computer should matter more for authentication success than specific computer label.
End of explanation
"""


