import pandas as pd
import matplotlib.pyplot as plt

%matplotlib inline

comp_data = {'Company':['GOOG','GOOG','MSFT','MSFT','FB','FB'],
       'Person':['Sam','Charlie','Amy','Vanessa','Carl','Sarah'],
       'Sales':[200,120,340,124,243,350]}

comp_df = pd.DataFrame(comp_data)
comp_df

# find unique company names
comp_df['Company'].unique()

"""
Explanation: Productivity with pandas

<h1>Table of Contents<span class="tocSkip"></span></h1>
<div class="toc"><ul class="toc-item"><li><span><a href="#unique---find-unique-rows" data-toc-modified-id="unique---find-unique-rows-1">unique - find unique rows</a></span><ul class="toc-item"><li><span><a href="#nunique---find-number-of-unique-rows" data-toc-modified-id="nunique---find-number-of-unique-rows-1.1">nunique - find number of unique rows</a></span></li><li><span><a href="#value_counts---find-unique-values-and-number-of-occurrences" data-toc-modified-id="value_counts---find-unique-values-and-number-of-occurrences-1.2">value_counts - find unique values and number of occurrences</a></span></li></ul></li><li><span><a href="#apply---batch-process-column-values" data-toc-modified-id="apply---batch-process-column-values-2">apply - batch process column values</a></span></li><li><span><a href="#sort_values---sorting-rows" data-toc-modified-id="sort_values---sorting-rows-3">sort_values - sorting rows</a></span></li><li><span><a href="#isnull---finding-null-values-throughout-the-DataFrame" data-toc-modified-id="isnull---finding-null-values-throughout-the-DataFrame-4">isnull - finding null values throughout the DataFrame</a></span></li><li><span><a href="#Working-with-time-series-data" data-toc-modified-id="Working-with-time-series-data-5">Working with time series data</a></span><ul class="toc-item"><li><span><a href="#Plotting-time-series" data-toc-modified-id="Plotting-time-series-5.1">Plotting time series</a></span></li><li><span><a href="#Sort-dataframe-by-time" data-toc-modified-id="Sort-dataframe-by-time-5.2">Sort dataframe by time</a></span></li><li><span><a href="#Add-a-counter-column-to-the-dataframe" data-toc-modified-id="Add-a-counter-column-to-the-dataframe-5.3">Add a counter column to the dataframe</a></span></li></ul></li></ul></div>

unique - find unique rows
Find unique rows in dataset
End of explanation
"""


comp_df['Company'].nunique()

"""
Explanation: nunique - find number of unique rows
More efficient than finding the unique array and finding the length of it.
End of explanation
"""


comp_df['Company'].value_counts()

"""
Explanation: value_counts - find unique values and number of occurrences
End of explanation
"""


comp_df['sq_sales'] = comp_df['Sales'].apply(lambda x:x*x)
comp_df

"""
Explanation: apply - batch process column values
Calling the apply() is similar to calling the map() in Python. It can apply an operation on all records of a selected column. For instance, to find the squared sales, do the following
End of explanation
"""


def cuber(row):
    return row['Sales'] * row['sq_sales']

comp_df['cu_sales'] = comp_df.apply(cuber, axis=1) 
#note - how the function is called as an obj
# note - how I need to set axis to 1, instead of 0 which is defualt.
comp_df

"""
Explanation: We can also define a function and call that within the apply() method. This can accept values of one or more columns to calculate a new column.
End of explanation
"""


comp_df.sort_values('Sales')

"""
Explanation: sort_values - sorting rows
End of explanation
"""


#sorting along multiple columns
comp_df.sort_values(['Company','Sales'])

"""
Explanation: Note how the index remains attached to the original rows.
End of explanation
"""


comp_df.isnull()

"""
Explanation: isnull - finding null values throughout the DataFrame
End of explanation
"""


registrant_df = pd.read_csv('./registrant.csv')
registrant_df.head()

"""
Explanation: Working with time series data
This section explains how to specify datatypes of columns while reading data and how to define column converters to ease certain data types.
End of explanation
"""


registrant_df.dtypes

"""
Explanation: The Registration Date should be of type datetime and the Current customer? should be of bool. However, are they?
End of explanation
"""


# define a function (lambda in this case) which will convert a column to bool depending on the 
# value of the cell
convertor_fn = lambda x: x in ['Yes', 'yes', 'YES']
convertor_map = {'Current customer?': convertor_fn}

# re-read data
registrant_df2 = pd.read_csv('./registrant.csv', 
                             parse_dates=['Registration Date'], 
                             converters = convertor_map)

registrant_df2.dtypes

"""
Explanation: Everything is a generic object. Let us re-read, this time knowing what their data types should be.
End of explanation
"""


registrant_df2.set_index('Registration Date', inplace=True)
registrant_df2.drop(axis=1, columns=['Unnamed: 0'], inplace=True) # drop bad column
registrant_df2.head()

"""
Explanation: Plotting time series
Now that the Registration date is datetime, we can plot the number of registrants by time. But before that, we need to set it as the index.
End of explanation
"""


registrant_df2.sort_index(inplace=True)

"""
Explanation: Sort dataframe by time
End of explanation
"""


registrant_df2['registration_count'] = range(1, len(registrant_df2)+1) # goes from 1 to 284

plt.figure(figsize=(10,7))
registrant_df2['registration_count'].plot(kind='line')
plt.title('Number of registrants over time');

"""
Explanation: Add a counter column to the dataframe
Note. It is important to count up only after sorting. Else the numbers are going to be all over the place.
End of explanation
"""


