# first, make sure we have the right modules installed
!pip install --upgrade chart-studio plotly

%matplotlib inline 

import matplotlib
import matplotlib.pyplot as plt
import chart_studio as plotly
import chart_studio.plotly as py
import plotly.express as px
import plotly.graph_objects as go
import pandas as pd
import folium
import warnings
from IPython.display import display
#matplotlib.rcParams['figure.figsize'] = (20.0, 10.0) # larger figure size
warnings.filterwarnings('ignore')

"""
Explanation: In-Class Coding Lab: Data Visualization
The goals of this lab are to help you understand:

The value of visualization: A picture is worth 1,000 words!
The various ways to visualize information
The basic requirements for any visualization
How to plot complex visualizations such as multi-series charts and maps
Visualization Tools:
Matplolib
Plot.ly
Folium Maps
End of explanation
"""


goers = pd.read_csv('https://raw.githubusercontent.com/mafudge/datasets/master/ist256/13-visualization/moviegoers.csv')
goers['age_group'] = ''
goers['age_group'][goers['age'] <=18] = 'Youth'
goers['age_group'][(goers['age'] >=19) & (goers['age'] <=55)] = 'Adult'
goers['age_group'][goers['age'] >=56] = 'Senior'

goers.sample(5)

"""
Explanation: Back to the movie goers data set
For this lab, we will once again use the movie goers dataset. As you may recall this data set is a survey demographic survey of people who go to the movies. Let's reload the data and setup our age_group feature again.
End of explanation
"""


gender = goers['gender'].value_counts()
gender

"""
Explanation: Visualizing Data
There are many ways your can visualize information. Which one is the most appropriate? It depends on the data, of course. 

Counting Categorial data belongs in charts like pie charts and bar charts.
Counting Numerical data is best suited for histograms.
Timeseries data and continuous data belongs in line charts.
A comparision of two continuous values is best suited for a scatter plot. 
Geographical data is best displauyed on maps.

Let's use this knowledge to plot some data in the goers DataFrame!
Males or Females?
The first thing we might want to visualize is a count of gender in the dataset. A pie chart is well suited for this task as it displays data as a portion of a whole. To create a pie chart we need the data to count and the labels for the counts.
Let's try it.
First we get the value counts as a series gender:
End of explanation
"""


gender_df = pd.DataFrame( { 'Gender' : gender.index,  "Counts" : gender })
display(gender_df)

"""
Explanation: Then we make it into a dataframe:
End of explanation
"""


display(gender_df)
display(gender_df.plot.pie( y = 'Counts')) # y are the values we are plotting

"""
Explanation: Then we plot! The index has the labels, and the value at the index is what we want to plot:
End of explanation
"""


#todo write code here


"""
Explanation: 1.1 You Code
Create a data frame and pie chart based on age_group first create a series of the value_counts() second, create the DataFrame with two columns AgeGroup and Counts then plot with .plot.pie(). 
Follow the steps we did in the previous three cells, but combine into one cell! Show both the dataframe and the pie.
End of explanation
"""


occ = goers['occupation'].value_counts()
occ_df = pd.DataFrame( { 'occupation' : occ.index,  "counts" : occ })
occ_df.plot.pie(y = 'counts')


"""
Explanation: Too many pieces of the pie?
Pie charts are nice, but they are only useful when you have a small number of labels. More that 5-7 labels and the pie becomes messy. For example take a look at this pie chart of occupation:
End of explanation
"""


occ_df.plot.bar()

"""
Explanation: That's crazy... and difficult to comprehend. Also pie charts visualize data as part of the whole. We have no idea how many students there are. Sometimes we want to know actual counts. This is where the bar chart comes in handy!
Raising the bar!
Let's reproduce the same plot as a bar:
End of explanation
"""


# todo write code here


"""
Explanation: Ahh. that's much better. So much easier to understand! Students make up the majority of occupations.
1.2 You Code
Write a one-liner to plot your age group dataframe as a Bar!
End of explanation
"""


ages = goers['age'].value_counts()
ages_df = pd.DataFrame( { 'age' : ages.index,  "counts" : ages })
ages_df.plot.bar(y = 'counts')

"""
Explanation: When bar charts fail...
Bar charts have the same problem as pie charts. Too many categories overcomplicate the chart, or show the data in a meaningless way. For example, let's create a bart chart for ages:
End of explanation
"""


goers.hist(column ='age', color='lightblue', edgecolor='blue')

"""
Explanation: Meaningless. For two key reasons:

too many categories
age is a continuous variable not a categorical variable. In plain English, this means there's a relationship between one age and the next.  20 < 21 < 22. This relationship is not represented in a bar chart, which only displays categories of data.

...Call in the Histogram!
What we want is a historgram, which takes a continuous variable and loads counts into "buckets".  Notice how we didn't have to lump data with value_counts(). Histograms can do that automatically because the age variable is continuous. Let's try it:
End of explanation
"""


goers.hist(column ='age', bins=15, color='pink', edgecolor='red')

"""
Explanation: The default histogram has 10 bins. You can tweak the number of bins in your plot with a named argument.  For example, here's 15 bins.
End of explanation
"""


# todo write code here


"""
Explanation: 1.3 You Code
Write a one-liner to make a histogram ofgoers ages with 7 bins bar color cyan and edge color black
End of explanation
"""


# todo: setup the credentials replace ??? and ??? with your Plot.ly username and api_key

plotly.tools.set_credentials_file(username='???', api_key='???')


"""
Explanation: Plot.ly
Plot.ly is data visualization as a service. You give it data, it gives you back a web-based plot. Plot.ly is free and works with a variety of environments and programming languages, including Python. 
For Python is has bindings so that you can use it just like matplotlib! No need to manually invoke the web service call.
To get started with plot.ly you must sign up for an account and get a set of credentials:

Visit https://plot.ly/settings/api
Create an account or sign-in with Google or GitHub
Generate your API key and paste your username and key in the code below:
End of explanation
"""


px.pie(gender_df,  labels="Gender", names='Gender', values = "Counts")

"""
Explanation: Plotly Express... easy as pie!
Using plot.ly is as easy as, or sometimes easier than matplotlib. The Plotly expresss module (imported here as px) allows for easy plotting of data frames. Here's an example of plotting the same pie chart from above:
End of explanation
"""


px.bar(gender_df,  labels="Gender", x='Gender', y = "Counts")

"""
Explanation: Notice that plot.ly is a bit more interactive. You can hover over the part of the pie chart and see counts!
Ploty Express... raising the bar!
Here's the same information in a bar chart using plotly express.
End of explanation
"""


# Run ir(px) to see the different plot types, then use help() to bring up help for a plot type.
dir(px)

"""
Explanation: Many chart types to choose from.
If you dir(px) you can see all the different chart types supported by Plotly Express. To learn how to plot one, use help(px.bar) for example to learn how to plot a bar chart. You can also consult https://plot.ly/python/ for more information.
End of explanation
"""


# todo: write code here


"""
Explanation: 1.4 You Code
Use Plotly Express's px to create a bar chart on the occ_df Data Frame. One line of code is all that is required!
End of explanation
"""


series = go.Bar(x=gender_df['Gender'], y=gender_df['Counts']) 

fig = go.Figure()
fig.add_trace(series)
fig.update_layout(title="Count of Moive Goers By Gender", xaxis_title = 'Gender', yaxis_title='Number of People')
fig.show()


"""
Explanation: PoP (Plain old Plotly)
Plotly Express is great for Pandas dataframes, but Plotly can plot anything. Basically the setup is :

make a figure object
add traces ( series) to the plot
set the labels,if you want
show the figure.  

Here's the same gender plot, Plotly style:
End of explanation
"""


projections = [800, 400]

fig2 = go.Figure()
fig2.add_trace(go.Bar(x=gender_df['Gender'], y=gender_df['Counts'], name="Actual")) # Bar Plot of actual
fig2.add_trace(go.Bar(x=gender_df['Gender'], y=projections, name="Projected"))        # Bar plot of Projected
fig2.update_layout(title="Count of Moive Goers By Gender", xaxis_title = 'Gender', yaxis_title='Number of People')
fig2.show()

    

"""
Explanation: Need more than one series?
When you use Plain old Plotly, its simple to add multiple series to the plot. All you do is add additional Graph objects using the add_trace() method  Let's make up some extra data and add it to the plot. Note how we add the name= argument to label each series. Also each series does not need to be the same plot type. You can mix Bar with Line etc...
End of explanation
"""


import random
occ_df['last_year'] =occ_df.apply(lambda r: random.randint(-5,5) + r['counts'], axis =1)
display(occ_df.head(3))
# TODO: Make your 2 series plotly plot here


"""
Explanation: 1.5 You Code
The following code adds a column 'last_year' to the occ_df. 
Create a two-series plotly plot of movie goers counts. Label counts as This year and last_year as Last Year, of course.
End of explanation
"""


CENTER_US = (39.8333333,-98.585522)
london = (51.5074, -0.1278)
map = folium.Map(location=CENTER_US, zoom_start=4)
display(map)

"""
Explanation: Folium with Leaflet.js
Folium is a Python module wrapper for Leaflet.js, which uses Open Street Maps. These are two, popular open source mapping libraries. Unlike Google maps API, its 100% free!
You can use Folium to render maps in Python and put data on the maps. Here's how easy it is to bring up a map:
End of explanation
"""


zipcodes = pd.read_csv('https://raw.githubusercontent.com/mafudge/datasets/master/zipcodes/free-zipcode-database-Primary.csv', dtype = {'Zipcode' :object})
data = goers.merge(zipcodes,  how ='inner', left_on='zip_code', right_on='Zipcode')
students = data[ data['occupation'] == 'student']
students.sample(3)

"""
Explanation: You can zoom right down to the street level and get a amazing detail. There are different maps you can use, as was covered in this week's reading. 
Mapping the student movie goers.
Let's take the largest category of movie goers and map their whereabouts. We will first need to import a data set to give us a lat/lng for the zip_code we have in the dataframe. We could look this up with Google's geolookup API, but that's too slow as we will be making 100's of requests. It's better to have them stored already and merge them with goers!
Let's import the zipcode database into a Pandas DataFrame, then merge it with the goers DataFrame:
End of explanation
"""


for row in students.to_records():
    pos = (row['Lat'],row['Long'])
    message = f"{row['age']} year old {row['gender']}  {row['occupation']} from {row['City']},{row['State']}"
    marker = folium.Marker(location=pos, 
                    popup=message
                          )
    map.add_child(marker)
display(map)

"""
Explanation: Let's explain the code, as a Pandas refresher course:

in the first line I added dtype = {'Zipcode' :object} to force the Zipcode column to be of type object without that, it imports as type int and cannot match with the goers DataFrame.
the next line merges the two dataframes together where the zip_code in goers (on_left) matches Zipcode in zipcodes (on_right)
the result data is a combined DataFrame, which we then filter to only student occupations, sorting that in the students DataFrame

Slapping those students on a map!
We're ready to place the students on a map. It's easy:

For each row in the students dataframe:
get the coordinates (lat /lng )
make a marker with the coordinates
add the marker to the map with add_children()

Here we go!
End of explanation
"""


## todo write code here!
from ipywidgets import interact_manual

occupations_list = list(data['occupation'].dropna().unique())
occupations_list.sort()

@interact_manual(occupation=occupations_list)
def main(occupation):
    CENTER_US = (39.8333333,-98.585522)
    map = folium.Map(location=CENTER_US, zoom_start=4)
    filter_df = data[ data['occupation'] == occupation ]
    for row in filter_df.to_records():
        pos = (row['Lat'],row['Long'])
        message = f"{row['age']} year old {row['gender']}  {row['occupation']} from {row['City']},{row['State']}"
        marker = folium.Marker(location=pos, 
                        popup=message
                              )
        map.add_child(marker)
    display(map)
    

"""
Explanation: Interactive Maps
Let's combine the map with an Interact drop down. That way we select an occupation first then display the pins for only that occupation.
Here is an explanation of the code:
lines 4-5  : create a unique list of occupations from the data frame
lines 9-10 : create the map (must be re-created with each new selection
lines 11   : apply pandas filter to the data frame based selection
lines 12-18: loop over each row in the data frame, create a marker and add it to the map
lines 19   : display the map
End of explanation
"""


# run this code to turn in your work!
from coursetools.submission import Submission
Submission().submit()

"""
Explanation: Metacognition
Rate your comfort level with this week's material so far.
1 ==> I don't understand this at all yet and need extra help. If you choose this please try to articulate that which you do not understand to the best of your ability in the questions and comments section below.
2 ==> I can do this with help or guidance from other people or resources. If you choose this level, please indicate HOW this person helped you in the questions and comments section below. 
3 ==> I can do this on my own without any help. 
4 ==> I can do this on my own and can explain/teach how to do it to others.
--== Double-Click Here then Enter a Number 1 through 4 Below This Line ==--  
Questions And Comments
Record any questions or comments you have about this lab that you would like to discuss in your recitation. It is expected you will have questions if you did not complete the code sections correctly.  Learning how to articulate what you do not understand is an important skill of critical thinking. Write them down here so that you remember to ask them in your recitation. We expect you will take responsilbity for your learning and ask questions in class.
--== Double-click Here then Enter Your Questions Below this Line ==--
End of explanation
"""


