import json
import os
from os import path

import pandas as pd

DATA_FOLDER = os.getenv('DATA_FOLDER')

events = pd.read_json(path.join(DATA_FOLDER, 'workup.json'))
events.head()

"""
Explanation: WorkUp Events
Author: pascal@bayes.org
Date: 2017-08-13
In July 2017, the WorkUp team gave us a programatical access to their events. This notebook explores how we could use it to give examples of pro events to our users to motivate them to go to some of them or others.
The whole dataset is directly downloadable at https://www.workuper.com/events/index_json.json and it is also available with the command docker-compose run --rm data-analysis-prepare data/workuper.json.
Loading and General View
First let's load the json file:
End of explanation
"""


events.describe(include='all').head(3)

"""
Explanation: Cool! Before exploring each individual fields, let's see how many events there are, and whether those fields are always set:
End of explanation
"""


pd.options.display.max_colwidth = 100
events.title.to_frame()

"""
Explanation: Hum, there are not that many rows: 13 only. However all fields seem to be set, except for favorite which is never set.
Extracting Useful Info
By a quick glance to the data above, we can classify fields between useful, irrelevant, and others to explore.
The ones that seem directly useful:
* title
* address combined with latitude and longitude
* date and dateend
* organiser
And then in a lesser extent (too much details for what we want to use them):
* description
* subscription_link
* website
* time
However the following ones are irrelevant to us as they seem only useful for the WorkUp database:
* favorite
* id
* status
* created_at
* updated_at
* user_id
So it leaves 3 fields: category, price and slug that we should explore a bit more.
Obvious Fields
Let's check quickly that the obvious fields have useful values. The titles:
End of explanation
"""


events[['address', 'latitude', 'longitude']]

"""
Explanation: Perfect, we can use it directly as a title to show to our users. Note that the title frequently involves the city, and sometimes the timing. Also we can see that the use of upper case letters is less than ideal.
The addresses:
End of explanation
"""


events[['date', 'dateend']]

"""
Explanation: Quickly comparing two addresses in Bordeaux, we can see that the latitude, longitude is probably the exact one (pretty cool). For our application we could filter events that are not so far from the user's target city. The address is not always formatted the same way so we will use mainly the lat/lng.
The dates:
End of explanation
"""


events.organiser.to_frame()

"""
Explanation: The dates are all in the future which probably indicates that WorkUp already filters the past ones. Note that the dateend is almost always the same as date which probably indicates that those events only last one day. For our purpose we will ignore dateend for now.
The organisers:
End of explanation
"""


events[['title', 'price']]

"""
Explanation: As for the title, this is pretty clean and useful.
Richer Fields
The description, exact time, the website or a direct subscription_link are not required for what we want. Our goal is not to speed up the process of our users subscribing to a specific event but to have them realize that there are many of them and that they should dig a bit more this way of enlarging their network or improving their job search.
However it would be useful to get all those details from a secondary page if they wanted to. From the WorkUp website, there are pages with full details that are accessible with a URL like this one: https://www.workuper.com/events/salon-des-10-000-emplois-marseille. The good thing is that the dataset contains the last part of the URL in the slug field. So we will keep this field to rebuild the full page URLs.
Others
Let's check the price field: we could decide to hide events that are not free, or at least warn our users early.
End of explanation
"""


events.category.iloc[0]

"""
Explanation: Cool! Most of them are free. However some of them are not, and one of them is really expensive. Here we would need a product decision on whether we show them and how.
Finally, let's check the category field:
End of explanation
"""


events['categories'] = events.category.apply(lambda l: json.loads(l))
events.categories.iloc[0]

"""
Explanation: Woops, this seems like a JSON encoded string (which means there was a double JSON encoding as we already decoded once to create the dataset). Let's decode it:
End of explanation
"""


all_categories = set(c for categories in events.categories.tolist() for c in categories)
all_categories

"""
Explanation: Now let's list all the available categories:
End of explanation
"""


