# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# # EDA and Data Visualizations 

# _by <NAME>_

# ### Contents:
#  - [Exploring Data on CDC Website Related to Asthma](#Exploring-Data-on-CDC-Website-Related-to-Asthma)
#     - [Find the US Cities with Highest Number of Adults with Asthma in 2017](#Find-the-US-Cities-with-Highest-Number-of-Adults-with-Asthma-in-2017)
#  - [EDA of AQI and Temperature Data](#EDA-of-AQI-and-Temperature-Data)
#      - [Heatmap](#Heatmap)
#      - [Pairplot](#Pairplot)
#      - [Visualizations Exploring Temperature and Pollutants](#Visualizations-Exploring-Temperature-and-Pollutants)
#      - [Visualization Exploring Cumulative AQI Regarding Time](#-Visualization-Exploring-Cumulative-AQI-Regarding-Time)
#

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# %matplotlib inline

# ## Exploring Data on CDC Website Related to Asthma
# [(back to top)](#EDA-and-Data-Visualizations)

df_asthma = pd.read_csv('../data/500_Cities__Current_asthma_among_adults_aged___18_years.csv')

df_asthma.head(2).T

# ### Find the US Cities with Highest Number of Adults with Asthma in 2017
# [(back to top)](#EDA-and-Data-Visualizations)

top_10 = df_asthma['Data_Value'].sort_values(ascending = False)[:10]
top_10

mask = df_asthma['Data_Value'] > 17 
# finding all cities with asthma rates higher than 17

high_asthma_rate_df = df_asthma[mask].sort_values(by = 'Data_Value', ascending = True)

high_asthma_rate_df = high_asthma_rate_df[['StateAbbr','CityName','Data_Value']]

plt.figure(figsize = (8,5))
plt.title('Top 5 US City Asthma Rate in 2017',fontsize = 17)
sns.barplot(data = high_asthma_rate_df, x = 'CityName', y = 'Data_Value');
plt.xlabel('City Name', fontsize = 13)
plt.ylabel("Percent of Adult's with Asthma", fontsize = 13);

# Cleveland with the highest US asthma rate for adults older than 18 years old based on CDC data from 2017. 17.5% of all adults in Cleveland have Asthma. Seeing that Cleveland had a high asthma rate narrowed allowed us to narrow in on one city to look at air quality  

# ## EDA of AQI and Temperature Data 
# [(back to top)](#EDA-and-Data-Visualizations)

df = pd.read_csv('../data/cleaned_aqi_and_temp_data_2017-2019.csv')

df.head()

df.info()

df["weekday"] = df["weekday"].astype("category")

df["month"] = df["month"].astype("category")

df['date'] = pd.to_datetime(df['date'])

df.drop('pct_change_aqi', axis = 1, inplace = True)

df.info()

df.set_index('date', inplace = True)

df.describe().T

# Mean of cumulative AQI is 43.040256 showing that on average the levels are not in dangerous zones but are near the moderate zone. The max is 115 which is considered unhealthy for sensitive groups and 50% of the daily cumulative AQI data is between 32 and 52 showing that 50% of the cumulative daily readings are nearing the moderate zone. 

df.isna().sum()

# ### Heatmap
# [(back to top)](#EDA-and-Data-Visualizations)

plt.figure(figsize=(15,15)) 
mask = np.zeros_like(df.corr())
mask[np.triu_indices_from(mask)] = True
sns.heatmap(df.corr(), annot = True, mask = mask, square = True);

# ##### Correlation
#
# - pm 2.5 mean and pm 2.5 aqi are highly correlated with cumulative aqi
#
# - there is multicolinearity seen between aqi readings and pollutant measurement values which is to be expected since the aqi values are scored based from the pollutant measurement
#
# - there is also multicolinearity between temperature values which is to expected has well since when we have lower average temperature then the low temperature of the day and the high temperature of the day will all be related. 

# ### Pairplot
# [(back to top)](#EDA-and-Data-Visualizations)

sns.pairplot(data = df, corner = True)

# ### Visualizations Exploring Temperature and Pollutants 
# [(back to top)](#EDA-and-Data-Visualizations)

# +
fig, ax = plt.subplots(nrows=1,ncols=2,figsize=(13,7))

sns.scatterplot(x = 'pm2.5_aqi_val', y = 'cumulative_aqi' , hue = 'temp_high', data = df, ax=ax[0]).set(
    title="PM 2.5 Daily AQI Value vs Cumulative Daily AQI \n with Relation to Daily High Temperature",
    xlabel='PM 2.5 AQI Value', 
    ylabel='Cumulative Daily AQI')


sns.scatterplot(x = 'pm2.5_mean', y = 'cumulative_aqi' , hue = 'temp_high', data = df, ax=ax[1]).set(
    title="PM 2.5 Mean Daily Value vs Cumulative Daily AQI \n with Relation to Daily High Temperature", 
    xlabel='PM 2.5 Mean Value', 
    ylabel='Cumulative Daily AQI', 
    label = 'High Temp');

# -

# This graphs shows that at times there is a direct correlation with cumulative daily aqi and pm 2.5 daily aqi value showing that on these days pm 2.5 is usually the highest pollutant aqi value on that day. This shows that pm 2.5 has a strong relationship with influencing total AQI value. We can also see as temperature is getting warmer there is a slight association with increase AQI and very high temperatures.

# +
fig, ax = plt.subplots(nrows=1,ncols=2,figsize=(13,7))

sns.scatterplot(x = 'ozone_aqi_val', y = 'cumulative_aqi' , hue = 'temp_high', data = df, ax=ax[0]).set(
    title="Ozone Daily AQI Value vs Cumulative Daily AQI \n with Relation to Daily High Temperature", 
    xlabel='Ozone AQI Value', 
    ylabel='Cumulative Daily AQI')


sns.scatterplot(x = 'ozone_max', y = 'cumulative_aqi' , hue = 'temp_high', data = df, ax=ax[1]).set(
    title="Ozone Max Daily Value vs Cumulative Daily AQI \n with Relation to Daily High Temperature",
    xlabel='Ozone Max Value', 
    ylabel='Cumulative Daily AQI', 
    label = 'High Temp');

# -

# We can also see a relationship between cumulative Daily AQI and Ozone AQI readings. As Ozone readings get to very high values, they are more likely to be the highest daily AQI reading compared to other pollutants. There is also a relationship with increasing temperature and higher AQI/ Ozone AQI and values. 

# ### Visualization Exploring Cumulative AQI Regarding Time
# [(back to top)](#EDA-and-Data-Visualizations)

sns.histplot(x = 'cumulative_aqi', data = df).set(
    title="Distribution of Daily Cumulative AQI Readings",
    xlabel='Cumulative Daily AQI Readings');

# We can see that most the  daily AQI values are around 40 and the distribution is skewed to the right indicating that the are fewer dangerous level but there are still daily levels reaching past 100 

df['cumulative_aqi'].resample('m').mean().plot(figsize=(12, 4), title='2017-2019 Average Monthly Cumulative AQI in Cleveland, OH');
plt.xlabel('Date', fontsize = 10);

# From the above chart, we can see that there was a low average monthly AQI level in Nov 2017. There was another low averaye monthly AQI in Oct 2018 and Oct 2019. However, there are spikes in average monthly AQI in May 2018, Dec 2018, and July 2019.

# +
high_mask = df['temp_avg'] > 75

high_temp_df = df[high_mask]

low_mask = df['temp_avg'] < 75

low_temp_df = df[low_mask]
# -

plt.figure(figsize=(12,8))
plt.title('2017-2019 Monthly Average Cumulative AQI in Cleveland in Relation to Average Daily Temperature')
high_temp_df['cumulative_aqi'].rolling(window=30).mean().plot(label = "Temp in F Greater than 75");
low_temp_df['cumulative_aqi'].rolling(window=30).mean().plot(label = "Temp in F Lower than 75");
plt.legend(loc='best');
plt.xlabel('Date', fontsize = 13)
plt.ylabel('Monthly Average AQI Level', fontsize = 13);

# From this chart, we can see that there usually higher temperature is associated with higher average AQI readings. However between 1-2019 and 5-2019 lower average temperatures were associated with higher AQI readings. 

df_2017 = df.loc['2017']

df_2018 = df.loc['2018']

df_2019 = df.loc['2019']

df_2017[['co_aqi_val','no2_aqi_val', 
         'ozone_aqi_val', 'pm10_aqi_val', 
         'pm2.5_aqi_val', 
         'so2_aqi_val' ]].resample('m').mean().plot(figsize=(12, 4), 
            title = '2017 Monthly Mean AQI Level Per Pollutant', xlabel = 'Month');

# The above chart shows for 2017 ozone AQI levels were the highest consitently over the year compared to other pollutants

df_2018[['co_aqi_val','no2_aqi_val', 'ozone_aqi_val', 
         'pm10_aqi_val', 'pm2.5_aqi_val', 
         'so2_aqi_val' ]].resample('m').mean().plot(figsize=(12, 4), title = '2018 Monthly Mean AQI Level Per Pollutant', xlabel = 'Month');

# The above chart shows that ozone and pm 2.5 showed the highest average monthly AQI readings over 2018

df_2019[['co_aqi_val','no2_aqi_val', 'ozone_aqi_val', 'pm10_aqi_val', 'pm2.5_aqi_val', 'so2_aqi_val' ]].resample('m').mean().plot(figsize=(12, 4), title = '2019 Monthly Mean AQI Level Per Pollutant', xlabel = 'Month');

# The above chart shows that we are noticing even more prevalence of pm 2.5 reaching highest monthly average AQI levels. Ozone also reaching high average monthly AQI levels compared to other pollutants. 

df[['co_aqi_val','no2_aqi_val', 
         'ozone_aqi_val', 'pm10_aqi_val', 
         'pm2.5_aqi_val', 
         'so2_aqi_val']].resample('q').mean().plot(figsize=(13, 6), 
            title = 'Quartly Mean Cumulative AQI Level Per Pollutant', xlabel = 'Quarter of a Year', ylabel = 'Mean Cumulative AQI');

# pm 2.5 AQI and ozone aqi consistently show strong relationship to high overall AQI readings.

df['pm2.5_aqi_val'].resample('m').mean().plot(figsize=(12, 4), 
            title = '2017-2019 Monthly Mean AQI Level for PM 2.5 Pollutant', xlabel = 'Month');

# From the above graph we can see that as the years progress, PM 2.5 AQI levels are increasing



StarcoderdataJupyterScriptsDedupFiltered