"""
# US Accident exploratory data analysis
"""
"""
## Information
This is a countrywide car accident dataset, which covers 49 states of the USA. The accident data are collected from February 2016 to December 2020.

Currently, there are about 1.5 million accident records in this dataset.

### Acknowledgement
Moosavi, Sobhan, Mohammad Hossein Samavatian, Srinivasan Parthasarathy, and Rajiv Ramnath. “A Countrywide Traffic Accident Dataset.”, 2019.

Moosavi, Sobhan, Mohammad Hossein Samavatian, Srinivasan Parthasarathy, Radu Teodorescu, and Rajiv Ramnath. "Accident Risk Prediction based on Heterogeneous Sparse Data: New Dataset and Insights." In proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, 2019.
"""
"""
### Dataset abbrevations:    

- ID - This is a unique identifier of the accident record.

- Severity - Shows the severity of the accident, a number between 1 and 4, where 1 indicates the least impact on traffic (i.e., short delay as a result of the accident) and 4 indicates a significant impact on traffic (i.e., long delay).

- Start_Time - Shows start time of the accident in local time zone.

- End_Time - Shows end time of the accident in local time zone. End time here refers to when the impact of accident on traffic flow

- Start_Lat - Shows latitude in GPS coordinate of the start point.

- Start_Lng - Shows longitude in GPS coordinate of the start point.

- End_Lat - Shows latitude in GPS coordinate of the end point.

- End_Lng - Shows longitude in GPS coordinate of the end point.

- Distance(mi) - The length of the road extent affected by the accident.

- Description - Shows natural language description of the accident.

- Number - Shows the street number in address record.

- Street - Shows the street name in address record.

- Side - Shows the relative side of the street (Right/Left) in address record.

- City - Shows the city in address record.

- County - Shows the county in address record.

- State - Shows the state in address record.

- Zipcode - Shows the zipcode in address record.

- Country - Shows the country in address record.

- Timezone - Shows timezone based on the location of the accident (eastern, central, etc.).

- Airport_Code - Denotes an airport-based weather station which is the closest one to location of the accident.

- Weather_Timestamp - Shows the time-stamp of weather observation record (in local time).

- Temperature(F) - Shows the temperature (in Fahrenheit).

- Wind_Chill(F) - Shows the wind chill (in Fahrenheit).

- Humidity(%) - Shows the humidity (in percentage).

- Pressure(in) - Shows the air pressure (in inches).

- Visibility(mi) - Shows visibility (in miles).

- Wind_Direction - Shows wind direction.

- Wind_Speed(mph) - Shows wind speed (in miles per hour).

- Precipitation(in) - Shows precipitation amount in inches, if there is any.

- Weather_Condition - Shows the weather condition (rain, snow, thunderstorm, fog, etc.)

- Amenity - A POI annotation which indicates presence of amenity in a nearby location.

- Bump - A POI annotation which indicates presence of speed bump or hump in a nearby location.

- Crossing - A POI annotation which indicates presence of crossing in a nearby location.

- Give_Way - A POI annotation which indicates presence of give_way in a nearby location.

- Junction - A POI annotation which indicates presence of junction in a nearby location.

- No_Exit - A POI annotation which indicates presence of junction in a nearby location.

- Railway - A POI annotation which indicates presence of railway in a nearby location.

- Roundabout - A POI annotation which indicates presence of roundabout in a nearby location.

- Station - A POI annotation which indicates presence of station in a nearby location.

- Stop - A POI annotation which indicates presence of stop in a nearby location.

- Traffic_Calming - A POI annotation which indicates presence of traffic_calming in a nearby location.

- Traffic_Signal - A POI annotation which indicates presence of traffic_signal in a nearby location.

- Turning_Loop - A POI annotation which indicates presence of turning_loop in a nearby location.

- Sunrise_Sunset - Shows the period of day (i.e. day or night) based on sunrise/sunset.

- Civil_Twilight - Shows the period of day (i.e. day or night) based on civil twilight.

- Nautical_Twilight - Shows the period of day (i.e. day or night) based on nautical twilight.

- Astronomical_Twilight - Shows the period of day (i.e. day or night) based on astronomical twilight.
"""
"""
### Questions:
1. Which city in US has reported the most number of accident cases?
2. Which 5 states reported the highest number of accident cases?
3. Which timezone reported the most number of accident cases?
4. Which street is most accident prone in US?
5. What time of the day are accidents most frequent in?
6. Which days of the week have the most accidents?
7. Which months have the most accidents?
8. What is the trend of accidents year over year (decreasing/increasing?)
9. Effects of road condition on the accident cases.
10. How did the weather conditions affected the cases?

"""
"""
The abbrevations for state can be checked from here: https://about.usps.com/who-we-are/postal-history/state-abbreviations.htm
"""
"""
## Data Preparation and Cleaning

1. Load the file using Pandas
2. Look at some information about the data & the columns
3. Fix any missing or incorrect values
"""
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
data_filename = "../input/us-accidents/US_Accidents_Dec20_updated.csv"
data_filename2 = "../input/us-2021-census-cities-populations-coordinates/us2021census.csv"
#1.Loading the file using Pandas
df = pd.read_csv(data_filename)
df.head(3)
print('The Dataset contains: Rows - {} and Columns - {}'.format(df.shape[0],df.shape[1]))
dff = pd.read_csv(data_filename2)
dff.head()
dff = dff.drop(["Latitude","Longitude","Type","Counties"],axis=1)
dff.head()
"""
The abbrevations for states can be checked from here: https://about.usps.com/who-we-are/postal-history/state-abbreviations.htm
"""
#2.Looking at some overall information and statistics about the data.
df.info()
"""
This shows the name of all the columns, it show which columns have null values. The data types used are: int, float, bool and object.
"""
df.describe()
"""
Insights-
- This tells us that majority of the accidents (> 75%) had a severity rating of ~2 i.e. the traffic was not highly impacted.
- The average length of the road extent affected by the accidents was ~0.6 miles. So, there was almost no traffic disturbance.
- Accidents took place despite having average visibility of ~9 miles. This could mean that low visibility was not a reason.
- For about 75% of the accidents, no precipitation was reported. This could mean that rain was not a problem.

"""
# 3. Fixing any missing or incorrect values.
df.isna().mean().sort_values(ascending=False)
a = (df.isna().sum().sort_values(ascending=False)/len(df))*100
plt.figure(figsize=(9,4))
plt.title("Percentage of null values in the dataset",size=17,color="grey")
plt.xlabel('\n Percentage (%) \n', fontsize=15, color='grey')
plt.ylabel('\n Columns \n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a[a!=0].plot(kind="barh",color="pink")
plt.show()
"""
- We can see that the Number column has almost 70% data missing, Precipitation(in) and Wind_Chill(F) also have >=30% data missing.
Since, Number column (which shows the street number) has majority of the data missing, we should drop it.
"""
df = df.drop("Number",axis=1)
"""
## Exploratory analysis and visualisation
"""
"""
### 1. Which city in US has reported the most number of accident cases?

"""
#As there are 10657 unique cities, we cant plot data for all of them.
#So, I only plot data of cities which had reported more than 10k accident cases.
a = df.City.value_counts()
plt.figure(figsize=(10,6))
plt.title("US cities where accident cases recorded are more than 10,000",size=17,color="grey",y=1.04)
#t = a[a>10000].plot(kind="pie",shadow=True,explode=(0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1),startangle=130,autopct="%1.1f%%",label="")
lab= a[a>10000].index
plt.pie(a[a>10000],shadow=True,explode=(0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1),startangle=90,labels=lab)
plt.show()
largest_acc_city = a[0]
largest_acc_city/5 #Divided by 5 because of 2016-2020
#Lets look at top 10 most populated cities:
dff.groupby("City")["Population"].sum().sort_values(ascending=False).head(10)
print(dff[dff.City=="Miami"].index.values)
for i in [43,3008,9872,14548,19114]:
  print(dff.iloc[i])
#The census dataset shows that more than 1 state has its city named Miami.
print(dff[dff.City=="Charlotte"].index.values)
print(dff[dff.City=="Houston"].index.values)
print(dff[dff.City=="Dallas"].index.values)
"""
- We see that Los Angeles had the largest number of accidents (it is also the 2nd most populated city of US), Miami is the city with 2nd highest no. of accident cases reported. 
- On average, 7996 accidents very reported in Los Angeles yearly from 2016-2020.
- As a surprise, Chicago being the 3rd most populated city reported quite less number of accidents. Source of the data could be an issue here.
- Miami city reported 2nd largest number of accidents. Possibly because more than 1 state has city named Miami (total 11 from wikipedia), the State census dataset also shows Miami city in different states,similarly for Charlotte,Dallas,Houstan etc.
- One anomaly is that New York didn't report accidents larger than 10,000 from 2016-2020 despite being the most populated city of US (possible reason is less data from this city).
"""
# Lets try to see the distribution of accidents in cities in log scale.
plt.figure(figsize=(9,4))
plt.xlabel('\n City \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
sns.histplot(df.City.value_counts(),log_scale=True,color="grey")
plt.show()
"""
No of accidents in cities are exponentially decreasing. We see large accounts near to 1, possibly because of not data recorded there.
"""
"""
We have 1167 cities which recorded only 1 accident case, this is another data source drawback.
"""
"""
### 2. Which 5 state had the most number of accidents?
"""
b = df.State.value_counts()
#b[b>50000].plot(kind="pie",radius=2.0,shadow=True,explode=(0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1),startangle=90,autopct="%1.1f%%",label="")
plt.figure(figsize=(10,6))
plt.title("US states where accident cases recorded are more than 50,000",size=17,y=1.04,color="grey")
lab= b[b>50000].index
plt.pie(b[b>50000],shadow=True,explode=(0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1),startangle=280,labels=lab)
plt.show()
# Lets look at the population of top 10 states.
dff.groupby("State")["Population"].sum().sort_values(ascending=False).head(10)
"""
- We see that California (CA) had the most number of accidents,followed by Florida, Oregon, Texas and New york. It appears to be right as California also has the largest population.
- In past 5 years, on average 246 accidents (daily) happened in California, implies approximately 10 accidents per hour.
"""
"""
### 3. Which timezone reported the most number of accident cases?
"""
timezone = pd.DataFrame(df.Timezone.value_counts()).reset_index().rename(columns={"index":"Timezone","Timezone":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases for different timezones in US (2016-2020)\n', size=20, color='grey')
plt.xlabel('\n Timezone \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.barplot(x=timezone.Timezone , y=timezone.Cases,palette="rainbow")
import matplotlib.ticker as ticker
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- These are the states that comes under the US/Eastern time zone: https://en.wikipedia.org/wiki/Eastern_Time_Zone
    - Connecticut,Delaware,Georgia,Maine,Maryland,Massachusetts,New Hampshire,New Jersey,New York,North Carolina,Ohio,Pennsylvania,Rhode Island,South Carolina,Vermont,Virginia,West Virginia.

- Two states are fully contained in the Pacific Time Zone: https://en.wikipedia.org/wiki/Pacific_Time_Zone
    - California,Washington.

- These are the states that comes under the Central Time Zone:
    - Alabama, Arkansas, Illinois, Iowa, Louisiana, Minnesota, Missouri, Mississippi, Oklahoma, and Wisconsin.(https://simple.wikipedia.org/wiki/Central_Time_Zone)

- Six states are fully contained in the Mountain Time Zone: https://en.wikipedia.org/wiki/Mountain_Time_Zone
    - Colorado,Montana,New Mexico,Utah,Wyoming,Arizona.

14 states are not included because they are split between different time zones.
"""
a1 = dff[(dff.State=="CT") | (dff.State=="CA")| (dff.State=="GA") | (dff.State=="ME") | (dff.State=="MD") | (dff.State=="MA") | (dff.State=="NH") | (dff.State=="NJ") | (dff.State=="NY") | (dff.State=="NC") | (dff.State=="OH") | (dff.State=="PA") | (dff.State=="RI") | (dff.State=="SC") | (dff.State=="WT") | (dff.State=="VA") | (dff.State=="VT")]["Population"].sum()
a2= dff[(dff.State=="CA") | (dff.State=="WA")]["Population"].sum()
a3 = dff[(dff.State=="AL") | (dff.State=="AR")| (dff.State=="IL") | (dff.State=="IA") | (dff.State=="LA") | (dff.State=="MN") | (dff.State=="MS") | (dff.State=="MO") | (dff.State=="OK") | (dff.State=="WI")]["Population"].sum()
a4 = dff[(dff.State=="CO") | (dff.State=="MT") | (dff.State=="NM") | (dff.State=="UT") | (dff.State=="AZ") | (dff.State=="WY")]["Population"].sum()
a = ["Eastern","Pacific","Central","Mountain"]
b=[a1,a2,a3,a4]
tot = a1+a2+a3+a4
c = [a1/(tot)*100,a2/(tot)*100,a3/(tot)*100,a4/(tot)*100]
for (i,j,k) in zip(a,b,c):
  print("Population in {} timezone region: {} ({:.2f}%)".format(i,j,k))
for (i,j) in zip(timezone.Timezone,range(0,4)):
    print("Percentage of accident cases reported in {} is {:.2f}%".format(i,(df.Timezone.value_counts()[j]/(df.Timezone.value_counts()[0] + df.Timezone.value_counts()[1] + df.Timezone.value_counts()[2] + df.Timezone.value_counts()[3]))*100))
"""
- US/Eastern timezone region reported the most number of accident cases (Census data also shows that the population in this timezone is also the largest), followed by US/Pacific, US/Central and US/Mountain.
- US/Eastern timezone region accounts for about 38.84% of the accident cases.
- Eastern and Pacific timezone regions accounts for about 76% of the accident cases. Interestingly, these two regions also accounts for about 72% of the population (excluding those 14 split timezone states)
- Us/Mountain region reported the lowest number of accident cases. It agrees with the fact that Mountain timezone region has the least population, thus lowest no of accidents are reported there.
"""
"""
###4 Which street is most accident prone in US?
"""
street = pd.DataFrame(df.Street.value_counts().reset_index().rename(columns={'index':'Street No.', 'Street':'Cases'}))
top_street = pd.DataFrame(street.head(10))
plt.figure(figsize=(9,4))
plt.title('\nTop 10 Accident Prone Streets in US (2016-2020)\n', size=20, color='grey')
plt.xlabel('\nStreet No.\n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.barplot(x=top_street["Street No."],y=top_street.Cases,palette="rainbow")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
len(df.Street)
#top_street["Cases"][0] + top_street["Cases"][1] + 
for (i,j) in zip(street["Street No."],range(0,10)):
    print("Percentage of accident cases reported on street: {} is {:.2f}%".format(i,(street.Cases[j]/len(street.Cases)*100)))
"""
- 26,645 (28.64%) accidents occured on Street I-5 N in the last 5 years (2016-2020) in US.
- On average 14 accidents were reported on street I-5 N.
"""
"""
### 5. What time of the day accidents are more frequent in?
"""
# Lets first parse the Start_Time column as a datetime column.
df.Start_Time = pd.to_datetime(df.Start_Time)
hour = pd.DataFrame(df.Start_Time.dt.hour.value_counts()).reset_index().rename(columns={"index":"Hour","Start_Time":"Accident_Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases for different hours of a day\n', size=20, color='grey')
plt.xlabel('\n Hours \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.barplot(x=hour.Hour,y=hour.Accident_Cases,palette="pastel")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
for (i,j) in zip(hour.Hour,range(0,24)):
  print("Percentage of accidents occuring at {} hour is {:.2f}%".format(i,(hour.Accident_Cases[j]/hour.Accident_Cases.sum())*100))
"""
- Highest number of accidents occur between 3:00PM - 6:00PM, and between 6:00AM - 9:00AM (Possibly because of work hours). 
- In evening, around 27% of the road accidents occurred in between 3:00PM to 6:00PM.
- Around 18% of the road accidents occurred in between 6:00AM to 9:00AM.
- The most-deadliest accident hour is 5:00PM implies the Evening Office-Returning Hours.
- The 2nd most-deadliest accident hour is 8:00AM implies the Morning Office-Going Hours.
- We see high number of accidents in the evening time frame because people are more tired at that time as compared to morning hours and thus have low concentration. 
- High no of accidents occur in the morning possibly because people are in a hurry to reach their destination.
"""
"""
### 6. Which day of the week accidents are more frequent in?

---


"""
day = pd.DataFrame(df.Start_Time.dt.day_name().value_counts()).reset_index().rename(columns={'index':'Day', 'Start_Time':'Cases'})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases for different days of a week\n', size=20, color='grey')
plt.xlabel('\n Day \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.barplot(x=day.Day,y=day.Cases,palette="muted")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
for (i,j) in zip(day.Day,range(0,7)):
  print("Percentage of accidents occuring at {} hour is {:.2f}%".format(i,(day.Cases[j]/day.Cases.sum())*100))
"""
- Thrusday reported the highest no of cases.
- It looks like overall it is evenly distributed on the business days.
- Only around 17% of the cases were reported on weekends.
- Working Days of the week have almost 2 times higher accident percentage, compared with the Weekend Days which is expected.
- As a surprise, there are less accidents occuring on the weekends, probably because of low vehichles on road? Lets verify it.
"""
"""
Lets see how the accidents are distributed over different hour of the day on the weekends.
"""
# Lets look at monday first.
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different hours on Monday\n', size=20, color='grey')
plt.xlabel('\n hours \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
monday = df.Start_Time[df.Start_Time.dt.dayofweek==0]
a = sns.histplot(monday.dt.hour,bins=24,color="grey")
a.xaxis.set_ticks(range(0,24))
# More accidents take place in the morning and in the evening after people are going to/coming from their job.
plt.show()
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different hours on Sunday\n', size=20, color='grey')
plt.xlabel('\n hours \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
sunday = df.Start_Time[df.Start_Time.dt.dayofweek==6]
sns.histplot(sunday.dt.hour,bins=24,color="grey")
plt.show()
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different hours on Saturday\n', size=20, color='grey')
plt.xlabel('\n hours \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
saturday = df.Start_Time[df.Start_Time.dt.dayofweek==5]
sns.histplot(saturday.dt.hour,bins=24,color="grey")
plt.show()
"""
- On the weekends, the distribution of accidents taking place over the day looks very different from that of on weekdays. It is more distributed through out the day, instead of peaking at a specific hour range.
"""
"""
### 7. Which month accidents are more frequent in?

"""
import calendar
month = pd.DataFrame(df.Start_Time.dt.month.value_counts()).reset_index().rename(columns={"index":"Month","Start_Time":"Accident_Cases"}).sort_values('Month')
month_names = list(calendar.month_name)[1:]
month.Month = month_names
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different months in US (2016-2020)\n', size=20, color='grey')
plt.ylabel('\n Months \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)

a = sns.barplot(x=month.Accident_Cases,y=month.Month,palette="plasma")
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
for (i,j) in zip(month.Month,range(0,12)):
  print("Percentage of accidents occuring in month {} is {:.2f}%".format(month.Month[j],(month.Accident_Cases[j]/month.Accident_Cases.sum())*100))
"""
- Around 18% of the cases are reported in December.
-  July is the month with least (3.54%) no. of road accidents in US.
- 45% of the road accidents occurred only within the 3 months, October to December
- We see towards the end of the year the number of accidents is quite high.
- Also, the dataset is from Feb 2016-Dec2020, we are missing January data for 1 year but that should not make much of a difference.

Lets try to figure out why there are more accidents towards the end of the year?
Lets try to look at it for different years.
"""
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different months in US (2016)\n', size=20, color='grey')
plt.xlabel('\n Months \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
year = df.Start_Time[df.Start_Time.dt.year==2016]
a = sns.histplot(year.dt.month,bins=12,color="yellow")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different months in US (2020)\n', size=20, color='grey')
plt.xlabel('\n Months \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
year = df.Start_Time[df.Start_Time.dt.year==2020]
a = sns.histplot(year.dt.month,bins=12,color="green")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()

plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different months in US (2019)\n', size=20, color='grey')
plt.xlabel('\n Months \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
year = df.Start_Time[df.Start_Time.dt.year==2019]
a = sns.histplot(year.dt.month,bins=12,color="red")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- We can see that for year 2016,2019,2020 less data was recorded for first 6 months. This could explain why we observed more accident cases reports in the last quarter of the year.
- The dataset is a compilation of records from different sources, it means that the source has missing data.
"""
"""
### 8. What is the trend of accidents year over year?
"""
year = pd.DataFrame(df.Start_Time.dt.year.value_counts()).reset_index().rename(columns={"index":"Year","Start_Time":"Accident_Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported yearly in US (2016-2020)\n', size=20, color='grey')
plt.xlabel('\n Year \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.barplot(x=year.Year, y = year.Accident_Cases,palette="rainbow")
a.yaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported yearly in US (2016-2020)\n', size=20, color='grey')
plt.xlabel('\n Year \n', fontsize=15, color='grey')
plt.ylabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.lineplot(data = year, marker='o', x='Year', y='Accident_Cases', color = '#734dff', label="Yearly Road Accidents")
a.xaxis.set_ticks(year.Year)
a.yaxis.set_major_formatter(ticker.EngFormatter())

plt.show()
for (i,j) in zip(year.Year,range(0,5)):
  print("Percentage of accidents occuring in year {} is {:.2f}%".format(i,(year.Accident_Cases[j]/year.Accident_Cases.sum())*100))
for (i,j) in zip(year.Year,range(0,5)):
  print("Average no. of accidents occuring per month in year {} is {}.".format(i,round(year.Accident_Cases[j]/12)))
for (i,j) in zip(year.Year,range(0,5)):
  print("Average no. of accidents occuring per day in year {} is {}.".format(i,round(year.Accident_Cases[j]/365)))
for (i,j) in zip(year.Year,range(0,5)):
  print("Average no. of accidents occuring per hour in year {} is {:.2f}.".format(i,(year.Accident_Cases[j]/365)/24))
"""
- We see that the trend is almost exponentially increasing year by year with a sharp rise in year 2020.
- Year 2020 reported half of the cases in the whole data set.
- 65661 accidents reported per month in year 2020.
- On Average, 2159 accidents were reported daily i.e 90 cases per hour, in year 2020 alone.
- Least no. of accident cases were reported in year 2016.
- From 2019 to 2020 the average accident/day has increased 3 times in US.
- Compared to year 2016, average no of accidents per hour has increased by 6 times.

"""
"""
### 9. Effect of road conditions on the accident cases.

"""
fig, ((ax1, ax2), (ax3, ax4), (ax5, ax6), (ax7, ax8)) = plt.subplots(nrows=4, ncols=2, figsize = (16,20))

road_conditions = ['Bump', 'Crossing', 'Give_Way', 'Junction', 'Stop', 'No_Exit', 'Traffic_Signal', 'Turning_Loop']
colors = [('#6662b3', '#00FF00'), ('#7881ff', '#0e1ce8'), ('#18f2c7', '#09ad8c'), ('#08ff83', '#02a352'), ('#ffcf87', '#f5ab3d'),
         ('#f5f53d', '#949410'), ('#ff9187', '#ffc7c2'), ('tomato', '#008000')]    
count = 0

def func(pct, allvals):
    absolute = int(round(pct/100*np.sum(allvals), 2))
    return "{:.2f}%\n({:,d} Cases)".format(pct, absolute)    

for i in [ax1, ax2, ax3, ax4, ax5, ax6, ax7, ax8]:
    
    size = list(df[road_conditions[count]].value_counts())
    if len(size) != 2:
        size.append(0)
    
    labels = ['False', 'True']
    
    
    
    i.pie(size, labels = labels, colors = colors[count],
                    autopct = lambda pct: func(pct, size), labeldistance=1.1,
                    textprops={'fontsize': 12}, explode=[0, 0.2])

    title = '\nPresence of {}'.format(road_conditions[count])

    i.set_title(title, fontsize = 18, color='grey')
    
    count += 1
"""
- Almost in every case (99.98%) Bumper was absent in the accident spot.

- In 5.7% cases, road accidents happened near the crossing.

- In 98.83% cases, there were no Stop near the accident area.

- 13.49% road accident cases recorded near the junctions.

- There are no accident cases recorded near the Turning Loop.

- 11.21% road accident cases recorded near the traffic signal.
"""
"""
### 10. How did the weather conditions affected the cases?
"""
df.head(2)
temp = pd.DataFrame(df["Temperature(F)"].value_counts()).reset_index().rename(columns={"index":"Temp","Temperature(F)":"Cases"})
plt.figure(figsize=(12,6))
plt.title('\n Accident cases reported for different temperature ranges \n', size=20, color='grey')
plt.ylabel('\n Temperature(F) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.scatterplot(x=temp.Cases[temp.Cases>100],y=temp.Temp,color="red")
a.xaxis.set_major_formatter(ticker.EngFormatter())

plt.show()
"""
- We see that maximum no of cases occured between temperature range: 40-80 F.
"""
humid = pd.DataFrame(df["Humidity(%)"].value_counts()).reset_index().rename(columns={"index":"Humidity","Humidity(%)":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different humidity \n', size=20, color='grey')
plt.ylabel('\n Humidity(%) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.scatterplot(x=humid.Cases,y=humid.Humidity,color="violet")
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()

"""
- We see that as the humidity increases the no of cases also increases.
"""
pressure = pd.DataFrame(df["Pressure(in)"].value_counts()).reset_index().rename(columns={"index":"Pressure","Pressure(in)":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different prerssures \n', size=20, color='grey')
plt.ylabel('\n Pressure(in) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.scatterplot(x=pressure.Cases,y=pressure.Pressure,color="purple")
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- Higest no of accident occured when the air pressure range is between 20(in) - 30(in).
"""
wind = pd.DataFrame(df["Wind_Chill(F)"].value_counts()).reset_index().rename(columns={"index":"Wind_Chill","Wind_Chill(F)":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different wind chill \n', size=20, color='grey')
plt.ylabel('\n Wind Chill(F) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.scatterplot(x=wind.Cases,y=wind.Wind_Chill)
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- Large no. of cases were reported for wind chill range between 51(F) - 71(F).
"""
wind = pd.DataFrame(df["Wind_Speed(mph)"].value_counts()).reset_index().rename(columns={"index":"Wind_Speed","Wind_Speed(mph)":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different wind speed \n', size=20, color='grey')
plt.ylabel('\n Wind_Speed(mph) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
a = sns.scatterplot(x=wind.Cases,y=wind.Wind_Speed[wind.Wind_Speed<50])
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
 - Maximum cases were recorded for the wind speed range between 5(mph) - 10(mph), which is not much and thus it is not a major cause behind the accidents.
"""
visibility = pd.DataFrame(df["Visibility(mi)"].value_counts()).reset_index().rename(columns={"index":"Visibility","Visibility(mi)":"Cases"})
plt.figure(figsize=(9,4))
plt.title('\n Accident cases reported for different visibility \n', size=20, color='grey')
plt.ylabel('\n Visibility(mi)) \n', fontsize=15, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
plt.xlim(1000,60000)
a = sns.scatterplot(x=visibility.Cases,y=visibility.Visibility[visibility.Visibility<20])
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- In maximum cases of road accident, the Visibility range is between 5(mi) - 10(mi), which is quite good and thus it is not a major cause behind the accidents.
"""
weather_condition_df = pd.DataFrame(df.Weather_Condition.value_counts().head(10)).reset_index().rename(columns={'index':'Weather_Condition', 'Weather_Condition':'Cases'})
plt.figure(figsize=(9,4))
plt.title('\nRoad Accident Percentage \nfor different Weather Condition in US (2016-2020)\n', size=20, color='grey')
plt.xlabel('\nAccident Cases\n', fontsize=15, color='grey')
plt.ylabel('\nWeather_Condition\n', fontsize=15, color='grey')
plt.xticks(fontsize=13)
plt.yticks(fontsize=12)
plt.xlim(0, 600000)
a = sns.barplot(x=weather_condition_df['Cases'], y=weather_condition_df['Weather_Condition'], palette='rainbow_r')
a.xaxis.set_major_formatter(ticker.EngFormatter())
plt.show()
"""
- We can see from this that Weather condition was Fair most of the time and thus it is not a major cause behind the accidents.
"""
"""
Insights:
- The majority of the accidents (> 75%) had a severity rating of ~2 i.e. the traffic was not highly impacted.
- The average length of the road extent affected by the accidents was ~0.6 miles. So, there was almost no traffic disturbance.
- We can see that the Number column has almost 70% data missing, Precipitation(in) and Wind_Chill(F) also have >=30% data missing.
Since, Number column (which shows the street number) has majority of the data missing, we should drop it.
- Los Angeles had the largest number of accidents (it is also the 2nd most populated city of US), Miami is the city with 2nd highest no. of accident cases reported. 
- On average, 7996 accidents very reported in Los Angeles yearly from 2016-2020.
- As a surprise, Chicago being the 3rh most populated city reported quite less number of accidents.
- Miami city reported 2nd largest number of accidents. Possibly because more than 1 state has city named Miami (total 11 from wikipedia), the State census dataset also shows Miami city in different states,similarly for Charlotte,Dallas,Houstan etc.
- One anomaly is that New York didn't report accidents larger than 10,000 despite being the most populated city of US (possible reason is less data from this city).
- We have 1167 cities which recorded only 1 accident case, this is another data source drawback.
- We see that California (CA) had the most number of accidents,followed by Florida, Oregon, Texas and New york. It appears to be right as California also has the largest population.
- In past 5 years, on average 246 accidents (daily) happened in California, implies approximately 10 accidents per hour.
- US/Eastern timezone region reported the most number of accident cases (Census data also shows that the population in this timezone is also the largest), followed by US/Pacific, US/Central and US/Mountain.
- US/Eastern timezone region accounts for about 38.84% of the accident cases.
- Eastern and Pacific timezone regions accounts for about 76% of the accident cases. Interestingly, these two regions also accounts for about 72% of the population (excluding those 14 split timezone states)
- Us/Mountain region reported the lowest number of accident cases. It agrees with the fact that Mountain timezone region has the least population, thus lowest no of accidents are reported there.
- On average 14 accidents were reported on street I-5 N.
- Highest number of accidents occur between 3:00PM - 6:00PM, and between 6:00AM - 9:00AM (Possibly because of work hours). 
- In evening, around 27% of the road accidents occurred in between 3:00PM to 6:00PM.
- Around 18% of the road accidents occurred in between 6:00AM to 9:00AM.
- The most-deadliest accident hour is 5:00PM implies the Evening Office-Returning Hours.
- The 2nd most-deadliest accident hour is 8:00AM implies the Morning Office-Going Hours.
- We see high number of accidents in the evening time frame because people are more tired at that time as compared to morning hours and thus have low concentration. 
- High no of accidents occur in the morning possibly because people are in a hurry to reach their destination.
- Thrusday reported the highest no of cases.
- It looks like overall the cases over different hour is evenly distributed on the business days.
- Only around 17% of the cases were reported on weekends.
- Working Days of the week have almost 2 times higher accident percentage, compared with the Weekend Days which is expected.
- As a surprise, there are less accidents occuring on the weekends.
- On the weekends, the distribution of accidents taking place over the day looks very different from that of on weekdays. It is more distributed through out the day, instead of peaking at a specific hour range.
- Around 18% of the cases are reported in December.
- July is the month with least (3.54%) no. of road accidents in US.
- 45% of the road accidents occurred only within the 3 months, October to December
- We see towards the end of the year the number of accidents is quite high.
- Also, the dataset is from Feb 2016-Dec2020, we are missing January data for 1 year but that should not make much of a difference.
- For year 2016,2019,2020 less data was recorded for first 6 months. This could explain why we observed more accident cases reports in the last quarter of the year.
- This dataset is a compilation of records from different sources, the accident cases for different months of different year shows that the sources has missing data.
- The trend of accident cases is almost exponentially increasing year by year with a sharp rise in year 2020.
- Year 2020 reported half of the cases in the whole data set.
- 65661 accidents reported per month in year 2020.
- On Average, 2159 accidents were reported daily i.e 90 cases per hour, in year 2020 alone.
- Least no. of accident cases were reported in year 2016.
- From 2019 to 2020 the average accident/day has increased 3 times in US.
- Compared to year 2016, average no of accidents per hour has increased by 6 times.
- Almost in every case (99.98%) Bumper was absent in the accident spot.
- In 5.7% cases, road accidents happened near the crossing.
- In 98.83% cases, there were no Stop near the accident area.
- 13.49% road accident cases recorded near the junctions.
- There are no accident cases recorded near the Turning Loop.
- 11.21% road accident cases recorded near the traffic signal.
- Maximum no of cases occured between temperature range: 40-80 F.
- Humidity increases the no of cases also increases.
- Higest no of accident occured when the air pressure range is between 20(in) - 30(in).
- Large no. of cases were reported for wind chill range between 51(F) - 71(F).
- Maximum cases were recorded for the wind speed range between 5(mph) - 10(mph), which is not much and thus it is not a major cause behind the accidents.
- In maximum cases of road accident, the Visibility range is between 5(mi) - 10(mi), which is quite good and thus it is not a major cause behind the accidents.
- Weather condition was Fair most of the time and thus it is not a major cause behind the accidents.
"""