"""
# About Me
Hello all, I am Hasan and this is my first kernel. I'm still learning python and data science libraries. So your comments are extremely helpful to me. 

# Introduction
Turkey stands between Europe and Asia. Geographically country is rather young hence it is more sismographically active. I found this dataset on Kaggle which originates from Kandilli Rasathanesi which is a reputable organisation affiliated with Bogazici University. I will try to analyse the data and create some visualisations. 
"""
#Importing the necessary libraries
import numpy as np
import pandas as pd
import numpy as np
import chardet
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
"""
# Data Cleaning
"""
"""
The dataset is in a txt file and whilst reading it, an encoding error has raised. I used *chardet* library to detect the encoding.
"""
with open('../input/datasetindex.txt', 'rb') as rawdata:
    result = chardet.detect(rawdata.read(1000000))
    
print(result)
"""
As seen above encoding is *ISO-8859-1* on a confidence level of 73%. 
"""
#reading the dataset
data = pd.read_table('../input/datasetindex.txt', sep='\t', encoding='ISO-8859-1', index_col=0)
data.info()
data.sample(5)
"""
Above it is seen that the column names are in Turkish and the last column is partially in Turkish. I am translated them and replace the column names with their English equivalents. 
"""
col_names = ['earthquake_code', 'date', 'time', 'latitude', 'longitude',
       'depth(km)', 'xM', 'MD', 'ML', 'Mw', 'Ms', 'Mb', 'type', 'location']
data.columns = col_names
data.head(3)
"""
Date and time are two different columns and they are not in *datetime* format. When I tried to convert them to datetime in one column, I recieved an error that there was a problem with seconds. One of the data point had higher than 59, which is impossible for a second. Below, you may see how I found that point and updated it. 
"""
#There was a wrong data of seconds, updated that
sec_err = data['time'].str.extract(r'\:(\d+)\.')[0].apply(lambda x: int(x))
print(sec_err[sec_err>59])
data.loc[13080, 'time'] = '10:03:59.00'
print(sec_err[sec_err>59])
"""
Since I eradicated the error I can now create one column of datetime and drop the others. 
"""
data['date_time'] = pd.to_datetime(data['date'] + ' ' + data['time'])
data = data.drop(['date', 'time'], axis=1)
"""
Whilst checking the location column I realised there are other countries in the data. I cleared them out. 
"""
countries = ['YUNANISTAN', 'GURCISTAN', 'RUSYA', 'IRAN', 'AZERBAYCAN', 'MAKEDONYA', 
             'BULGARISTAN', 'SURIYE', 'IRAK', 'ROMANYA', 'ARNAVUTLUK', 'MISIR', 
             'KIBRIS RUM KESIMI', 'UKRAYNA', 'YUNANiSTAN', 'iRAN', 'BULGARiSTAN',
             'GÜRCiSTAN', 'MISIR', 'SURiYE', 'ISRAIL', 'ONiKi ADALAR YUNANiSTAN',
             'KIBRIS RUM KESiMi']
for country in countries:
    data = data[data.location != country]
    
#delete the points out of Turkey's geolocation
data = data[data.latitude >= 36]
data = data[data.longitude >= 26]
"""
The location data is very detailed and not structured. It shows the provinces and cities. City names are included generally in paranthesis. I am creating a new column showing only the city names. 
"""
data['city'] = data['location'].str.extract(r'\((.+)\)')
data.city = data.city.fillna(data[data.city.isnull()].location)
"""
Let's explore the data. Here is the 10 worst earthquakes of Turkey. The heaviest earthquate was occured in the city of Erzincan in 1939. 
"""
data[['date_time', 'city', 'xM']].sort_values('xM', ascending=False).head(10)
"""
# Visualisations
"""
data.xM.plot.hist(bins=10)
"""
The frequency of earthquakes are shown above. The most frequent eathquakes have a magnitude between 3 and 4. The Richter magnitude scale categorises earthquakes below 5 as minor. I plotted some non-minor insights below. The month of May is a bit unlucky, the most of the earthqaukes more than 5 happening on May. Also Wednesday is the most shakey day of the week. 
"""
data['year'] = data.date_time.apply(lambda x: x.year)
data['month'] = data.date_time.apply(lambda x: x.month)
data['weekday'] = data.date_time.apply(lambda x: x.dayofweek)
dataover5 = data[data.xM >= 5]

plt.figure(figsize=(15,5))
plt.subplot(1,2,1)
dataover5.month.value_counts().sort_index().plot.bar()
plt.xlabel('Months of the year')
plt.subplot(1,2,2)
dataover5.weekday.value_counts().sort_index().plot.bar()
plt.xlabel('Days of the week')
"""
By looking all data, May is still the worst month of all, however, the most shakey day is Sunday. This refutes a popular Turkish pun 'Tuesday shakes' (Salı sallanır).  
"""
plt.figure(figsize=(15,5))
plt.subplot(1,2,1)
data.month.value_counts().sort_index().plot.bar()
plt.xlabel('Months of the year')
plt.subplot(1,2,2)
data.weekday.value_counts().sort_index().plot.bar()
plt.xlabel('Days of the week')
"""

Below I plotted the frequency of the earthquakes by years. The graph on left shows the occurence of earthquakes are on an increasing trend. The occurences peak somewhere in 1970s, this probably means that there were new technologies to capture the data from then. The graph on the right shows the frequency of eathquakes with magnitude higher then 5. 
"""
plt.figure(figsize=(15,5))
plt.subplot(1,2,1)
plt.plot(data.year.value_counts().sort_index())
plt.subplot(1,2,2)
plt.plot(data[data.xM >= 5].year.value_counts().sort_index())
"""
The graphics above gives their trends differently by years. Below the total earthquakes and the ones more than 5 can be seen in the same graph. 
"""
data.year.value_counts().sort_index(ascending=False).plot.area()
data[data.xM >= 5].year.value_counts().sort_index(ascending=False).plot.area()
plt.legend(['all earthquakes', '5 plus'])
"""
Maps are better to comprehend the geologic data. Below you may see the projection of earthquakes with magnitudes more than 5. Seismicly most active areas of Turkey are Aegean Region and Eastern Anatolia. 
"""
sel = data[data.xM >= 5]
lon = sel['longitude'].values
lat = sel['latitude'].values
xM = sel['xM'].values

fig = plt.figure(figsize=(12, 6))
m = Basemap(projection='lcc', resolution='l', lat_0=39, lon_0=35, width=1.7E6, height=1E6)
m.bluemarble()
m.drawcoastlines(color='gray')
m.drawcountries(color='red')

m.scatter(lon, lat, latlon=True, c=xM, s=xM*2, cmap='YlOrRd', alpha=0.7)

plt.colorbar()
plt.clim(5, 8)