"""
# Project: Is playing home advantageous to the soccer team?

## Table of Contents
<ul>
<li><a href="#intro">Introduction</a></li>
<li><a href="#wrangling">Data Wrangling</a></li>
<li><a href="#eda">Exploratory Data Analysis</a></li>
<li><a href="#conclusions">Conclusions and Limitations</a></li>
</ul>
"""
"""
<a id='intro'></a>
## Introduction
> In this report, we will be analysing the Europeon Soccer Dataset to find out the titled question. This dataset comes from Kaggle and is well suited for data analysis and machine learning. It contains data for 25k+ soccer matches, 10k+ players, and teams from several European countries from 2008 to 2016.
"""
"""
### Question
> Do teams playing home have an advantage? 
"""
"""
We will try to validate the possibility the above question.
"""
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import sqlite3

%matplotlib inline
"""
<a id='wrangling'></a>
## Data Wrangling
### General Properties
"""
"""
The data is present is a SQLite file, which is a kind of database, that contains data in different tables. We will be using the `Match` table for analysis.
"""
conn = sqlite3.connect('../input/soccer/database.sqlite')
cursor = conn.cursor()

sql = """SELECT * FROM Match""" # sql query to get data
# loading the data
match = pd.read_sql(sql, conn, index_col='id')
match.shape
match.head()
# selecting relevant col for matches
selected_cols = ['country_id', 'league_id', 'season', 'stage', 'date', 'match_api_id',
       'home_team_api_id', 'away_team_api_id', 'home_team_goal', 'away_team_goal']

match_data = match[selected_cols]

# selecting relevant col for betting data,
betting_data = match[['home_team_api_id', 'away_team_api_id', 'home_team_goal',
       'away_team_goal','B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA','IWH',
         'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD',
     'WHA', 'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA']]
# no any missing values or wrong data type was found except `date`
# we will ignore it for a this time
match_data.info()
match_data.describe()
# data types look right,
betting_data.info()
# But, there are many sources for which we don't have betting scores for all matches
betting_data.isnull().any(axis=1).sum()   # there are 23K rows which have atleast one missing value
# lets select data for top 3 sources by data availability - B365, BW, WH
betting_data = betting_data.drop(columns=['IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA',
     'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA'])
betting_data.info()
# still we have around 3K rows with atleast one missing value
betting_data.isnull().any(axis=1).sum()   
# lets check the number of matches for which we don't have betting score at all
betting_data.loc[:,"B365H":"WHA"].isnull().all(axis=1).sum()
# lets remove these records
all_indexes = betting_data.loc[:,"B365H":"WHA"].isnull().all(axis=1)
idx_to_del = all_indexes[all_indexes==True].index

betting_data = betting_data.drop(idx_to_del)

betting_data.loc[:,"B365H":"WHA"].isnull().all(axis=1).sum() # checking if we deleted the correct records
# now we have 45 rows with atleast one missing value
betting_data.isnull().any(axis=1).sum() 
betting_data.info()
# lets fill them with mean values
mean_dict = betting_data.loc[:,'B365H':].mean().to_dict()
betting_data.fillna(value=mean_dict, inplace=True)
# No any missing values. Yay!
betting_data.isnull().any(axis=1).sum() 
"""
<a id='eda'></a>
## Exploratory Data Analysis

<!-- ### Research Question 1 (Replace this header name!) -->
### Does team playing home have high changes to win?
"""
# lets look at big picture. 
match_data[['home_team_goal','away_team_goal']].describe()
"""
> It is seen from the mean that home teams generally score higher than away team, in turn win the match  
> Let's check the distribution of match outcomes
"""
# let's calculate the match outcome. This determines who won the match or it was a draw
def outcome(m):
    if m.home_team_goal > m.away_team_goal:
        return 'Home'
    elif m.home_team_goal < m.away_team_goal:
        return 'Away'
    elif m.home_team_goal == m.away_team_goal:
        return 'Draw'
match_data['Outcome'] = match_data.apply(outcome, axis=1);
# lets look what we got
match_data.Outcome.value_counts().plot(kind='pie');
match_data.Outcome.value_counts(normalize=True).round(4)*100
"""
> The above composition suggests that Home teams won around `46%` of matches, while away team won `29%`. A quarter of all matches were a draw.  
> This asserts that playing at home boosts the chances of a team to win the match
"""
"""
Let's explore this advantage. How exactly a team playing home is benefitted?  
### Does a team scores higher playing at home in comparison to away locations?
"""
"""
Let's see if this holds in team-wise comparison
"""
# calculating the mean goals a team scored when it played home and away
team_mean_home_scores = match_data.groupby(['home_team_api_id'])['home_team_goal'].mean()
team_mean_away_scores = match_data.groupby(['away_team_api_id'])['away_team_goal'].mean()
# merging both dataset based on team id
team_scores_home_away = pd.concat([team_mean_home_scores,team_mean_away_scores], axis=1, join='inner')
# lets examine the scores individually
sns.distplot(team_scores_home_away.home_team_goal);
sns.distplot(team_scores_home_away.away_team_goal);
"""
> Both the features follow normal distribution
"""
# comparing a team's mean goals when played home and away
team_scores_home_away['score_higher_at'] = team_scores_home_away.apply(outcome, axis=1)
team_scores_home_away.score_higher_at.value_counts(normalize=True).plot(kind='pie');
team_scores_home_away.score_higher_at.value_counts(normalize=True).round(4)*100
"""
> It was seen that `96%` teams have scored higher at home location in comparison to their away scores
"""
team_scores_home_away.plot.scatter(x='home_team_goal',y='away_team_goal', alpha=0.2);
team_scores_home_away[['home_team_goal','away_team_goal']].corr()
"""
> There is a `strong correlation` between a team's home scoring and away scoring.
"""
"""
### And How large this difference in score can be?
"""
# computing the score difference where teams have scored higher at home
team_scores_home_away['pos_score_diff'] = team_scores_home_away.home_team_goal - team_scores_home_away.away_team_goal
# deleting negative differences
team_scores_home_away[team_scores_home_away.pos_score_diff<0]=np.NaN
team_scores_home_away.pos_score_diff.plot(kind='box');
"""
> It looks like the team can score `0.2` to `0.5` goals extra playing at home. However, in extreme cases, this can go high upto `0.8`
"""
"""
### How do betting odds compare for home and away teams? 
"""
betting_data.describe()
# lets look at the betting odds distribution from the three sources 

f, axes = plt.subplots(1, 2, figsize=(10, 5), sharey=True, sharex=True)
sns.kdeplot(betting_data.BWH, shade=False, ax=axes[0]);
sns.kdeplot(betting_data.B365H, shade=False, ax=axes[0]);
sns.kdeplot(betting_data.WHH, shade=False, ax=axes[0]);

sns.kdeplot(betting_data.BWA, shade=False, ax=axes[1]);
sns.kdeplot(betting_data.B365A, shade=False, ax=axes[1]);
sns.kdeplot(betting_data.WHA, shade=False, ax=axes[1]);
"""
> Looks like all the sources have similar odds, 
"""
# lets check if the odds are in favour of home team or not for B365 odds
betting_data['B365Fav'] = betting_data['B365H'] < betting_data['B365A']
betting_data.B365Fav.value_counts(normalize=True).round(4)*100
"""
> For B365 scores, `70%` of matches have home odds less than away odds, which indicates home teams are favourable to win
"""
"""
<a id='conclusions'></a>
## Conclusions

So, the answer to our question - 'Is playing home advantageous to the soccer team?' is **Yes!!**. Below are our findings:  
1. A team playing home has almost 1.5 time the chances to win the match.
2. 96% out of 299 teams scored higher goals playing home than their respective mean away score.
3. Teams can score upto 0.5 goals extra playing home
4. There is a strong correlation between a team's home score and away scores. Generally this suggests that a team will score similar at both home and away conditions.
5. 70% matches have home team as favourable (from odd scores)
"""
"""
**Limitations**
1. Current analysis consider *mean* values for calculation. However, *mean* can be effected by presence of outliers in dataset (some teams may have scored exceptionally high goals in some matches). We could have used *median* instead (which can be seen same for both home and away teams).
2. The analysis performed does not has any statistical inference.
3. This dataset does not contains data for all matches. (sample data) There could have been different number of samples available for a team's matches played at home and away.
"""