"""
This note book is about predicting the score difference between two teams in a basket ball game, their scores and winner in a single model.

The task at hand is clearly both classification and regression in one model.

As the names imply, the dataset games_season.csv provides data on the season games and games_tourney.csv provides data on the tournaments. Tournaments are played at the end of seasons, so it will be apt to build and train our prediction models on season data initially and evaluate the model preeictive performance on the tournament data. 

We will predict score differences and add the same to games_tourney.csv to create a file similar to games_season_enriched.csv file.

In the process, we will progressively build and improvise various models.
"""
from numpy.random import seed
seed(9)
import tensorflow
tensorflow.random.set_seed(9)
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All" 
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
"""
We will use only the games_tourney.csv and games_season.csv as games_season_enriched.csv is same as games_tourney.csv but with a predicted column, which we will create as an output of this exercise.
"""
games_season = pd.read_csv('/kaggle/input/basket-ball-dataset/games_season.csv')
print(games_season.info())
games_season.head()
games_tourney = pd.read_csv('/kaggle/input/basket-ball-dataset/games_tourney.csv')
print(games_tourney.info())
games_tourney.head()
games_season_enriched = pd.read_csv('/kaggle/input/basket-ball-dataset/games_season_enriched.csv')
print(games_season_enriched.columns)
games_tourney.shape
games_tourney.describe()
print(games_tourney.season.unique())
print(games_tourney.won.unique())
print(games_tourney.home.unique())
print(len(games_tourney.team_1.unique()))
print(len(games_tourney.team_2.unique()))
import matplotlib.pyplot as plt 
import seaborn as sns

sns.displot(x='seed_diff',data=games_tourney)
plt.show()
sns.displot(x='score_diff', data=games_tourney)
plt.show()
sns.lineplot(x='seed_diff', y= 'score_diff', data=games_tourney)
plt.show()
sns.barplot(x='home', y= 'won', data=games_season)
plt.show()
"""
From the graphs we can infer that the small datasets indicate that the score and seed difefrences are near normally distributed. We see seed difference is positively correlated to score difference (which detremines who wins), so also whether the ground is home turf or not. he data is clean integer data ready for predictions and can be straight forward adopted for model building.
"""
"""
Let us construct our first, basic deep learning model with one input and one output to predict score_diff, given seed_diff. For this purpose we use the games_tourney.csv dataset as games_season.csv does not have seed difference feature.
"""
# Input/dense/output layers
from keras.layers import Input, Dense
input_tensor = Input(shape=(1,), name='my_input_data_tensor')
output_tensor = Dense(1, name='my_output_layer')(input_tensor)

# Build the model
from keras.models import Model
model = Model(input_tensor, output_tensor)

# Compile the model
model.compile(optimizer='adam', loss='mean_absolute_error')
model.summary()
"""
Let us visualize our model.
"""
from keras.utils import plot_model
plot_model(model, to_file = 'model.png')
img = plt.imread('model.png')
plt.imshow(img)
plt.show()
games_tourney.groupby('season').team_1.count()
"""
Let us split the dataset into train & test sets with train set upto season 2011 (3430 observations) and use remaining as test set.
"""
games_tourney_train = games_tourney.iloc[:3430,]
games_tourney_test = games_tourney.iloc[3430:,]
# Now fit the model
model.fit(games_tourney_train['seed_diff'], games_tourney_train['score_diff'],
          epochs=1,
          batch_size=128,
          validation_split=.1,
          verbose=True)
# let us evaluate the model performance

# X variable from the test data
X_test = games_tourney_test['seed_diff']

# y variable from the test data
y_test = games_tourney_test['score_diff']

# Evaluate the model on the test data and make predictions
print(model.evaluate(X_test, y_test, verbose=False))
preds = model.predict(X_test,verbose=False)
preds[:10]
"""
This preliminary model predicts the score difference between the teams with an avearge deviation of 11.61
"""
"""
Now, let us develop a model to predict score difference by developing team strengths given two teams in the games_season.csv, rather than using the seed difference column given in the games_tourney.csv dataset. First we start with building team strength look up model using a team look up embed layer. The model will learn the strength scores, and the difference can be viewed to predict who wins the game and to predict the score difference.
"""
# how many teams played in all seasons?
print(games_season['team_1'].max())
print(games_season['team_1'].min())
from keras.layers import Embedding

# Count the unique number of teams
n_teams = np.unique(games_season['team_1']).shape[0]

# Create an embedding layer
team_lookup = Embedding(input_dim=n_teams,
                        output_dim=1,
                        input_length=1,
                        name='Team-Strength')
from keras.layers import Flatten

# Create an input layer for the team ID
teamid_in = Input(shape=(1,))

# Lookup the input in the team strength embedding layer
strength_lookup = team_lookup(teamid_in)

# Flatten the output
strength_lookup_flat = Flatten()(strength_lookup)

# Combine the operations into a single, re-usable model
team_strength_model = Model(teamid_in, strength_lookup_flat, name='Team-Strength-Model')
"""
Now we define team strength look up model for team_1 and team_2.
"""
# Input layer for team 1
team_in_1 = Input(shape=(1,), name= 'Team-1-In')

# Separate input layer for team 2
team_in_2 = Input(shape=(1,), name= 'Team-2-In')

# Lookup team 1 in the team strength model
team_1_strength = team_strength_model(team_in_1)

# Lookup team 2 in the team strength model
team_2_strength = team_strength_model(team_in_2)
# Import the Subtract layer from keras
from keras.layers import Subtract

# Create a subtract layer using the inputs from the previous exercise
score_diff = Subtract()([team_1_strength, team_2_strength])
score_diff
# Subtraction layer from previous exercise
score_diff = Subtract()([team_1_strength, team_2_strength])

# Create the model
model = Model([team_in_1, team_in_2], score_diff)

# Compile the model
model.compile(optimizer='adam', loss='mean_absolute_error')
model.summary()
plot_model(model, to_file = 'model.png')
img = plt.imread('model.png')
plt.imshow(img)
plt.show()
# Get the team_1 column from the regular season data
input_1 = games_season['team_1']

# Get the team_2 column from the regular season data
input_2 = games_season['team_2']

# Fit the model to input 1 and 2, using score diff as a target
model.fit([input_1,input_2],
          games_season['score_diff'],
          epochs=1,
          batch_size=2048,
          validation_split=.1,
          verbose=True)
# Get team_1 from the tournament data
input_1 = games_tourney['team_1']

# Get team_2 from the tournament data
input_2 = games_tourney['team_2']

# Evaluate the model using these inputs and make predictions
print(model.evaluate([input_1, input_2], games_tourney['score_diff'], verbose=False))
preds = model.predict([input_1, input_2], verbose=False)
preds[:10]
"""
As we know home is a factor influencing who wins the game, in effect predicting score difference, let us add that to our model.
"""
from keras.layers import Concatenate
# Create an Input for each team
team_in_1 = Input(shape=(1,), name='Team-1-In')
team_in_2 = Input(shape=(1,), name='Team-2-In')

# Create an input for home vs away
home_in = Input(shape=(1,), name='Home-In')

# Lookup the team inputs in the team strength model
team_1_strength = team_strength_model(team_in_1)
team_2_strength = team_strength_model(team_in_2)

# Combine the team strengths with the home input using a Concatenate layer, then add a Dense layer
out = Concatenate()([team_1_strength, team_2_strength, home_in])
out = Dense(1)(out)
# Make a Model
model = Model([team_in_1, team_in_2, home_in], out)

# Compile the model
model.compile(optimizer='adam', loss='mean_absolute_error')
model.summary()
plot_model(model, to_file = 'model.png')
img = plt.imread('model.png')
plt.imshow(img)
plt.show()
# Fit the model to the games_season dataset
model.fit([games_season['team_1'], games_season['team_2'], games_season['home']],
          games_season['score_diff'],
          epochs=1,
          verbose=True,
          validation_split=.1,
          batch_size=2048)

# Evaluate the model on the games_tourney dataset
print(model.evaluate([games_tourney['team_1'], games_tourney['team_2'], games_tourney['home']], games_tourney['score_diff'], verbose=False))
"""
Now let us predict the score difference and add the predictions to the tournament dataset.
"""
# Predict
games_tourney['pred'] = model.predict([games_tourney['team_1'],games_tourney['team_2'], games_tourney['home']])
games_tourney.head()
"""
Now, we will build our next model with 3 inputs, 'seed_diff', 'home', and 'pred'.
"""
# Create an input layer with 3 columns
input_tensor = Input(shape=(3,))

# Pass it to a Dense layer with 1 unit
output_tensor = Dense(1)(input_tensor)

# Create a model
model = Model(input_tensor, output_tensor)

# Compile the model
model.compile(optimizer='adam', loss='mean_absolute_error')
model.summary()
"""
Let us split the games_tourney data into - before season 2010 as training set, and the ones from season 2010 onwards as test set. Then we fit, evaluate & predict our model on these datsets.
"""
games_tourney_train = games_tourney[games_tourney['season'] < 2010]
games_tourney_test = games_tourney[games_tourney['season'] >= 2010]
print(games_tourney_train.shape)
games_tourney_test.shape
# Fit the model
model.fit(games_tourney_train[['home', 'seed_diff', 'pred']],
          games_tourney_train['score_diff'],
          epochs=1,
          verbose=True)
# Evaluate the model on the games_tourney_test dataset
print(model.evaluate(games_tourney_test[['home', 'seed_diff', 'pred']],
               games_tourney_test['score_diff'], verbose=False))
preds = model.predict(games_tourney_test[['home', 'seed_diff', 'pred']])
preds[:10]
"""
With this the mean absolute error for the predicted score diffrence is smaller than the earlier 2 input model, which was about 11.68
"""
"""
Now we use the games_tourney data to build a model that makes two predictions: the scores of both teams in a given game. This is ultiple target regression problem un like all those so far, which were single regression target problems.
"""
# Define the input
input_tensor = Input(shape=(2,))

# Define the output
output_tensor = Dense(2)(input_tensor)

# Create a model
model = Model(input_tensor, output_tensor)

# Compile the model
model.compile(optimizer='adam', loss='mean_absolute_error')
"""
To build the model, we again use games_tourney.csv data to create train and test sets.
"""
games_tourney_train = games_tourney[games_tourney['season'] < 2012]
games_tourney_test = games_tourney[games_tourney['season'] >= 2012]
games_tourney_train = games_tourney[games_tourney['season'] < 2012]
games_tourney_test = games_tourney[games_tourney['season'] >= 2012]
# Fit the model
model.fit(games_tourney_train[['seed_diff', 'pred']],
  games_tourney_train[['score_1', 'score_2']],
  verbose=False,epochs=1000,batch_size=64)
"""
We can evaluate the model for the error and also see our predictions for score_1 and score_2
"""
print(model.evaluate(games_tourney_test[['seed_diff', 'pred']], games_tourney_test[['score_1', 'score_2']], verbose=False))
pred_scores = model.predict(games_tourney_test[['seed_diff', 'pred']], verbose=False)
pred_scores = pd.DataFrame(pred_scores, columns = ['score_1', 'score_2'])
pred_scores.head(10)
(pred_scores['score_1']-pred_scores['score_2']).mean()
"""
Now we will create a different type of 2-output model. We will predict the score difference, instead of both team's scores and predict the probability that team 1 won the game. This is both regression and classification predictions using one model with inputs being seed difference and predicted score difference.

In this model, we will turn off the bias for each layer as the inputs have means of very close to zero. So your model shouldn't need the bias term to fit the data well.
"""
print(games_tourney_train[['seed_diff', 'pred']].mean())
# Create an input layer with 2 columns
input_tensor = Input(shape=(2,))

# Create the first output
output_tensor_1 = Dense(1, activation='linear', use_bias=False)(input_tensor)

# Create the second output (use the first output as input here)
output_tensor_2 = Dense(1, activation='sigmoid', use_bias=False)(output_tensor_1)

# Create a model with 2 outputs
model = Model(input_tensor, [output_tensor_1, output_tensor_2])
# Import the Adam optimizer
from keras.optimizers import Adam

# Compile the model with 2 losses and the Adam optimzer with a higher learning rate
model.compile(loss=['mean_absolute_error', 'binary_crossentropy'], optimizer=Adam(lr=.01))

# Fit the model to the tournament training data, with 2 inputs and 2 outputs
model.fit(games_tourney_train[['seed_diff', 'pred']],
          [games_tourney_train[['score_diff']], games_tourney_train[['won']]],
          epochs=10,
          verbose=True,
          batch_size=16384)
"""
Let us have a look at the weights for this model. 
"""
# Print the model weights
print(model.get_weights())
"""
The last weight of the model converts the predicted score difference to a predicted win probability. If we multiply the predicted score difference by the last weight of the model and then apply the sigmoid function, we get the win probability of the game.
"""
"""
We can consider a close game as with 1 point difference and blowout game is with a 10 point difference. Let us find out probabilities for such games.
"""
# Import the sigmoid function from scipy
from scipy.special import expit as sigmoid

# Weight from the model
weight = 0.30

# Print the approximate win probability predicted close game
print(sigmoid(1 * weight))

# Print the approximate win probability predicted blowout game
print(sigmoid(10 * weight))
"""
57% is a situation where either team can win and 95% is a situation where team_1 wins.
"""
"""
Now, finally we evaluate our model performance on test data.
"""
# Evaluate the model on new data
print(model.evaluate(games_tourney_test[['seed_diff', 'pred']],
               [games_tourney_test[['score_diff']], games_tourney_test[['won']]], verbose=False))
"""
The first number is the sum of both the loss functions, and  the next 2 numbers are the loss functions defined in the model, 'mean_absolute_error' and 'binary_crossentropy'.
"""
"""
Make the final predictions.
"""
# make predictions
final_predictions = model.predict(games_tourney_test[['seed_diff', 'pred']], verbose=False)
print(len(final_predictions))
print(final_predictions[0][:10],'\n','\n', final_predictions[1][:10])
"""
The first list the predicted score differences between the two teams and the second list is the predicted probability that the first tea is the winner.
"""
print("------ End of Notebook ------")