import numpy as np
import streamlit as st

from src.app.utils import (
    _calc_elo_metrics,
    _calc_hyperparameter_df,
    _calc_longest_winstreak,
    _calc_money_won,
    _calc_track_df,
    _calc_win_rate,
    _plot_cdf,
    _plot_elo_history,
    _plot_roc_curve,
    _plot_slice,
    _plot_track_elo,
    _plot_waterfall,
    _plot_win_proba,
    load_elo_df,
    load_elo_system,
    load_history,
    load_player_stats,
    load_players_list,
    load_race_df,
    load_study,
)

st.title("Predicting Of The Winner Of A Race Using Muli-Elo")


race_df = load_race_df()
history_df = load_history()
players_list_df = load_players_list()
elo_system = load_elo_system()
elo_df = load_elo_df()
study = load_study()


# MODEL SUMMARY ####################################################################################

st.markdown(
    r"""

## Introduction

This website presents the result of the **BIG DATA predictions** contest.
The contest revolves around a dataset of race results to be analyzed.
An example of the dataset is shown below.
We use a multivariate Elo approach adapted from [this paper](https://arxiv.org/pdf/1910.12581.pdf)
to model the skill of individual racers.
This can also be used to predict the outcome of races.

In the first section, we give an overview of the model used and how it was adapted to this problem.
In the second section, we analyze the individual racers and their Elo.
Finally, in the third section, we analyze the odds of winning for two different racers
and what factors influence those odds.
"""
)

st.write(race_df.head())

st.markdown(
    r"""
## Model

In this section, we provide an overview of the model that was applied to the race dataset.
We used a multivariate Elo approach with some modifications to improve predictive performance,
which we will discuss later.
But first, we begin with the standard Elo model.

### Standard Elo Model
The Elo rating system was originally developed for rating chess players and is based on a pairwise
comparison of data in which two chess players compete against each other.
A mathematical overview of the Elo model can be found [here](https://arxiv.org/pdf/1806.02643.pdf).
It states the following:

> Suppose $n$ agents play a series of pairwise matches against each other.
> Elo assigns a rating $r_i$ to each player $i \in 1..n$ based on their wins and losses, which we
> represent as $n$ vector $r$. The predicted probability that $i$ wins against $j$,
> given their Elo ratings, is.
> $$
\hat{p}_{ij} := \sigma\left(\alpha \cdot (r_i - r_j)\right)
>$$
> where
> $$
\sigma(x) := \frac{1}{1 + e^{-x}}
\;\;\;
\text{and}
\;\;\;
\alpha := \frac{\log(10)}{400}.
> $$
> The logit loss is used to measure the adjustment, which is defined as follows
> $$
\ell(p_{ij}, \hat{p}_{ij}) = - p_{ij} \log{\hat{p}_{ij}} - (1 - p_{ij}) \log{(1 - \hat{p}_{ij})}
> $$
> where $p_{ij}$ is the probability that $i$ wins against $j$. Suppose in the $t$-th race,
> player $i$ faces $j$. If player $i$ wins, $S^t_{ij} = 1$ and if player $i$ loses, $S^t_{ij} = 0$.
> Online gradient descent on $\ell$ yields
> $$
r^{t + 1}_i := r^t_{i} + K \cdot \nabla_{r_i} \ell(S^t_{ij},
\hat{p}_{ij}) = r^t_{i} + K \cdot (S^t_{ij} - \hat{p}^t_{ij}).
> $$

### Multidimensionales Elo-Modell

The Elo model is great for estimating player skill when the game remains the same.
In our case, we want to model a player's skill on different tracks where different
skills might be required.
We can assume that a player who is very good on one track will do reasonably well on another track,
since some skills are transferable.
Nevertheless, the player may have a track that he plays a lot and is particularly good at.
We can model this relationship using a multivariate elomodel.
The model we use to model race performance is adpated from
[this](https://arxiv.org/pdf/1910.12581.pdf) paper.
We assume that a player's ability $i \in 1..n$ on a track $d \in 1..D$ can be modeled as a
combination of a base ability $r^{base}_i$, which is transferable to all tracks,
and a track-specific ability $r^{track}_{id}$. This can be written as
$$
r^{total}_{id} = r^{base}_i + c \cdot r^{track}_{id}
$$
Where $c \in [0,1]$ determines the importance of trace-specific capabilities.

### Extensions of the Elo model

The Elo model was originally developed to get a good estimate of a player's abilities.
Not necessarily to accurately predict the outcome of a match.
We were inspired by the website
[fivethirtyeight.com](https://fivethirtyeight.com/methodology/how-our-nfl-predictions-work/),
where an Elo model is extended with some modifications to better predict the outcome of race in
the NFL.
In the following section, we present the specific extensions we used to better predict the
outcome of races.

#### Challenger Advantage

When we look at the data, we see that the challenger's win rate is slightly lower than the
opponent's win rate.
This could be because the opponent is better prepared than the challenger in this situation.
This is similar to the home field advantage in football.
We model this relationship by giving the challenger an Elo advantage/disadvantage.
This is modeled by adding a constant value $a \in \mathbb{R}$. If player $i$ is the challenger,
then the new Elo for that player is
$$
r^{new}_{i} = r^{base}_i + a.
$$


#### Money Importance

The more money wagered in a race, the more important it is and the more this race should affect
the elo gain/loss.
For this reason, the $K$ factory should scale with the amount of money.
Since the amount of money wagered is between $30$ and $1,000,000$, we model the effect
of money that is bet as

$$
K^{money} = K^{base} \cdot m
\cdot \left( 1 + \frac{\log(money) - \log(30)}{\log(1,000,000) - \log(30)} \right)
$$

with the money importance factory $m \in [0,\infty)$.
Logarithmic scaling was chosen so that a $1 million race would not completely overwrite the results
of a $30 race.

#### Adaptive K

For the adaptive K-factor, we took inspiration from the
[fivethirtyeight.com](https://fivethirtyeight.com/features/serena-williams-and-the-difference-\
between-all-time-great-and-greatest-of-all-time/) model for tennis.
The $K$ factor determines how much the Elo rating should change after a match result.
In the example, the $32$ K-factor means that a player could lose at most $32$ points
(if his probability was 1 and he lost).
Instead of using a constant $K$, we instead let it depend on $n$,
the number of races a player has played.
The equation for $K$ is then:

$$
K^{adaptive} = \frac{1}{(n + M)^s} \cdot K^{base}
$$

where $M \in \mathbb{N}$ and $s \in [0, 1]$.
When a new player is added and we haven't seen any race results yet,
we are not sure about his abilities.
For this reason, the estimated elo very volatile at the beginning.
With more race, the skill estimate becomes more stable and we decrease K.

### Model Training

In the previous section, we introduced a number of hyperparameters such as $\beta$ or $c$.
But we still need to find out which values are the most optimal and best fit to the dataset.
We use the Baysian optimization framework [optuna](https://optuna.org) to
find the best hyperparameter by maximizing the log likelihood on the training dataset.
"""
)

# MODEL FIT ########################################################################################

st.markdown(
    r"""
### Evaluating Model Fit

As with all data science projects, we divide the dataset by date into a training dataset and a
test dataset.
Hyperparameter optimization is performed on the training dataset and the fit is evaluated on the
test dataset.
Below is a figure showing the fit of the model by plotting the modeled cumulative
distribution function against the estimated cdf of the test data set.
"""
)

fig = _plot_cdf(elo_system, elo_df)
st.plotly_chart(fig)

# PREDICTION PERFORMANCE ###########################################################################

st.markdown(
    r"""
### Prediction Performance

The Elo model estimates the probability of winning for all pairs of players.
This can be used to predict the winner of a race.
In this section we look at the predictive performance of the Elo model by looking at
some metrics and plots.
It is noted that the accuracy of the model is $\gt 0.78$.
This is a reasonable performance for racing dataset where the
outcome is highly volatile.
"""
)

elo_logit, elo_accuracy, elo_f1, elo_auc = _calc_elo_metrics(elo_df)

col1, col2, col3, col4 = st.columns(4)
col1.metric("Logit Loss", np.round(elo_logit, 2))
col2.metric("Accuracy", np.round(elo_accuracy, 2))
col3.metric("F1 Score", np.round(elo_f1, 2))
col4.metric("AUC Score", np.round(elo_auc, 2))

fig = _plot_roc_curve(elo_df)
st.plotly_chart(fig)

# Hyperparameter ###################################################################################

st.markdown(
    r"""
### Hyperparameter

Let's take a look at the hyperparameters learned by the model.
These hyperparameters are fitted using Bayesian optimization and
minimize the log-likelihood on the training dataset.
We can observe that the challenger advantage $a$ takes a negative
value, which means that the challenger is at a disadvantage.
It should also be noted that the importance of money $i$ approaches zero,
meaning that a high stakes race is not much more important than a normal race.
"""
)

hyperparameter_df = _calc_hyperparameter_df(elo_system)
st.write(hyperparameter_df)

st.markdown(
    r"""
In the slice graph below we can see which values of each parameter contributed to the best
performance.
"""
)

selected_parameter = st.selectbox(
    "Hypterparameter",
    [
        k
        for k in list(elo_system.rater.__dict__.keys())
        if k not in ["Omega", "nb_tracks", "luck"]
    ],
)
fig = _plot_slice(study, selected_parameter)
st.plotly_chart(fig)


# RANKING ##########################################################################################

st.header("Players Rankings")

st.markdown(
    r"""
Now that we have calculated an Elo score for each player, we can create a ranking of the
best players based on their score.
Note that the players with the highest Elo are not necessarily the ones with the highest win rate.
The Elo also depends on the strength of your opponent and your performance in the last races.
"""
)

st.write(
    players_list_df.sort_values("current_elo", ascending=False)
    .head(10)
    .reset_index(drop=True)
    .drop(columns=["max_elo"])
    .rename(columns={"current_elo": "elo"})
    .style.format(
        {"current_elo": "{:.1f}", "max_elo": "{:.1f}", "win_rate": "{:.2f} %"}
    )
)

# Player stats #####################################################################################

st.header("Player Stats")

st.markdown(
    r"""
Let's look at the performance of an individual player and calculate some metrics.
You can choose another **player_id** to analyse a different player.
"""
)

player_id = st.selectbox("Player ID", players_list_df["player_id"].unique(), index=2)
player_df = load_player_stats(history_df, player_id)

max_elo = player_df["elo_challenger_after"].max()
longeset_win_streak = _calc_longest_winstreak(player_df)
win_rate = _calc_win_rate(player_df)
money_won = _calc_money_won(player_df)
base_elo = elo_system.id2player[player_id].elo[0]
track_elo = np.mean(elo_system.id2player[player_id].elo[1:])


col1, col2, col3, col4, col5 = st.columns(5)
col1.metric("Current Base Elo", str(np.round(base_elo, 2)), np.round(track_elo, 2))
col2.metric("Max Elo", str(np.round(max_elo, 2)))
col3.metric("Win Rate", str(np.round(win_rate * 100, 1)) + " %")
col4.metric("Longest Win Streak", str(longeset_win_streak))
col5.metric("Money Earned", str(money_won))

st.subheader("Elo over Time")

st.markdown(
    r"""
In the graph below you can see the evolution of the Elo points for the selected player over time.
Green points represent a won game and red points represent a lost game.
You can see that the Elo value is much more volatile at the beginning of a player's history.
This is due to the adaptive $K$-factor mentioned above.
"""
)

history_fig = _plot_elo_history(player_df)
st.plotly_chart(history_fig)

st.subheader("Elo by Track")

st.markdown(
    r"""
Below is a table of player's elo for every track.
This is possible because we model the elo for every track separately.
We can observe, that the elo on tracks where the player has a high winrate is better
that on track with lower winrates.
"""
)

track_df = _calc_track_df(player_df, elo_system, player_id)
st.write(track_df.style.format({"elo": "{:.1f}", "win_rate": "{:.2f} %"}))
st.markdown(
    r"""
    The Elo values for the different tracks are shown in the bar chart below.
    The base Elo value is marked with a red line.
"""
)
track_elo_fig = _plot_track_elo(elo_system, player_id)
st.plotly_chart(track_elo_fig)

# Compare #####################################################################################

st.header("Compare Players")

st.markdown(
    r"""
    With the Elo model we can compare two players and predict their chances of winning.
    Since players have slightly different elo's at different tracks,
    the chance of winning depends on the track on which the race is being held.
    Since the challenger has a slight disadvantage, the outcome also depends
    on which player is the challenger.
    You can select your own two players which will be compared.
"""
)

track_id = st.selectbox("Track ID", sorted(history_df["track_id"].unique()))

challenger_col, opponent_col = st.columns(2)
challenger_col.subheader("Challenger")
challenger_id = challenger_col.selectbox(
    "Player ID", players_list_df["player_id"].unique(), key="challenger_id", index=2
)
challenger_elo = players_list_df.query("player_id == @challenger_id")[
    "current_elo"
].iloc[0]

opponent_col.subheader("Opponent")
opponent_id = opponent_col.selectbox(
    "Player ID", players_list_df["player_id"].unique(), key="opponent_id", index=4
)
opponent_elo = players_list_df.query("player_id == @opponent_id")["current_elo"].iloc[0]

challenger_col.metric(
    "Elo", np.round(challenger_elo, 2), np.round(challenger_elo - opponent_elo, 2)  # type: ignore
)
opponent_col.metric(
    "Elo", np.round(opponent_elo, 2), np.round(opponent_elo - challenger_elo, 2)  # type: ignore
)

proba = elo_system._get_proba(challenger_id, opponent_id, track_id)

fig = _plot_win_proba(proba)
st.plotly_chart(fig)

st.markdown(
    r"""
    The probability of winning is determined by the Elo difference.
    What this difference is made up of is shown in the following graph.
    The three factors are each player's base Elo, the track-specific skill,
    and the challenger's disadvantage.
"""
)

fig = _plot_waterfall(elo_system, challenger_id, opponent_id, track_id)
st.plotly_chart(fig)

st.markdown(
    r"""
------------------------
**References:**

* [A Multivariate Elo-based Learner Model for Adaptive Educational Systems]
(https://arxiv.org/pdf/1910.12581.pdf)
* [Re-evaluating Evaluation]
(https://arxiv.org/pdf/1806.02643.pdf)
* [How Our NFL Predictions Work]
(https://fivethirtyeight.com/methodology/how-our-nfl-predictions-work/)
"""
)
