# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="Tce3stUlHN0L"
# ##### Copyright 2020 The TensorFlow Authors.

# + cellView="form" id="tuOe1ymfHZPu"
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# + [markdown] id="8yo62ffS5TF5"
# # Inspect and debug decision forest models
#
# <table class="tfo-notebook-buttons" align="left">
#   <td>
#     <a target="_blank" href="https://www.tensorflow.org/decision_forests/tutorials/advanced_colab"><img src="https://www.tensorflow.org/images/tf_logo_32px.png" />View on TensorFlow.org</a>
#   </td>
#   <td>
#     <a target="_blank" href="https://colab.research.google.com/github/tensorflow/decision-forests/blob/main/documentation/tutorials/advanced_colab.ipynb"><img src="https://www.tensorflow.org/images/colab_logo_32px.png" />Run in Google Colab</a>
#   </td>
#   <td>
#     <a target="_blank" href="https://github.com/tensorflow/decision-forests/blob/main/documentation/tutorials/advanced_colab.ipynb"><img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />View on GitHub</a>
#   </td>
#   <td>
#     <a href="https://storage.googleapis.com/tensorflow_docs/decision-forests/documentation/tutorials/advanced_colab.ipynb"><img src="https://www.tensorflow.org/images/download_logo_32px.png" />Download notebook</a>
#   </td>
# </table>
#

# + [markdown] id="84wIz8LPiLDF"
# In this colab, you will learn how to inspect and create the structure of a model directly. We assume you are familiar with the concepts introduced in the
# [beginner](beginner_colab.ipynb) and [intermediate](intermediate_colab.ipynb)
# colabs.
#
# In this colab, you will:
#
# 1.  Train a Random Forest model and access its structure programatically.
#
# 1.  Create a Random Forest model by hand and use it as a classical model.

# + [markdown] id="Rzskapxq7gdo"
# ## Setup

# + id="mZiInVYfffAb"
# Install TensorFlow Dececision Forests.
# !pip install tensorflow_decision_forests

# Use wurlitzer to capture training logs.
# !pip install wurlitzer

# + id="RsCV2oAS7gC_"
import tensorflow_decision_forests as tfdf

import os
import numpy as np
import pandas as pd
import tensorflow as tf
import math
import collections

try:
  from wurlitzer import sys_pipes
except:
  from colabtools.googlelog import CaptureLog as sys_pipes

from IPython.core.magic import register_line_magic
from IPython.display import Javascript


# + [markdown] id="xV3klWJnyCgH"
# The hidden code cell limits the output height in colab.

# + cellView="form" id="XAWSjWrQmVE0"
#@title

# Some of the model training logs can cover the full
# screen if not compressed to a smaller viewport.
# This magic allows setting a max height for a cell.
@register_line_magic
def set_cell_height(size):
  display(
      Javascript("google.colab.output.setIframeHeight(0, true, {maxHeight: " +
                 str(size) + "})"))


# + [markdown] id="M_D4Ft4o65XT"
# ## Train a simple Random Forest
#
# We train a Random Forest like in the [beginner colab](beginner_colab.ipynb):

# + id="tTW2aBiVcU3E"
# Download the dataset
# !wget -q https://storage.googleapis.com/download.tensorflow.org/data/palmer_penguins/penguins.csv -O /tmp/penguins.csv

# Load a dataset into a Pandas Dataframe.
dataset_df = pd.read_csv("/tmp/penguins.csv")

# Show the first three examples.
print(dataset_df.head(3))

# Convert the pandas dataframe into a tf dataset.
dataset_tf = tfdf.keras.pd_dataframe_to_tf_dataset(dataset_df, label="species")

# Train the Random Forest
model = tfdf.keras.RandomForestModel(compute_oob_variable_importances=True)
model.fit(x=dataset_tf)

# + [markdown] id="b7Xie0bhcw8_"
# Note the `compute_oob_variable_importances=True`
# hyper-parameter in the model constructor. This opion computes the Out-of-bag (OOB)
# variable importance during training. This is a popular
# [permutation variable importance](https://christophm.github.io/interpretable-ml-book/feature-importance.html) for Random Forest models.
#
# Computing the OOB Variable importance does not impact the final model, it will slow the training on large datasets.
#
# Check the model summary:

# + id="fsQYD-jFc2EH"
# %set_cell_height 300

model.summary()

# + [markdown] id="dtvAH26EfSgY"
# Note the multiple variable importances with name `MEAN_DECREASE_IN_*`.

# + [markdown] id="xTwmx8A0c4TU"
# ## Plotting the model
#
# Next, plot the model.
#
# A Random Forest is a large model (this model has 300 trees and ~5k nodes; see the summary above). Therefore, only plot the first tree, and limit the nodes to depth 3.

# + id="ZRTrXDz_dIAQ"
tfdf.model_plotter.plot_model_in_colab(model, tree_idx=0, max_depth=3)

# + [markdown] id="lOlieoz2c-GA"
# ## Inspect the model structure
#
# The model structure and meta-data is
# available through the **inspector** created by `make_inspector()`.
#
# **Note:** Depending on the learning algorithm and hyper-parameters, the
# inspector will expose different specialized attributes. For examples, the
# `winner_take_all` field is specific to Random Forest models.

# + id="KHc8IcW1c8ER"
inspector = model.make_inspector()

# + [markdown] id="RDdUhqaSsNnQ"
# For our model, the available inspector fields are:

# + id="jx54DFjRsA7k"
[field for field in dir(inspector) if not field.startswith("_")]

# + [markdown] id="_QJFITMQsgtK"
# Remember to see [the API-reference](https://tensorflow.org/decision_forests/api_docs/python/tfdf/inspector/AbstractInspector) or use `?` for the builtin documentation.

# + id="YCGkpRkssdCb"
# ?inspector.model_type

# + [markdown] id="nd-fOgmjd1oK"
# Some of the model meta-data:

# + id="Iu_To_z9d35G"
print("Model type:", inspector.model_type())
print("Number of trees:", inspector.num_trees())
print("Objective:", inspector.objective())
print("Input features:", inspector.features())

# + [markdown] id="Zs7b8EBud9JM"
# `evaluate()` is the evaluation of the model computed during training. The dataset used for this evaluation depends on the algorithm. For example, it can be the validation dataset or the out-of-bag-dataset .
#
# **Note:** While computed during training, `evaluate()` is never an evaluation on the
# training dataset.

# + id="uVN-j0E4Q1T3"
inspector.evaluation()

# + [markdown] id="2r6Yrjb7f5KH"
# The variable importances are:

# + id="qoqhhmGjf7ED"
print(f"Available variable importances:")
for importance in inspector.variable_importances().keys():
  print("\t", importance)

# + [markdown] id="8QUW8w-UmCoW"
# Different variable importances have different semantics. For example, a feature
# with a **mean decrease in auc** of `0.05` means that removing this feature from
# the training dataset would reduce/hurt the AUC by 5%.

# + id="OoSG5T8ShSdG"
# Mean decrease in AUC of the class 1 vs the others.
inspector.variable_importances()["MEAN_DECREASE_IN_AUC_1_VS_OTHERS"]

# + [markdown] id="q_-kLTNjhaQo"
# Finaly, access the actual tree structure:

# + id="l4N_heuzhcUS"
inspector.extract_tree(tree_idx=0)

# + [markdown] id="B8u_0p80hoeP"
# Extracting a tree is not efficient. If speed is important, the model inspection can be done with the `iterate_on_nodes()` method instead. This method is a Depth First Pre-order traversals iterator on all the nodes of the model.
#
# **Note:** `extract_tree()` is implemented using `iterate_on_nodes()`.
#
# For following example computes how many times each feature is used (this is a
# kind of structural variable importance):

# + id="OUEpes34iHg8"
# number_of_use[F] will be the number of node using feature F in its condition.
number_of_use = collections.defaultdict(lambda: 0)

# Iterate over all the nodes in a Depth First Pre-order traversals.
for node_iter in inspector.iterate_on_nodes():

  if not isinstance(node_iter.node, tfdf.py_tree.node.NonLeafNode):
    # Skip the leaf nodes
    continue

  # Iterate over all the features used in the condition.
  # By default, models are "oblique" i.e. each node tests a single feature.
  for feature in node_iter.node.condition.features():
    number_of_use[feature] += 1

print("Number of condition nodes per features:")
for feature, count in number_of_use.items():
  print("\t", feature.name, ":", count)

# + [markdown] id="CD39OmGbnPww"
# ## Creating a model by hand
#
# In this section you will create a small Random Forest model by hand. To make it
# extra easy, the model will only contain one simple tree:
#
# ```
# 3 label classes: Red, blue and green.
# 2 features: f1 (numerical) and f2 (string categorical)
#
# f1>=1.5
#     ├─(pos)─ f2 in ["cat","dog"]
#     │         ├─(pos)─ value: [0.8, 0.1, 0.1]
#     │         └─(neg)─ value: [0.1, 0.8, 0.1]
#     └─(neg)─ value: [0.1, 0.1, 0.8]
# ```

# + id="fGGe5IxdnuEa"
# Create the model builder
builder = tfdf.builder.RandomForestBuilder(
    path="/tmp/manual_model",
    objective=tfdf.py_tree.objective.ClassificationObjective(
        label="color", classes=["red", "blue", "green"]))

# + [markdown] id="DRnJ2u-Moqbf"
# Each tree is added one by one.
#
# **Note:** The tree object (`tfdf.py_tree.tree.Tree`) is the same as the one returned by `extract_tree()` in the previous section.

# + id="cmAddPhAo0tG"
# So alias
Tree = tfdf.py_tree.tree.Tree
SimpleColumnSpec = tfdf.py_tree.dataspec.SimpleColumnSpec
ColumnType = tfdf.py_tree.dataspec.ColumnType
# Nodes
NonLeafNode = tfdf.py_tree.node.NonLeafNode
LeafNode = tfdf.py_tree.node.LeafNode
# Conditions
NumericalHigherThanCondition = tfdf.py_tree.condition.NumericalHigherThanCondition
CategoricalIsInCondition = tfdf.py_tree.condition.CategoricalIsInCondition
# Leaf values
ProbabilityValue = tfdf.py_tree.value.ProbabilityValue

builder.add_tree(
    Tree(
        NonLeafNode(
            condition=NumericalHigherThanCondition(
                feature=SimpleColumnSpec(name="f1", type=ColumnType.NUMERICAL),
                threshold=1.5,
                missing_evaluation=False),
            pos_child=NonLeafNode(
                condition=CategoricalIsInCondition(
                    feature=SimpleColumnSpec(name="f2",type=ColumnType.CATEGORICAL),
                    mask=["cat", "dog"],
                    missing_evaluation=False),
                pos_child=LeafNode(value=ProbabilityValue(probability=[0.8, 0.1, 0.1], num_examples=10)),
                neg_child=LeafNode(value=ProbabilityValue(probability=[0.1, 0.8, 0.1], num_examples=20))),
            neg_child=LeafNode(value=ProbabilityValue(probability=[0.1, 0.1, 0.8], num_examples=30)))))

# + [markdown] id="DjWdgRNNqEAD"
# Conclude the tree writing

# + id="cJqn4khxqH6t"
builder.close()

# + [markdown] id="_oxxXAn7qK-z"
# Now you can open the model as a regular keras model, and make predictions:

# + id="<KEY>"
manual_model = tf.keras.models.load_model("/tmp/manual_model")

# + id="qlC4N-LuqWWR"
examples = tf.data.Dataset.from_tensor_slices({
        "f1": [1.0, 2.0, 3.0],
        "f2": ["cat", "cat", "bird"]
    }).batch(2)

predictions = manual_model.predict(examples)

print("predictions:\n",predictions)

# + [markdown] id="mxJyp1mKFPXb"
# Access the structure:
#
# **Note:** Because the model is serialized-and-deserialized, you need to use an alternative but equivalent form.

# + id="IjcyMHJUFO_B"
yggdrasil_model_path = manual_model.yggdrasil_model_path_tensor().numpy().decode("utf-8")
print("yggdrasil_model_path:",yggdrasil_model_path)

inspector = tfdf.inspector.make_inspector(yggdrasil_model_path)
print("Input features:", inspector.features())

# + [markdown] id="muW1hgmotx8J"
# And of course, you can plot this manually constructed model: 

# + id="bqahDVg3t1xM"
tfdf.model_plotter.plot_model_in_colab(manual_model)

StarcoderdataJupyterScriptsDedupFiltered