title: Find Distance to Decision Boundary in Decision Trees
body: <p>I want to find the distance of samples to the decision boundary of a trained decision trees classifier in <a href="https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier" rel="nofollow noreferrer">scikit-learn</a>. The features are all numeric and the feature space could be of any size. </p>
  
  <p>I have this visualization so far for an example 2D case based on <a href="https://scikit-learn.org/stable/auto_examples/ensemble/plot_voting_decision_regions.html#sphx-glr-auto-examples-ensemble-plot-voting-decision-regions-py" rel="nofollow noreferrer">here</a>:</p>
  
  <pre><code>import numpy as np
  import matplotlib.pyplot as plt
  
  from sklearn.tree import DecisionTreeClassifier
  from sklearn.datasets import make_moons
  
  # Generate some example data
  X, y = make_moons(noise=0.3, random_state=0)
  
  # Train the classifier
  clf = DecisionTreeClassifier(max_depth=2)
  
  clf.fit(X, y)
  
  # Plot
  x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
  y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
  xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.1), np.arange(y_min, y_max, 0.1))
  
  Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
  Z = Z.reshape(xx.shape)
  
  plt.contourf(xx, yy, Z, alpha=0.4)
  plt.scatter(X[:, 0], X[:, 1], c=y, s=20, edgecolor='k')
  plt.xlabel('a'); plt.ylabel('b');
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/f6tTRm.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/f6tTRm.png" alt="enter image description here"></a></p>
  
  <p>I understand that for some other classifiers like SVM, this distance can be mathematically calculated [<a href="https://stackoverflow.com/questions/32074239/sklearn-getting-distance-of-each-point-from-decision-boundary">1</a>, <a href="https://stackoverflow.com/questions/44579826/does-the-decision-function-in-scikit-learn-return-the-true-distance-to-the-hyper?noredirect=1&amp;lq=1">2</a>, <a href="https://stackoverflow.com/questions/28256058/plotting-decision-boundary-of-logistic-regression?noredirect=1&amp;lq=1">3</a>]. The rules learned after training a decision trees define the boundaries and may also be helpful to algorithmically calculate the distances  [<a href="https://stackoverflow.com/questions/51912370/plot-decision-tree-over-dataset-in-scikit-learn">4</a>, <a href="https://stackoverflow.com/questions/43929400/retrieve-decision-boundary-lines-x-y-coordinate-format-from-sklearn-decision-t">5</a>, <a href="https://www.jeremyjordan.me/decision-trees/" rel="nofollow noreferrer">6</a>]: </p>
  
  <pre><code># Plot the trained tree
  from sklearn import tree
  import graphviz 
  dot_data = tree.export_graphviz(clf, feature_names=['a', 'b'],  class_names=['1', '2'], filled=True)  
  graph = graphviz.Source(dot_data)  
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/QKmhZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/QKmhZ.png" alt="enter image description here"></a></p>