<p dir="auto">I looked at the issues and did not find something related, so I hope it's not a duplicate.</p>
<h4 dir="auto">Description</h4>
<p dir="auto">I fitted a <code class="notranslate">DecisionTreeClassifier</code>on a single feature and had a look at the <code class="notranslate">threshold</code> attribute of the <code class="notranslate">Tree</code> object. I observed that there were a lot of thresholds equal to -2, though my feature took only positive values. Using this <a href="https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html#sphx-glr-auto-examples-tree-plot-unveil-tree-structure-py" rel="nofollow">example</a> from the documentation, I realized that all leaf nodes had a threshold equal to -2. Since leaf nodes don't have a threshold, I would have expected <code class="notranslate">np.nan</code> for these thresholds.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np
from sklearn.tree import DecisionTreeClassifier


X = np.arange(10)[:, None]
y = [0] * 5 + [1] * 5

clf = DecisionTreeClassifier(criterion='entropy')
clf.fit(X, y)
print(clf.tree_.threshold)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">tree</span> <span class="pl-k">import</span> <span class="pl-v">DecisionTreeClassifier</span>


<span class="pl-v">X</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">arange</span>(<span class="pl-c1">10</span>)[:, <span class="pl-c1">None</span>]
<span class="pl-s1">y</span> <span class="pl-c1">=</span> [<span class="pl-c1">0</span>] <span class="pl-c1">*</span> <span class="pl-c1">5</span> <span class="pl-c1">+</span> [<span class="pl-c1">1</span>] <span class="pl-c1">*</span> <span class="pl-c1">5</span>

<span class="pl-s1">clf</span> <span class="pl-c1">=</span> <span class="pl-v">DecisionTreeClassifier</span>(<span class="pl-s1">criterion</span><span class="pl-c1">=</span><span class="pl-s">'entropy'</span>)
<span class="pl-s1">clf</span>.<span class="pl-en">fit</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-en">print</span>(<span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">threshold</span>)</pre></div>
<h4 dir="auto">Expected Results</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="array([ 4.5, nan , nan])"><pre class="notranslate"><span class="pl-en">array</span>([ <span class="pl-c1">4.5</span>, <span class="pl-s1">nan</span> , <span class="pl-s1">nan</span>])</pre></div>
<h4 dir="auto">Actual Results</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="array([ 4.5, -2. , -2. ])"><pre class="notranslate"><span class="pl-en">array</span>([ <span class="pl-c1">4.5</span>, <span class="pl-c1">-</span><span class="pl-c1">2.</span> , <span class="pl-c1">-</span><span class="pl-c1">2.</span> ])</pre></div>
<h4 dir="auto">Bigger example (using the example from the doc)</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np
from sklearn.tree import DecisionTreeClassifier


rng = np.random.RandomState(42)
X = np.abs(rng.randn(1000, 1)) # All the values of X are non-negative
y = rng.randint(2, size=1000)

clf = DecisionTreeClassifier(criterion='entropy')
clf.fit(X, y)
clf.tree_.threshold

n_nodes = clf.tree_.node_count
children_left = clf.tree_.children_left
children_right = clf.tree_.children_right
feature = clf.tree_.feature
threshold = clf.tree_.threshold

node_depth = np.zeros(shape=n_nodes, dtype=np.int64)
is_leaves = np.zeros(shape=n_nodes, dtype=bool)
stack = [(0, -1)]  # seed is the root node id and its parent depth
while len(stack) &gt; 0:
    node_id, parent_depth = stack.pop()
    node_depth[node_id] = parent_depth + 1

    if (children_left[node_id] != children_right[node_id]):
        stack.append((children_left[node_id], parent_depth + 1))
        stack.append((children_right[node_id], parent_depth + 1))
    else:
        is_leaves[node_id] = True

print(&quot;Number of leaf nodes: {0}&quot;.format(is_leaves.sum()))
print(&quot;Number of thresholds equal to -2: {0}&quot;.format(
    (clf.tree_.threshold == -2).sum()))
print(&quot;All leaf nodes have a threshold equal to -2: {0}&quot;.format(
    np.all(is_leaves == (clf.tree_.threshold == -2)))
)

Number of leaf nodes: 505
Number of thresholds equal to -2: 505
All leaf nodes have a threshold equal to -2: True"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">tree</span> <span class="pl-k">import</span> <span class="pl-v">DecisionTreeClassifier</span>


<span class="pl-s1">rng</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-v">RandomState</span>(<span class="pl-c1">42</span>)
<span class="pl-v">X</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">abs</span>(<span class="pl-s1">rng</span>.<span class="pl-en">randn</span>(<span class="pl-c1">1000</span>, <span class="pl-c1">1</span>)) <span class="pl-c"># All the values of X are non-negative</span>
<span class="pl-s1">y</span> <span class="pl-c1">=</span> <span class="pl-s1">rng</span>.<span class="pl-en">randint</span>(<span class="pl-c1">2</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span><span class="pl-c1">1000</span>)

<span class="pl-s1">clf</span> <span class="pl-c1">=</span> <span class="pl-v">DecisionTreeClassifier</span>(<span class="pl-s1">criterion</span><span class="pl-c1">=</span><span class="pl-s">'entropy'</span>)
<span class="pl-s1">clf</span>.<span class="pl-en">fit</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">threshold</span>

<span class="pl-s1">n_nodes</span> <span class="pl-c1">=</span> <span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">node_count</span>
<span class="pl-s1">children_left</span> <span class="pl-c1">=</span> <span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">children_left</span>
<span class="pl-s1">children_right</span> <span class="pl-c1">=</span> <span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">children_right</span>
<span class="pl-s1">feature</span> <span class="pl-c1">=</span> <span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">feature</span>
<span class="pl-s1">threshold</span> <span class="pl-c1">=</span> <span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">threshold</span>

<span class="pl-s1">node_depth</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">zeros</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span><span class="pl-s1">n_nodes</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">np</span>.<span class="pl-s1">int64</span>)
<span class="pl-s1">is_leaves</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">zeros</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span><span class="pl-s1">n_nodes</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">bool</span>)
<span class="pl-s1">stack</span> <span class="pl-c1">=</span> [(<span class="pl-c1">0</span>, <span class="pl-c1">-</span><span class="pl-c1">1</span>)]  <span class="pl-c"># seed is the root node id and its parent depth</span>
<span class="pl-k">while</span> <span class="pl-en">len</span>(<span class="pl-s1">stack</span>) <span class="pl-c1">&gt;</span> <span class="pl-c1">0</span>:
    <span class="pl-s1">node_id</span>, <span class="pl-s1">parent_depth</span> <span class="pl-c1">=</span> <span class="pl-s1">stack</span>.<span class="pl-en">pop</span>()
    <span class="pl-s1">node_depth</span>[<span class="pl-s1">node_id</span>] <span class="pl-c1">=</span> <span class="pl-s1">parent_depth</span> <span class="pl-c1">+</span> <span class="pl-c1">1</span>

    <span class="pl-k">if</span> (<span class="pl-s1">children_left</span>[<span class="pl-s1">node_id</span>] <span class="pl-c1">!=</span> <span class="pl-s1">children_right</span>[<span class="pl-s1">node_id</span>]):
        <span class="pl-s1">stack</span>.<span class="pl-en">append</span>((<span class="pl-s1">children_left</span>[<span class="pl-s1">node_id</span>], <span class="pl-s1">parent_depth</span> <span class="pl-c1">+</span> <span class="pl-c1">1</span>))
        <span class="pl-s1">stack</span>.<span class="pl-en">append</span>((<span class="pl-s1">children_right</span>[<span class="pl-s1">node_id</span>], <span class="pl-s1">parent_depth</span> <span class="pl-c1">+</span> <span class="pl-c1">1</span>))
    <span class="pl-k">else</span>:
        <span class="pl-s1">is_leaves</span>[<span class="pl-s1">node_id</span>] <span class="pl-c1">=</span> <span class="pl-c1">True</span>

<span class="pl-en">print</span>(<span class="pl-s">"Number of leaf nodes: {0}"</span>.<span class="pl-en">format</span>(<span class="pl-s1">is_leaves</span>.<span class="pl-en">sum</span>()))
<span class="pl-en">print</span>(<span class="pl-s">"Number of thresholds equal to -2: {0}"</span>.<span class="pl-en">format</span>(
    (<span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">threshold</span> <span class="pl-c1">==</span> <span class="pl-c1">-</span><span class="pl-c1">2</span>).<span class="pl-en">sum</span>()))
<span class="pl-en">print</span>(<span class="pl-s">"All leaf nodes have a threshold equal to -2: {0}"</span>.<span class="pl-en">format</span>(
    <span class="pl-s1">np</span>.<span class="pl-en">all</span>(<span class="pl-s1">is_leaves</span> <span class="pl-c1">==</span> (<span class="pl-s1">clf</span>.<span class="pl-s1">tree_</span>.<span class="pl-s1">threshold</span> <span class="pl-c1">==</span> <span class="pl-c1">-</span><span class="pl-c1">2</span>)))
)

<span class="pl-v">Number</span> <span class="pl-s1">of</span> <span class="pl-s1">leaf</span> <span class="pl-s1">nodes</span>: <span class="pl-c1">505</span>
<span class="pl-v">Number</span> <span class="pl-s1">of</span> <span class="pl-s1">thresholds</span> <span class="pl-s1">equal</span> <span class="pl-s1">to</span> <span class="pl-c1">-</span><span class="pl-c1">2</span>: <span class="pl-c1">505</span>
<span class="pl-v">All</span> <span class="pl-s1">leaf</span> <span class="pl-s1">nodes</span> <span class="pl-s1">have</span> <span class="pl-s1">a</span> <span class="pl-s1">threshold</span> <span class="pl-s1">equal</span> <span class="pl-s1">to</span> <span class="pl-c1">-</span><span class="pl-c1">2</span>: <span class="pl-c1">True</span></pre></div>
<h4 dir="auto">Versions</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="System:
    python: 3.7.1 (default, Dec 14 2018, 13:28:58)  [Clang 4.0.1 (tags/RELEASE_401/final)]
executable: /Users/johann.faouzi/anaconda3/bin/python
   machine: Darwin-16.7.0-x86_64-i386-64bit

BLAS:
    macros: SCIPY_MKL_H=None, HAVE_CBLAS=None
  lib_dirs: /Users/johann.faouzi/anaconda3/lib
cblas_libs: mkl_rt, pthread

Python deps:
       pip: 18.1
setuptools: 40.6.3
   sklearn: 0.20.1
     numpy: 1.15.4
     scipy: 1.1.0
    Cython: 0.29.2
    pandas: 0.23.4"><pre class="notranslate"><span class="pl-v">System</span>:
    <span class="pl-s1">python</span>: <span class="pl-c1">3.7</span>.<span class="pl-c1">1</span> (<span class="pl-s1">default</span>, <span class="pl-v">Dec</span> <span class="pl-c1">14</span> <span class="pl-c1">2018</span>, <span class="pl-c1">13</span>:<span class="pl-c1">28</span>:<span class="pl-c1">58</span>)  [<span class="pl-v">Clang</span> <span class="pl-c1">4.0</span><span class="pl-c1">.1</span> (<span class="pl-s1">tags</span><span class="pl-c1">/</span><span class="pl-v">RELEASE_401</span><span class="pl-c1">/</span><span class="pl-s1">final</span>)]
<span class="pl-s1">executable</span>: <span class="pl-c1">/</span><span class="pl-v">Users</span><span class="pl-c1">/</span><span class="pl-s1">johann</span>.<span class="pl-s1">faouzi</span><span class="pl-c1">/</span><span class="pl-s1">anaconda3</span><span class="pl-c1">/</span><span class="pl-s1">bin</span><span class="pl-c1">/</span><span class="pl-s1">python</span>
   <span class="pl-s1">machine</span>: <span class="pl-v">Darwin</span><span class="pl-c1">-</span><span class="pl-c1">16.7</span>.<span class="pl-c1">0</span><span class="pl-c1">-</span><span class="pl-s1">x86_64</span><span class="pl-c1">-</span><span class="pl-s1">i386</span><span class="pl-c1">-</span><span class="pl-c1">64</span><span class="pl-s1">bit</span>

<span class="pl-v">BLAS</span>:
    <span class="pl-s1">macros</span>: <span class="pl-v">SCIPY_MKL_H</span><span class="pl-c1">=</span><span class="pl-c1">None</span>, <span class="pl-v">HAVE_CBLAS</span><span class="pl-c1">=</span><span class="pl-c1">None</span>
  <span class="pl-s1">lib_dirs</span>: <span class="pl-c1">/</span><span class="pl-v">Users</span><span class="pl-c1">/</span><span class="pl-s1">johann</span>.<span class="pl-s1">faouzi</span><span class="pl-c1">/</span><span class="pl-s1">anaconda3</span><span class="pl-c1">/</span><span class="pl-s1">lib</span>
<span class="pl-s1">cblas_libs</span>: <span class="pl-s1">mkl_rt</span>, <span class="pl-s1">pthread</span>

<span class="pl-v">Python</span> <span class="pl-s1">deps</span>:
       <span class="pl-s1">pip</span>: <span class="pl-c1">18.1</span>
<span class="pl-s1">setuptools</span>: <span class="pl-c1">40.6</span>.<span class="pl-c1">3</span>
   <span class="pl-s1">sklearn</span>: <span class="pl-c1">0.20</span>.<span class="pl-c1">1</span>
     <span class="pl-s1">numpy</span>: <span class="pl-c1">1.15</span>.<span class="pl-c1">4</span>
     <span class="pl-s1">scipy</span>: <span class="pl-c1">1.1</span>.<span class="pl-c1">0</span>
    <span class="pl-v">Cython</span>: <span class="pl-c1">0.29</span>.<span class="pl-c1">2</span>
    <span class="pl-s1">pandas</span>: <span class="pl-c1">0.23</span>.<span class="pl-c1">4</span></pre></div>
<p dir="auto">A while ago we talked about the possibility of a version 2 with breaking changes. Specifically, this came up in the context of SLEP006: metadata routing, but there are other things we have wanted to break which we can do all in a single release. In this issue we can talk about the possibility of such a release, and maybe a few things we'd like to include in it.</p>
<p dir="auto">I'll try to summarize the challenges of keeping backward compatibility (BC) in the context of SLEP006:</p>
<ul dir="auto">
<li><code class="notranslate">MetaEstimator(Estimator()).fit(X, y, sample_weight)</code> will raise according to SLEP006 since <code class="notranslate">sample_weight</code> is not requested by any object. For backward compatibility, during the transition period, it'll only warn and assume <code class="notranslate">sample_weight</code> is requested by <code class="notranslate">Estimator()</code>.</li>
<li>What should the behavior of <code class="notranslate">GridSearchCV(Estimator(), scorer=scorer).fit(X, y, sample_weight)</code> be during the transition period? If we keep backward compatibility, it should warn (whatever the warning be), route <code class="notranslate">sample_weight</code> to <code class="notranslate">Estimator</code>, but not to the <code class="notranslate">scorer</code>. And that's only the case because we're keeping BC. From the user's perspective, it's very weird.</li>
<li><code class="notranslate">Pipeline</code> ATM in certain methods like <code class="notranslate">fit</code>, <code class="notranslate">fit_transform</code>, <code class="notranslate">predict</code>, routes <code class="notranslate">*_params</code> to the <code class="notranslate">fit/etc</code> of the last estimator only. And <code class="notranslate">decision_function</code> and <code class="notranslate">transform</code>, <code class="notranslate">score_samples</code> have no routing at all. Keeping this BC, proves to be challenging, and a bit nasty. ref: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1351889841" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/24270" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/24270/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/24270">#24270</a></li>
<li>In quite a few meta-estimators, we check if the sub-estimator has <code class="notranslate">sample_weight</code> in the signature of <code class="notranslate">fit</code>, and we pass the given sample weight to it if that's the case, and not if that's not the case. With routing, we would have a much better idea of when to pass sample weight and when not, but if we're keeping BC, it's really challenging to see if we should forward sample weights or not. Both AdaBoost (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1319604557" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/24026" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/24026/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/24026">#24026</a>) and Bagging (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1349525104" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/24250" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/24250/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/24250">#24250</a>) have been very challenging, to the point that we might end up having to check if a sub-estimator is a meta-estimator itself or not, and check signature of sub-estimator's <code class="notranslate">fit</code>, and check the given routing info to keep BC.</li>
</ul>
<p dir="auto">I'm pretty sure there are other issues we haven't foreseen, and they'll come up as we work on this project. We thought it'd be easy once we get to this point, but BC has proven to make it very tricky and has stalled the progress.</p>
<p dir="auto">All of the above is adding substantial complexity which is probably error prone, and buggy. We have had to spend quite a bit of time every time we go back to review those bits to understand why we've done what we've done. Even as a temporary solution they're not very maintainable.</p>
<p dir="auto">My proposal here is to have a version 2 which is not BC with previous releases in certain ways, and target a few breaking changes for it. We can aim to have the release in October 2023 or April/May 2024 as the version 2.</p>
<p dir="auto">cc <a class="team-mention js-team-mention notranslate" data-error-text="Failed to load team members" data-id="11523" data-permission-text="Team members are private" data-url="/orgs/scikit-learn/teams/core-devs/members" data-hovercard-type="team" data-hovercard-url="/orgs/scikit-learn/teams/core-devs/hovercard" href="https://github.com/orgs/scikit-learn/teams/core-devs">@scikit-learn/core-devs</a></p>