title: Why does a column of 1s impact the results of a decision tree classifier?
body: <p>I was testing sklearn's <code>Pipeline</code> on a randomly generated classification problem:</p>
  <pre><code>import numpy as np
  import pandas as pd
  
  from sklearn.tree import DecisionTreeClassifier, plot_tree
  from sklearn.preprocessing import StandardScaler, PolynomialFeatures
  from sklearn.datasets import make_classification
  from sklearn.model_selection import train_test_split
  from sklearn.pipeline import Pipeline
  from sklearn.metrics import accuracy_score
  
  x, y = make_classification(n_samples=100, n_features=5, random_state=10)
  
  x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=.2, random_state=0)
  
  model = DecisionTreeClassifier(random_state=0)
  pipe = Pipeline(steps=[('scale', StandardScaler()),
                         ('poly', PolynomialFeatures(degree=2, include_bias=False)),
                         ('model', model)])
  pipe.fit(x_train, y_train)
  
  pipe_pred = pipe.predict(x_test)
  accuracy_score(y_test, pipe_pred)
  </code></pre>
  <p>This results in an accuracy score of <code>.85</code>. However, when I change the <code>PolynomialFeatures</code> argument <code>include_bias</code> to <code>True</code>, which just inserts a single column of 1s into the array, the accuracy score becomes <code>.90</code>. For visualization, below I have plotted the individual trees for the results when bias is <code>True</code> and when <code>False</code>:</p>
  <h3>When <code>include_bias=True</code>: <a href="https://i.stack.imgur.com/BYVUV.png" rel="nofollow noreferrer">True</a></h3>
  <h3>When <code>include_bias=False</code>: <a href="https://i.stack.imgur.com/vsqmd.png" rel="nofollow noreferrer">False</a></h3>
  <p>These images were generated by <code>plot_tree(pipe['model'])</code>.</p>
  <p>The datasets are the same except when <code>include_bias=True</code> an additional column of 1s is inserted into column 0. So the column indexes for the <code>include_bias=True</code> data correspond to the <code>i + 1</code> column index in the <code>include_bias=False</code> data. (e.g. <code>with_bias[:, 5] == without_bias[:, 4]</code>)</p>
  <p>Based on my understanding, the column of 1s shouldn't have an impact on the Decision Tree. What am I missing?</p>