title: Bootstrapping with logistic regression in Python - constructing test vector
body: <p>I am attempting to use bootstrapping to get an average estimate of the MSE of my model. I am attempting to follow this guide, for context: <a href="https://machinelearningmastery.com/a-gentle-introduction-to-the-bootstrap-method/" rel="nofollow noreferrer">https://machinelearningmastery.com/a-gentle-introduction-to-the-bootstrap-method/</a> Here is my code:</p>
  <pre><code>models_mse = [None]*100
  for i in range(100):
    boot = resample(X, replace = True, n_samples = math.floor(len(X) / 2),random_state = 1)
    y_new = [y for y in X if y not in boot]
    X_train, X_test, y_train, y_test = train_test_split(boot, y_new, test_size=0.1, random_state=42)
    reg = LogisticRegression()
    reg.fit(X_train, y_train)
    MSE = mean_squared_error(y_test, reg.predict(X_test))
    models_mse.append(MSE)
  </code></pre>
  <p>this produces the error:</p>
  <pre><code>ValueError: Found input variables with inconsistent numbers of samples: [497, 0]
  </code></pre>
  <p>Meaning, <code>y_new</code> is empty. From my understanding of bootstrapping, we treat our sample as the &quot;population&quot; and we resample with replacement on this pseudopopulation to generate more samples to estimate parameters of the pseudopopulation, namely in my case MSE. Here, I have some encoded data <code>X</code> using onehot. It looks like this:</p>
  <pre><code>array([[1., 0., 0., ..., 0., 0., 1.],
         [1., 0., 1., ..., 1., 0., 0.],
         [1., 0., 1., ..., 0., 0., 1.],
         ...,
         [0., 1., 0., ..., 0., 0., 1.],
         [0., 1., 0., ..., 1., 0., 0.],
         [0., 1., 0., ..., 1., 0., 0.]])
  </code></pre>
  <p>and is a numpy array:</p>
  <pre><code>&gt;&gt;&gt;type(X)
  numpy.ndarray
  </code></pre>
  <p>My question is the following: why is <code>y_new</code> empty when I have observed differences between <code>boot</code> and <code>X</code>? Furthermore, I have arbitrarily decided to have half the samples of the original sample. Is there a more systematic way to choose the number of samples when bootstrapping? And lastly, is my setup in general correct for the problem I am trying to solve?</p>
  <p>Thanks.</p>
  <p>Edit:
  After some revisions from user comments, my new code is:</p>
  <pre><code>models_mse = [None]*100
  for i in range(100):
    boot = resample(X_train, replace = True, n_samples = X_train.shape[0],random_state = 1)
    reg = LogisticRegression()
    reg.fit(boot, y_train)
    MSE = mean_squared_error(y_test, reg.predict(X_test))
    models_mse.append(MSE)
  </code></pre>
  <p>which does not throw any errors. However, the MSE is exactly the same for each model, which is strange as shouldn't <code>boot</code> be different for each iteration?</p>