Why does GridSearchCV (sklearn) change the value of n_samples? I thought n_samples is the number of training examples. But when using GridSearchCV, n_samples becomes 32 rather than 50.

Error when using GridSearchCV:


  Expected n_neighbors &lt;= n_samples,  but n_samples = 32, n_neighbors =
  50


Training examples:

print(X_train.shape[0]) => 50

print(len(y_train)) => 50

This works:

neigh = KNeighborsClassifier(n_neighbors=50)
neigh.fit(X_train, y_train) 
result = neigh.predict(X_test)


This fails:

from sklearn.model_selection import GridSearchCV

grid_params = { 
    "n_neighbors" : [50]
}

g = GridSearchCV(KNeighborsClassifier(), grid_params)
g.fit(X_train, y_train)


I'm confused why n_samples becomes 32 when using GridSearchCV.
The CV stands for CrossValidation, meaning it will split up your training set in a number of folds (in this case 3), train on n-1 of those folds and test on the remaining one. This is why your training is now done on 32 instead of 50 samples. Crossvalidation is useful for estimating how well your model (including specific hyperparameters) does on unseen data.