title: Cross-validation and scores
body: <p>I'm using training data set (i.e., X_train, y_train) when tuning the hyperparameters of my model. I need to use the test data set (i.e., X_test, y_test) as a final check, to make sure my model isn't biased.
  I wrote</p>
  <pre><code>folds = 4
  
  X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=(1/folds), random_state=38, stratify=y)
  
  clf_logreg = Pipeline(steps=[('preprocessor', preprocessing),
                        ('model', LogisticRegression(solver='lbfgs', max_iter=100))])
  
  
  cv = KFold(n_splits=(folds - 1))
  scores_logreg = cross_val_score(clf_logreg, X_train, y_train, cv = cv)
  </code></pre>
  <p>and, to get f1-score,</p>
  <pre><code>cross_val_score(clf_logreg, X_train, y_train, scoring=make_scorer(f1_score, average='weighted'),
      cv=cv)
  </code></pre>
  <p>This returns</p>
  <p>scores_logreg: <code>[0.94422311, 0.99335548, 0.97209302]</code>
  and for f1: <code>[0.97201365, 0.9926906 , 0.98925453]</code></p>
  <p>For checking the test, is it right to write</p>
  <pre><code>cross_val_score(clf_logreg, X_test, y_test, scoring=make_scorer(f1_score, average='weighted'), cv=cv) # not sure if it is ok to let cv
  </code></pre>
  <p>or maybe</p>
  <pre><code>predicted_logreg= clf_logreg.predict(X_test)
  f1 = f1_score(y_test, predicted_logreg)
  </code></pre>
  <p>Value returned are different.</p>