and evaluate it on the test set svm svc(**best_parameterssvm fit(x_trainvaly_trainvaltest_score svm score(x_testy_testprint("best score on validation set{ }format(best_score)print("best parameters"best_parametersprint("test set score with best parameters{ }format(test_score)out[ ]size of training set size of validation set size of test set best score on validation set best parameters{' ' 'gamma' test set score with best parameters the best score on the validation set is %slightly lower than beforeprobably because we used less data to train the model (x_train is smaller now because we split our dataset twicehoweverthe score on the test set--the score that actually tells us how well we generalize--is even lowerat so we can only claim to classify new data correctlynot correctly as we thought beforethe distinction between the training setvalidation setand test set is fundamentally important to applying machine learning methods in practice any choices made based on the test set accuracy "leakinformation from the test set into the model thereforeit is important to keep separate test setwhich is only used for the final evaluation it is good practice to do all exploratory analysis and model selection using the combination of training and validation setand reserve the test set for final evaluation--this is even true for exploratory visualization strictly speakingevaluating more than one model on the test set and choosing the better of the two will result in an overly optimistic estimate of how accurate the model is grid search with cross-validation while the method of splitting the data into traininga validationand test set that we just saw is workableand relatively commonly usedit is quite sensitive to how exactly the data is split from the output of the previous code snippet we can see that gridsearchcv selects ' ' 'gamma' as the best parameterswhile the output of the code in the previous section selects ' ' 'gamma' as the best parameters for better estimate of the generalization performanceinstead of using single split into training and validation setwe can use cross-validation to evaluate the performance of each parameter combination this method can be coded up as followsgrid search