 eave one out cv in this strategy for cross validationwe select random single data point from the initial training dataset and that becomes our validation set so we have single point only in our validation set and the rest - observations become our training set this means that if have data points in training set than we will be developing iterations of each model with different training set and validation set each time such that the validation set has one observation and the rest ( go into the training set this may become infeasible if the dataset size is large but in practice the error can be estimated by performing some small number of iterations due to the computational complexity of this measureit is mostly suitable for small datasets and rarely used in practice  -fold cv the other strategy for cross-validation is to split the training dataset into equal subsets out of these subsets we train the model on - subsets and keep one subset as validation set this process is repeated times and the error is averaged over the models that are obtained by developing different iterations of the model we keep changing the validation set in each of these iterations which ensures that in each iterationthe model is trained on different subset of data this practice of cross-validation is quite effective in practiceboth for model selection and hyperparameter optimization natural question for this strategy is to select the appropriate number of foldsas they will both control our error approximation and the computational runtime of our cv process there are mathematical ways to select the most appropriate but in practice good choice of ranges from - soin most caseswe can do -fold or -fold validation and be confident of the results that we obtained hyperparameter tuning strategies based on our discussions until nowwe have all the prerequisites for tuning our model we know what hyperparameters arehow the performance of model can be evaluatedand how we can use crossvalidation to search through the parameter space for optimal value of our algorithm' hyperparameters in this sectionwe discuss two major strategies that tie all this together to determine the most optimal hyperparameters fortunatelythe scikit-learn library has an excellent built-in support for performing hyperparameter search with cross-validation there are two major ways in which we can search our parameter space for an optimal model these two methods differ in the way we will search for themsystemic versus random in this section we will discuss these two methods along with hands-on examples the takeaway from this section is to understand the processes so that you can start leveraging on your own datasets also note that even if we don' mention it explicitlywe will always be using cross validation to perform any of these searches grid search this is the simplest of the hyperparameter optimization methods in this method we will specify the grid of values (of hyperparametersthat we want to try out and optimize to get the best parameter combinations then we will build models on each of those values (combination of multiple parameter values)using cross-validation of courseand report the best parameterscombination in the whole grid the output will be the model using the best combination from the grid although it is quite simpleit suffers from one serious drawback that the user has to manually supply the actual parameterswhich may or may not contain the most optimal parameters