Cross validation Vs. Train Validate Test
If k-fold cross-validation is used to optimize the model parameters, the training set is split into k parts. Training happens k times, each time leaving out a different part of the training set. Typically, the error of these k-models is averaged. This is done for each of the model parameters to be tested, and the model with the lowest error is chosen. The test set has not been used so far.

Only at the very end the test set is used to test the performance of the (optimized) model.

# example: k-fold cross validation for hyperparameter optimization (k=3)

original data split into training and test set:

|---------------- train ---------------------|         |--- test ---|

cross-validation: test set is not used, error is calculated from
validation set (k-times) and averaged:

|---- train ------------------|- validation -|         |--- test ---|
|---- train ---|- validation -|---- train ---|         |--- test ---|
|- validation -|----------- train -----------|         |--- test ---|

final measure of model performance: model is trained on all training data
and the error is calculated from test set:

|---------------- train ---------------------|--- test ---|


In some cases, k-fold cross-validation is used on the entire data set if no parameter optimization is needed (this is rare, but it happens). In this case there would not be a validation set and the k parts are used as a test set one by one. The error of each of these k tests is typically averaged.

# example: k-fold cross validation

|----- test -----|------------ train --------------|
|----- train ----|----- test -----|----- train ----|
|------------ train --------------|----- test -----|