fundamentals of machine learning if anything about the model has been tuned based on test set performancethen your measure of generalization will be flawed splitting your data into trainingvalidationand test sets may seem straightforwardbut there are few advanced ways to do it that can come in handy when little data is available let' review three classic evaluation recipessimple hold-out validationkfold validationand iterated -fold validation with shuffling simple hold-out validation set apart some fraction of your data as your test set train on the remaining dataand evaluate on the test set as you saw in the previous sectionsin order to prevent information leaksyou shouldn' tune your model based on the test setand therefore you should also reserve validation set schematicallyhold-out validation looks like figure the following listing shows simple implementation total available labeled data training set train on this listing held-out validation set figure simple holdout validation split evaluate on this hold-out validat ion num_validation_samples np random shuffle(datashuffling the data is usually appropriate validation_data data[:num_validation_samplesdata data[num_validation_samples:defines the validation set defines the training set training_data data[:model get_model(model train(training_datavalidation_score model evaluate(validation_datatrains model on the training dataand evaluates it on the validation data at this point you can tune your modelretrain itevaluate ittune it again model get_model(model train(np concatenate([training_datavalidation_data])test_score model evaluate(test_datalicensed to once you've tuned your hyperparametersit' common to train your final model from scratch on all non-test data available