training deep leaning models figure - overfitting and underfitting hyperparameter tuning tuning the hyperparameters of the model (such as learning rate or momentumcan be done manuallyvia grid search (where in grid is defined over small set of values)or via random search (where the values of hyperparameters are drawn at random from distribution defined by the userin grid searchthe practitioner has to create small subset of potential values (since compute resources are finitefor each hyperparameter in the network the training process essentially loops