How does the validation set get used in the training phase? I am confused about how the validation set is used during the training phase (neural network like CNN)?
In a platform like Matlab or python(Keras), I split my dataset into train set, validation set and test set.
I knew that validation set is used to tune hyperparameters(like the number of neurons and learning rate), suppose SDG optimizer is used, how the tuning hyperparameters happened based on validation set?
Does the validation set just give an indicator of how the neural network performs on unseen data(validation set) then based on that I manually set hyperparameters? or something automatically(the optimizer) tune hyperparameters?
The validation set is used just to give an indication about the network's performance. The hyperparameters are tuned based on the training data itself.
Based on the optimizer, the neural network calculates the loss for a training sample. Based on the computed loss, the weights of the network are updated. This is repeated for every sample in the training set.
The validation set just uses the latest weights of the network on unseen data to check the model's performance.