Hyper parameters and ValidationSet
Not completely true. In validation set, we find the best hyperparameters, but not with the same parameters of the model. That is, for every value of the hyperparameters we run the training process and obtain the loss for that set of hyperparameters, and we select the hyperparameters values with the lowest validation loss.

Edit

Ridge regression example: Ridge regression depends on one parameter, $\lambda$, that penalizes your model weights. Ridge regression has a single hyperparameter. As ridge regression is a linear model, it has also some weights $\theta_0, \dots, \theta_n$ (parameters) to train. The way to proceed to choose $\lambda$ and $\theta_0, \dots, \theta_n$ is the following:


For each value of $\lambda$ in a grid, we train by gradient descent the ridge regression model (thus optimizing the training loss) and obtain a set of parameters $\theta_0^*, \dots, \theta_n^*$. We measure the loss of the trained ridge model in the validation set.
We choose the $\lambda$ that minimizmes the loss in the validation set and the parameters $\theta_0^*, \dots, \theta_n^*$. If we wish, and we don't have much data, we can retrain the ridge model with the optimal $\lambda$ in the concatenation of the training and validation sets, obtaining some parameters $\hat\theta_0, \dots, \hat\theta_n$. 
We compute the test error with the parameters obtained in the last step and report that as the most realistic value of the error of the model.