Why can't I choose my hyper-parameter in the training set?
I wouldn't say you can not tune the hyper-parameters in the trainig dataset, but the purpose of doing so is different than in the validation set.

In general, what it is intended with a ML algorithm is how to optimally classify or perform regression given some training data. Once the model is trained, it will be used with new data to obtain predictions, thus:


We need some training data. With this dataset, the model will look for the optimal weights and biases that minimizes the selected loss function.
We need some upper bound to our performance metric: for example, we can decide that our model has to perform like a human, which has showed an average metric of 95% for some specific task. The upper bound can be the best score in a certan benchmark, etc.
We start training the model on the training data and we evaluate its metric, lets take the accuracy. If the accuracy is too low, we can tune the hyper-paramters until the accuracy increases in the training data (no evaluation dataset is used here). The difference between our model's accuracy and the upper bound is taken as the bias of the model (this bias has not to be confused with the biases of the neurons). So, the accuracy on the training data gives us the bias of our model.
Once we decided our bias is reasonable (optimally 0) arises the question of how the model will perform when it is fed with data that has not be used for training (the real application). The difference between the accuracy on this unseen data and the accuracy on the training data is the variance of the model. This unseen data is the validation set and give us an idea of how the model generalizes to new data. If the variance is high, then the model poorly generalizes to new data. Then, we perform hyper-paramater tunning and evaluating the accuracy on the validation data until the variance is low enough, trying to not worsening the bias (variance-bias trade-off).


Example: desired accuracy = 95%. Training accuracy = 93%. Validation accuracy = 82% -> Bias = 2%, Variance = 11%

Summarizing:


Tunning in the training data = decreasing bias
Tunning in the validation data = decreasing variance


What's more unclear is the role of the test dataset. I usually use it when I decided that the model will be no longer changed and I need the final metric that describes the model. The test set can also be used to have an idea about how the model performs with data which have not be intended to work with. In general, the test set gives the power of the model for the inference task.