Why exactly using a test set for model evaluation is a bad idea? I don't understand why using the test set for model evaluation is a bad idea.

I completely understand why you should not use your test set to train your model (because in that case, you would be memorizing and you just cannot tell whether your model will generalize well or not if you don't have a separate test set). But why is it that simply using your test set to test (not train) your model is bad? You won't be changing any parameters of the model (because you are not training).

For instance, at the end of this video, Luis says we are breaking what he calls the "Golden rule" (i.e. never use your testing data for training). However, all I can see he is doing is using the test set to verify which model performs better to then be able to make a selection on which model he will use in the end.
In the linked video, a polynomial is fitted to some data. The degree of the polynomial has to be chosen, but it cannot be trained. It is therefore a hyperparameter. In order to choose the degree, several instances of the algorithm, each one trained with a polynomial of different degree, have to be evaluated on a data set that was not used for training. However, if we used the test set for assessing the values of hyperparameters, we would also be coupling the resulting model to the data in the test set, as the chosen hyperparameter (and therefore the whole model) would depend on the test set data.

The test set should only be used for the final evaluation in order to know how your algorithm is going to behave on data that has never been seen before by it. This is why there is normally a validation set used to tune the hyperparameters.