Optimal selection of k in K-NN I am currently reviewing some concepts related to Machine Learning, and I started to wonder about the hyperparameter selection of K-NN classifier.
Suppose you need to solve a classification task with a number of classes equal to M: I was thinking that the best choice for the parameter K of our classifier should be for K &gt; M.
In this way, we are avoiding all the pathological cases in which a sample may be in the middle of all the M classes and then have a tie. For instance, consider the following example in which we have M=3 and each geometrical shape represents a class:

Assume that K&lt;=M: for sure you will have a tie for a sample in the middle of samples 1, 2 and 3. This tie could be avoided if K &gt; M.
Clearly this is just a toy example, but I think it is sufficient to illustrate my thoughts. I have tried to look for an answer but I wasn't able to find any resource mentioning this, am I wrong in some way or this reasoning may be sound?
Your idea isn't wrong, however in k-NN there always might be a case where you have the same number of votes for 2 or more classes (e.g. you have $k=6$ and you have 3 samples of one class vs 3 of another). With your solution you are just overcoming one very small case of ties and the $k$ that you choose might not be the optimal $k$ for classification, which is your primary objective (i.e. you might hurt your classification performance with a sub-optimal $k$ so that you don't get one type of ties).
Because there are also other times of ties, you always need to have a tiebreaker condition. I have seen a lot of strategies for this, like reducing $k$ by one until you break the tie, selecting the class with the minimum distance, selecting the class with the fewest samples overall, etc.
Since you already have such a condition, you could also handle your situation, without selecting a sub-optimal $k$.