I get what you're saying, nobody likes to be put into a negative stereotype. The difference is that some stereotypes especially racial ones lead to real systemic oppression, and hate crimes while others are more of a social bias. It's usually understood they mean creepy men.