How do altcoins tell each other apart? So there are hundreds of live and even thriving altcoins today. They are all basically a clone of Bitcoin and operate basically the same.

But when you run the mining client for any given coin, how does it know which coin to work on? How does the software know to connect to Litecoin and not Dogecoin, or some other scrypt coin? I realize that when you mine you download the whole blockchain and work on it locally, then when you solve the proof of work for the next block you broadcast it to the network, but this doesn't seem like enough. How do the blocks stay with their respective coins?
Because all these altcoins run on different TCP ports and furthermore (fair) nodes won't relay invalid messages (wrong blockchain, i.e. based on different genesis block, different magic value,  etc.) or IP addresses of nodes running other altcoins.