Do all Bitcoin alternatives need to have a coin limit?
E-Gulden has a limit of 21 Million.

Most, but not all cryptos have a limit, for example DogeCoin recently announced that there would be a minimum reward of 10,000 doge for every block after the previously announced reward schedule was supposed to end.