Why can PDAs have infinite stack but FSMs cannot have infinite state?
One of the crucial properties of a model of computation is that each automaton has a finite description. Both FSMs and PDAs have finite description. In contrast, an &quot;infinite state machine&quot; won't necessarily have a finite description, unless it is constrained somehow.
Also, as mentioned in the comments, an unconstrained infinite state machine can accept any language. Indeed, we can construct a machine whose states are $\Sigma^*$. We define the transition function so that after reading the word $w$, the machine is at state $w$. Thus, by choosing the accepting states, we can obtain any language, making the model uninteresting.