Can a finite state machine represent every possible discrete deterministic system?
Any real discrete system will have a finite number of memory cells (and a finite number of states as a consequence), so it can be theoretically represented by a finite state machine.

If we also consider abstractions, then a PDA or a Turing machine are examples of discrete systems with infinite number of states.

I mention abstractions because they can represent some real systems for which FSMs are useless, even if strictly speaking such abstractions cannot be implemented in silicon. For example, a theoretical FSM representing a micro-controller with 128 bytes of RAM will have more states than there are atoms in the known universe. Representing such an MCU with a state machine is practically impossible, and if stack overflows are avoided in the implementation, I'd call it a PDA rather than an FSM.