how to implement several large state machines in hardware?
implementing a bigger state machine would possibly reduce fmax of a design anyway.

That would imply a state machine where the &quot;next state&quot; decision is overly complex. That's a criterion for &quot;not well-designed&quot;!

One way to deal with such an issue is to use a soft core to implement the state machine in software.

A processor is a state machine. Instructions and data alter its state.
It's just meant to be more general.

However, having multiple softcore processors e.g Nios II may not be practical as well.

I'm not advocating for this here, you should probably just optimize your regular FSMs, but having multiple processors in one design is far from unusual. Actually, especially for things that handle multiple interfaces, it's somewhat usual to have application-specific processors for each and let them talk via a common internal bus or shared memory.
(In fact, often a layered approach: hardware FSM to handle the physical layer, software FSM to handle logical things)

Now if it becomes clear that the design requires multiple state machines where each has maybe 10s of states,

You're certainly not calling &quot;10s of states&quot; a large state machine? You might want to read protocol standards (USB, Ethernet...)

does this mean that the design must be simply implemented fully in software and not in hardware or techniques exist to implement multiple large state machines which also ensure that we get large fmax?

Certainly not.