When to use Stateful LSTM?
As for stateful LSTM and its understanding, refer to here. Quoting an answer from there:

"I’m given a big sequence (e.g. Time Series) and I split it into smaller sequences to construct my input matrix X. Is it possible that the LSTM may find dependencies between the sequences? 

No it’s not possible unless you go for the stateful LSTM. Most of the problems can be solved with stateless LSTM so if you go for the stateful mode, make sure you really need it. In stateless mode, long term memory does not mean that the LSTM will remember the content of the previous batches."

Therefore, stateful is useful if you wish to save the state of the neurons for the next training session instead of resetting it.


  "My goal here is to generate sequences that looks like something that
  the model have seen already."


Then maybe this sequence2sequence (seq2seq) LSTM encoder-decoder is exactly what you need.