Stateful LSTM : Using different training window
I don’t see anything wrong with that. In fact it sounds like a good form of data augmentation. 

It does sound like you are training with a batch size of 1, which may be slow. You could think about creating batches in a similar way, where the first elements of the batches are the sequences 1-10, 11-20, ..., the second elements of the batches are 2-12, 13-23, ..., all the way to 9-19, 10-29, ...