Create an RNN on text sources with different lengths
I think that if you append a token &lt;EOS&gt; (end of sentence) at the end of each sentence when you merge, this would not be a problem, because the RNN would learn to cut sentences and to generate independently if you shuffle your data and train with several shuffles.
However, as you say your data is heterogeneous, you might consider to first run some clustering algorithm and then train your RNN to generate conditionally to a cluster. To build the training dataset, you can merge all data for each cluster. This is a bit more complex than simply &quot;automatic generation&quot; but it depends on what your goal is.