Efficient recurrent network for sequences of varying length Suppose I have a bunch of sequences of varying lenghts. The absolute majority of them are short, just a few dozens items long. However, very few of them are significantly longer - more than a hundred items long. The question is, how to organize them efficiently as an input to recurrent layer?

Padding doesn't work, since many sequences need to be heavily padded. Limiting batch size is not an option - these sequences are obtained as parts of a larger structure.
Sequence bucketing

Depending on the input length of sequences you can dynamically change the padding and speed it up.

Take a look at here Sequence bucketing