In sequence models, is it possible to have training batches with different timesteps each to reduce the required padding per input sequence? I want to train an LSTM model with variable length inputs. Specifically I want to use as little padding as possible while still using minibatches.
As far as I understand each batch requires a fixed number of timesteps for all inputs, necessitating padding. But different batches can have different numbers of timesteps for the inputs, so in each batch inputs only have to be padded to the length of the longest input-sequence in that same batch. This is what i want to implement.
What I need to do:

Dynamically create batches of a given size during training, the inputs within each batch are padded to the longest sequence within that same batch.
The training data is shuffled after each epoch, so that inputs appear in different batches across epochs and are padded differently.

Sadly my googling skills have failed me entirely. I can only find examples and resources on how to pad the entire input set to a fixed length, which is what i had been doing already and want to move away from. Some clues point me towards tensorflow's Dataset API, yet I can't find examples of how and why it would apply to the problem I am facing.
I'd appreciate any pointers to resources and ideally examples and tutorials on what I am trying to accomplish.
The answer to your needs is called &quot;bucketing&quot;. It consists of creating batches of sequences with similar length, to minimize the needed padding.
In tensorflow, you can do it with tf.data.experimental.bucket_by_sequence_length. Take into account that previously it was in a different python package (tf.contrib.data.bucket_by_sequence_length), so the examples online may containt the outdated name.
To see some usage examples, you can check this jupyter notebook, or other answers in stackoverflow, or this tutorial.