As shown in Figure , the individual elements must be combined to enable the creation of mini-batches for training. However, the objects are flexible and can be combined in several different ways. To investigate the most efficient way we constructed a small transformer model and trained it with several different ways of organizing the code and training. First, we put the tokenization and collating of the mini-batches into the training loop itself. Secondly, we put the tokenization into the dataset and padded it with a customized collate function in the dataloader. Alternatively the dataset could produce padding of a fixed width. Lastly we added both the tokenization, padding and collating into a custom collate function used by the dataloader. The timings for training an epoch of test data (USPTO-50K has 40.000 training samples) are shown in Table . shows the speedup that can be obtained by bundling together SMILES of similar length in the mini-batches. Sorting the samples by length is the optimal case for speed-up, but would likely be detrimental to training as the mini-batches would be the same in each epoch. The BucketBatchSampler object balances the need for similar size batches with the need for random batches, by dividing the data into a number of "buckets" based on the length of the SMILES strings. Mini-batches are then drawn randomly from each bucket, ensuring different mini-batches of similar length SMILES for each epoch. Padding is thus minimized and it is evident from Table that the training performance in terms of speed is close to the optimally obtainable with the sorted lengths. The full example code for the different code organizations can be found as a %delimited examples_training.py script in the examples directory in the code package.