How to shuffle the batches themselves in pytorch?
How to keep the sequences in each batch unshuffled, while shuffling the batches?
Inspired by the question asked here.

While this is not a direct answer to your question. I want to address an issue with the answer you posted yourself. In my opinion, doing the following, is a very bad idea:
dataloader = random.sample(list(dataloader), len(dataloader))

This defeats the whole purpose of creating a dataset and a data loader in the first place. Because as soon as you call list(dataloader) you end up compiling down your dataset into a single list of tensors. In other words, it will call __getitem__ for each index in the dataset. A data loader is designer to load data batch by batch (or more depending on the number of workers) avoiding to load the entire dataset in memory at once.
This is even more important when working with images which requires image loading from the file system. This is critical and I believe you shouldn't be doing this at all.
Take a look here, with a dummy dataset:
class DS(Dataset):
    def __getitem__(self, _):
        return torch.rand(100)

    def __len__(self):
        return 10000

dl = DataLoader(DS(), batch_size=16)
x = list(dl)

Here x will contain 10,000 tensors of size 100, which your computer can handle. Now imagine having a dataset made up of 10,000 512x512 RGB images, you just can't hold that much in memory!
Futhermore, something I haven't even mentioned is data augmentation. Which is only possible when retaining a data loader (i.e. a generator). So the transformation are computed on the input data at runtime vs at compile-time (if you will) when using list(dataloader).

I would instead suggest you make your Dataset generate that unshuffled sequence for each item, then make a DataLoader out of it with shuffle=True. This feels a lot more natural than generating a DataLoader only to compile it down. Use your dataset class as it's supposed to. It should be the one constructing each sequence (i.e. datapoint), or as @Prune puts it a &quot;single observation object&quot;.
