How can I save PyTorch's DataLoader instance?
I want to save PyTorch's torch.utils.data.dataloader.DataLoader instance, so that I can continue training where I left off (keeping shuffle seed, states and everything).

It's quite simple. One should design their own Sampler which takes the starting index and shuffles the data by itself:
import random
from torch.utils.data.dataloader import Sampler


random.seed(224)  # use a fixed number


class MySampler(Sampler):
    def __init__(self, data, i=0):
        random.shuffle(data)
        self.seq = list(range(len(data)))[i * batch_size:]

    def __iter__(self):
        return iter(self.seq)

    def __len__(self):
        return len(self.seq)

Now save the last index i somewhere and the next time instantiate the DataLoader using it:
train_dataset = MyDataset(train_data)
train_sampler = MySampler(train_dataset, last_i)
train_data_loader = DataLoader(dataset=train_dataset,                                                         
                               batch_size=batch_size, 
                               sampler=train_sampler,
                               shuffle=False)  # don't forget to set DataLoader's shuffle to False

It's quite useful when training on Colab.
