Is Pytorch DataLoader Iteration order stable?
Is the iteration order for a Pytorch Dataloader guaranteed to be the same (under mild conditions)?

For instance:

dataloader = DataLoader(my_dataset, batch_size=4,
                        shuffle=True, num_workers=4)
print("run 1")
for batch in dataloader:
  print(batch["index"])

print("run 2")
for batch in dataloader:
  print(batch["index"])



So far, I've tried testing it and it appears to not be fixed, same order for both runs. Is there a way to make the order the same? Thanks

edit: i have also tried doing

unlabeled_sampler = data.sampler.SubsetRandomSampler(unlabeled_indices)
unlabeled_dataloader = data.DataLoader(train_dataset, 
                sampler=unlabeled_sampler, batch_size=args.batch_size, drop_last=False)


and then iterating through the dataloader twice, but the same non-determinism results. 

The short answer is no, when shuffle=True the iteration order of a DataLoader isn't stable between iterations. Each time you iterate on your loader the internal RandomSampler creates a new random order.

One way to get a stable shuffled DataLoader is to create a Subset dataset using a shuffled set of indices.

shuffled_dataset = torch.utils.data.Subset(my_dataset, torch.randperm(len(my_dataset)).tolist())
dataloader = DataLoader(shuffled_dataset, batch_size=4, num_workers=4, shuffled=False)

