two pytorch DistributedSampler same seeds different shuffling multiple GPU-s
I am trying to load two version (original and principal component pursuit (PCP) cleaned version) of the very same image data set for training a modell using pytorch on a multiple GPUs remote machine.
I would like to ensure the same shuffling order for both the original and the PCP cleaned data. To achieve this, I use torch.utils.data.DistributedSampler(datasetPCP, shuffle=True, seed=42) and torch.utils.data.DistributedSampler(dataset, shuffle=True, seed=42) samplers to pass these to the dataloaders for train my modell on 3 GPUs are present on the remote machine I use.
As far as I understood the same seed of the two sampler should ensure the exact same shuffling of the loaded data. However this is not the case.
Could anybody point me in the right direction?
Thanks a lot!

DistributedSampler is for distributed data training where we want different data to be sent to different processes so it is not what you need. Regular dataloader will do just fine.
Example:
import torch
from torch.utils.data.dataset import Dataset
from torch.utils.data import DataLoader, RandomSampler


class ToyDataset(Dataset):
    def __init__(self, type):
        self.type = type

    def __getitem__(self, idx):
        return f'{self.type}, {idx}'

    def __len__(self):
        return 10

def get_sampler(dataset, seed=42):
    generator = torch.Generator()
    generator.manual_seed(seed)
    sampler = RandomSampler(dataset, generator=generator)
    return sampler


original_dataset = ToyDataset('original')
pcp_dataset = ToyDataset('pcp')

original_loader = DataLoader(original_dataset, batch_size=2, sampler=get_sampler(original_dataset))
pcp_loader = DataLoader(pcp_dataset, batch_size=2, sampler=get_sampler(pcp_dataset))

for data in original_loader:
    print(data)

for data in pcp_loader:
    print(data)

Output:
['original, 2', 'original, 6']
['original, 1', 'original, 8']
['original, 4', 'original, 5']
['original, 0', 'original, 9']
['original, 3', 'original, 7']
['pcp, 2', 'pcp, 6']
['pcp, 1', 'pcp, 8']
['pcp, 4', 'pcp, 5']
['pcp, 0', 'pcp, 9']
['pcp, 3', 'pcp, 7']

