pytorch dataloader default_collate argument use with to(device)
Ive been trying to integrate the to(device) inside my dataloader using to(device) as seen in https://github.com/pytorch/pytorch/issues/11372
I defined it on FashionMNIST in the following way:
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
batch_size = 32
trainset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/',
                                 download=True,
                                 train=True,
                                 transform=transforms.ToTensor())
rain_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=False, collate_fn=lambda x: default_collate(x).to(device))

But i get the following error:
AttributeError: 'list' object has no attribute 'to'
It seems that the output of default collate is a list of length 2 with the first element being the image tensor and the second the labels tensor (since its the output of next(iter(train_loader)) with collate_fn=None), so I tried with the following defined function:
def to_device_list(l, device):
    return [l[0].to(device), l[1].to(device)]
train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=False, collate_fn=lambda x: to_device_list(x, device))

And I got the following error:
AttributeError: 'tuple' object has no attribute 'to'
Any help please on how to do it?

The fashion mnist dataset returns a tuple of img and target, where the img is tensor and target is int value for class.
Now, your dataloader takes batch size samples from dataset class to get list of samples. Note, this list of samples is now, List[Tuple[Tensor, int]](using typing annotation here). Then it calls, collate function to convert List[Tuple[Tensor, int]] into List[Tensor], where this list has 2 tensors. The first tensor is stacked array of images of size [32, 1, 28, 28], where 32 was batch size and second tensor is tensor array of int values(class labels).
The default_collate function, just converts array of structures to structures of array.
Now, when you use collate_fn=lambda x: default_collate(x).to(device), notice that default_collate returns you a list of tensors. So calling .to on list wont work and should be called on all elements of the list.
Solution
Use
collate_fn=lambda x: list(map(lambda x: x.to(device), default_collate(x))))

The map function transfers each element of list(from default_collate) to cuda, and finally, call list, since map is evaluated lazy in python3.
