Pytorch DataLoder Is Very Slow
I have a problem with the DataLoader form Pytorch, because is very slow.
I did a test to show this, here is the code:
data = np.load('slices.npy')
data = np.reshape(data, (-1, 1225))
data = torch.FloatTensor(data).to('cuda')
print(data.shape)
# ==&gt; torch.Size([273468, 1225])

class UnlabeledTensorDataset(TensorDataset):
    def __init__(self, data_tensor):
        self.data_tensor = data_tensor
        self.samples = data_tensor.shape[0]

    def __getitem__(self, index):
        return self.data_tensor[index]
    
    def __len__(self):
        return self.samples

test_set = UnlabeledTensorDataset(data)
test_loader = DataLoader(test_set, batch_size=data.shape[0])

start = datetime.datetime.now()
with torch.no_grad():
    for batch in test_loader:
        print(batch.shape)     # ==&gt; torch.Size([273468, 1225])
        y_pred = model(batch)
        loss = torch.sqrt(criterion(y_pred, batch))
        avg_loss = loss
print(round((datetime.datetime.now() - start).total_seconds() * 1000, 2))
# ==&gt; 1527.57  (milliseconds)   !!!!!!!!!!!!!!!!!!!!!!!!

start = datetime.datetime.now()
with torch.no_grad():
    print(data.shape)     # ==&gt; torch.Size([273468, 1225])
    y_pred = model(data)
    loss = torch.sqrt(criterion(y_pred, data))
    avg_loss = loss
print(round((datetime.datetime.now() - start).total_seconds() * 1000, 2))
# ==&gt; 2.0     (milliseconds)    !!!!!!!!!!!!!!!!!!!!!!!!

I will like to use the DataLoader but I want a way to fix the slow issue, dose anyone know why this is happening?

The time difference seems logical to me:

On one end you're looping over test_loader and doing 1225 inferences.

On the other, you are doing a single inference.


