Why we need image.to('CUDA') when we have model.to('CUDA')
I'm taking a course on PyTorch.
And I'm wondering why we need to separately tell to torch.utils.data.DataLoader output on what device it's running on. If the model is already on CUDA why doesn't it automatically change the inputs accordingly? This pattern seems funny to me:

model.to(device)

for ii, (inputs, labels) in enumerate(trainloader):

    # Move input and label tensors to the GPU
    inputs, labels = inputs.to(device), labels.to(device)


Is there a use case where I'd like to have model running on GPU but my inputs to be on CPU mode or vice versa? 

When you call model.to(device) (assuming device is a GPU) your model parameters will be moved to your GPU. Regarding to your comment: they are moved from CPU memory to GPU memory then.

By default newly created tensors are created on CPU, if not specified otherwise. So this applies also for your inputs and labels.

The problem here is that all operands of an operation need to be on the same device! If you leave out the to and use CPU tensors as input you will get an error message.

Here is an short example for illustration:

import torch

# device will be 'cuda' if a GPU is available
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# creating a CPU tensor
cpu_tensor = torch.rand(10)
# moving same tensor to GPU
gpu_tensor = cpu_tensor.to(device)

print(cpu_tensor, cpu_tensor.dtype, type(cpu_tensor), cpu_tensor.type())
print(gpu_tensor, gpu_tensor.dtype, type(gpu_tensor), gpu_tensor.type())

print(cpu_tensor*gpu_tensor)


Output:

tensor([0.8571, 0.9171, 0.6626, 0.8086, 0.6440, 0.3682, 0.9920, 0.4298, 0.0172,
        0.1619]) torch.float32 &lt;class 'torch.Tensor'&gt; torch.FloatTensor
tensor([0.8571, 0.9171, 0.6626, 0.8086, 0.6440, 0.3682, 0.9920, 0.4298, 0.0172,
        0.1619], device='cuda:0') torch.float32 &lt;class 'torch.Tensor'&gt; torch.cuda.FloatTensor
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-15-ac794171c178&gt; in &lt;module&gt;()
     12 print(gpu_tensor, gpu_tensor.dtype, type(gpu_tensor), gpu_tensor.type())
     13 
---&gt; 14 print(cpu_tensor*gpu_tensor)

RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 'other'

