Usage of data.to(device) with cuda GPUs
I'm trying to implement a neural network to run across 8 GPUs and i just want clarification on what exactly these commands do

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
data.to(device)


Will this automatically spread the training across the 8 GPUs ?
Thanks!

No. The code snippet will move the model and data to GPU if CUDA is available, otherwise, it will put them in CPU.


torch.device('cuda') refers to the current cuda device
torch.device('cuda:0') refer to the cuda device with index=0


To use all the 8 GPUs, you can do something like:

if torch.cuda.device_count() &gt; 1:
    model = torch.nn.DataParallel(model)


Note:


torch.cuda.device_count() returns the number of GPUs available.
You do not need to call: data = torch.nn.DataParallel(data)


Why? Because torch.nn.DataParallel


  is a container that parallelizes the application of the given module by splitting the input across the specified devices by chunking in the batch dimension (other objects will be copied once per device). In the forward pass, the module is replicated on each device, and each replica handles a portion of the input. During the backward pass, gradients from each replica are summed into the original module.


The batch size should be larger than the number of GPUs used.
