CUDA vs. DataParallel: Why the difference?
I have a simple neural network model and I apply either cuda() or DataParallel() on the model like following.

model = torch.nn.DataParallel(model).cuda()


OR,

model = model.cuda()


When I don't use DataParallel, rather simply transform my model to cuda(), I need to explicitly convert the batch inputs to cuda() and then give it to the model, otherwise it returns the following error. 


  torch.index_select received an invalid combination of arguments - got (torch.cuda.FloatTensor, int, torch.LongTensor)


But with DataParallel, the code works fine. Rest of the other things are same. Why this happens? Why when I use DataParallel, I don't need to transform the batch inputs explicitly to cuda()?

Because, DataParallel allows CPU inputs, as it's first step is to transfer inputs to appropriate GPUs.

Info source: https://discuss.pytorch.org/t/cuda-vs-dataparallel-why-the-difference/4062/3
