Why override Dataset instead of directly pass in input and labels, pytorch
Sorry if what I say here is wrong -- new to pytorch. 

From what I can tell there are two main ways of getting training data and passing through a network. One is to override Dataset and the other is to just prepare your data correctly and then iterate over it, like shown in this example: pytorch classification example

which does something like

rnn(input, hidden, output)
for i in range(input.size()[0]):
    output, hidden = rnn(input[i], hidden)


The other way would be to do something like

for epoch in range(epochs):
    for data, target in trainloader:
        computer model etc


where in this method, trainloader is from doing something like

trainloader = DataLoader(my_data)


after overriding getitem and len

My question here, is what are the differences between these methods, and why would you use one over the other? Also, it seems to me that overriding Dataset doesn't work for something that has lets say an input layer of size 100 nodes with an output of 10 nodes, since when you return getitem it needs a pair of (data, label). This seems like a case where I probably don't understand how to use Dataset very well, but that is why I'm asking in the first place. I think I read something about a collate function which might help in this scenario?

Dataset class and the Dataloader class in PyTorch help us to feed our own training data into the network. Dataset class is used to provide an interface for accessing all the training or testing samples in your dataset. In order to achieve this, you have to implement at least two methods, __getitem__ and __len__ so that each training sample can be accessed by its index. In the initialization part of the class, we load the dataset (as float type) and convert them into Float torch tensors. __getitem__ will return the features and target value.

What are the differences between these methods?

In PyTorch either you can prepare your data such that the PyTorch DataLoader can consume it and you get an iterable object or you can overload the default DataLoader to perform some custom operations like if you want to do some preprocessing of text/images, stack frames from videos clips, etc.

Our DataLoader behaves like an iterator, so we can loop over it and fetch a different mini-batch every time.

Basic Sample

from torch.utils.data import DataLoader

train_loader = DataLoader(dataset=train_data, batch_size=16, shuffle=True)
valid_loader = DataLoader(dataset=valid_data, batch_size=16, shuffle=True)

# To retrieve a sample mini-batch, one can simply run the command below — 
# it will return a list containing two tensors:
# one for the features, another one for the labels.

next(iter(train_loader))
next(iter(valid_loader))


Custom Sample

import torch
from torch.utils.data import Dataset, Dataloader

class SampleData(Dataset):
def __init__(self, data):
        self.data = torch.FloatTensor(data.values.astype('float'))

    def __len__(self):
        return len(self.data)

    def __getitem__(self, index):
        target = self.data[index][-1]
        data_val = self.data[index] [:-1]
        return data_val,target

train_dataset = SampleData(train_data)
valid_dataset = SampleData(valid_data)


device = "cuda" if torch.cuda.is_available() else "cpu"
kwargs = {'num_workers': 1, 'pin_memory': True} if device=='cuda' else {}
train_loader = DataLoader(train_dataset, batch_size=train_batch_size, shuffle=True, **kwargs)
test_loader = DataLoader(valid_dataset, batch_size=test_batch_size, shuffle=False, **kwargs)



Why would you use one over the other?

It solely depends on your use-case and the amount of control you want. PyTorch has given you all the power and it is you who is going to decide how much you want to. Suppose you are solving a simple image classification problem, then,

You can simply put all the images in a root folder with each subfolder containing the samples belonging to a particular class and label the folder with the class name. When training we just need to specify the path to the root folder and the PyTorch DataLoader will automatically pick images from each folder and training the model. 

But on the other hand, if you have classifying video clips or video sequences generally known as video tagging in a large video file then you need to write your custom DataLoader to load the frames from the video, stack it and give input to the DataLoader.

Use can find some useful links below for further reference:

https://pytorch.org/docs/stable/data.html

https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel

https://pytorch.org/tutorials/beginner/data_loading_tutorial.html
