Running into IndexError when following PyTorch CrossEntropyLoss example for classification
I am trying to follow exemplary usage of PyTorch's CrossEntropyLoss function for a classification problem using the fashionMNIST dataset.
I think the issue has to do with the dimension of the data output from my model. I'm not sure though, which makes figuring this out more difficult. Only running on Google Colab right now so I don't have easy access to a debugger (as far as I know).
Here's my code, please let me know what I am doing wrong!
import torch.nn as nn
import torch.optim as optim

model = nn.Sequential(
    nn.Linear(784, 1024),
    nn.ReLU(),
    nn.Linear(1024, 10),
    nn.ReLU(),
)
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.0)
loss_fn = nn.CrossEntropyLoss()

# Model Training
for epoch in range(1):
  for data, label in trainset:
    optimizer.zero_grad()
    output = model(torch.flatten(data))
    loss = loss_fn(output, label)
    loss.backward()
    optimizer.step()

And then the error:
IndexError                                Traceback (most recent call last)
&lt;ipython-input-3-6d9a9ea2e921&gt; in &lt;module&gt;()
     16     optimizer.zero_grad()
     17     output = model(torch.flatten(data))
---&gt; 18     loss = loss_fn(output, label)
     19     loss.backward()
     20     optimizer.step()

3 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in log_softmax(input, dim, _stacklevel, dtype)
   1603         dim = _get_softmax_dim('log_softmax', input.dim(), _stacklevel)
   1604     if dtype is None:
-&gt; 1605         ret = input.log_softmax(dim)
   1606     else:
   1607         ret = input.log_softmax(dim, dtype=dtype)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)


It looks like you haven't converted your dataset into DataLoader object, mostly that's the reason you're facing this issue.
However, coming back to your problem. the loss function expecting the shape of the output variable from the model to be a 2-D tensor, something like [1, 10] where 1 is the number of batches (which should be 1 as you've used only 1 data at a time and the 10 is the number of logits/classes, so 10 predictions). So, in short, to solve your issue, change the line of loss calculation with the following line-
loss = loss_fn(output.unsqueeze(0), torch.Tensor([label]).type(torch.LongTensor))

Here, .unsqueeze() increases the dimension of a tensor, and the label is an int, hence we've turned it into a 1-D tensor before sending it into the loss function.
Note: As you haven't mentioned much in your question about what you're trying to do exactly, what type of data you're working with, so it's not obvious but I don't know why you're using ReLU() as your output layer's activation (It's something we almost never do unless you've some special reason) however seeing that you're using CrossEntropyLoss() loss, which indicates you're maybe trying to do multi-class classification, in that case, I would suggest you remove the ReLU() after the last layer, then your model will able to learn different classes using softmax() activation (which is inbuilt within CrossEntropyLoss() loss) so you'll get what you want from the model.
