title: What happens when the label's dimension is different from neural network's output layer's dimension in PyTorch?
body: <p>It makes intuitive sense to me that the label's dimension should be the same as the neural network's last layer's dimension. However, with some experiments using PyTorch, it turns out that it somehow works.</p>
  <p>Code:</p>
  <pre><code>import torch
  import torch.nn as nn
  
  X = torch.tensor([[1],[2],[3],[4]], dtype=torch.float32) # training input
  Y = torch.tensor([[2],[4],[6],[8]], dtype=torch.float32) # training label
  
  model = nn.Linear(1,3)
  learning_rate = 0.01
  optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)
  
  for epoch in range(10):
      y_pred model(X)
      loss = nn.MSELoss(Y, y_pred)
      loss.backward()
      optimizer.zero_grad()
      optimizer.step()
  </code></pre>
  <p>In the above code,  <code>model = nn.Linear(1,3)</code> is used instead of <code>model = nn.Linear(1,1)</code>. As a result, while Y.shape is (4,1), y_pred.shape is (4,3).</p>
  <p>The code works with a warning saying that &quot;Using a target size that is different to the input size will likely lead to incorrect results due to broadcasting. &quot;</p>
  <p>I got the following output when I executed <code>model(torch.tensor([10], dtype=torch.float32))</code>:</p>
  <p><code>tensor([20.0089, 19.6121, 19.1967], grad_fn=&lt;AddBackward0&gt;)</code></p>
  <p>All three outputs seems correct to me. But how is the loss calculated if the sizes of the data are different?</p>
  <p>Should we in any case use a target size that is different to the input size? Is there a benefit for this?</p>