Linear regression with pytorch
I tried to run linear regression on ForestFires dataset. 
Dataset is available on Kaggle and gist of my attempt is here: 
https://gist.github.com/Chandrak1907/747b1a6045bb64898d5f9140f4cf9a37

I am facing two problems:


Output from prediction is of shape 32x1 and target data shape is 32. 



  input and target shapes do not match: input [32 x 1], target [32]¶


Using view I reshaped predictions tensor.


  y_pred = y_pred.view(inputs.shape[0])


Why there is a mismatch in shapes of predicted tensor and actual tensor?


SGD in pytorch never converges. I tried to compute MSE manually using 



  print(torch.mean((y_pred - labels)**2))


This value does not match 


  loss = criterion(y_pred,labels)


Can someone highlight where is the mistake in my code?

Thank you.

Problem 1

This is reference about MSELoss from Pytorch docs: https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss

Shape:
 - Input: (N,∗) where * means, any number of additional dimensions
 - Target: (N,∗), same shape as the input


So, you need to expand dims of labels: (32) -> (32,1), by using: torch.unsqueeze(labels, 1) or labels.view(-1,1)

https://pytorch.org/docs/stable/torch.html#torch.unsqueeze


  torch.unsqueeze(input, dim, out=None) → Tensor
  
  Returns a new tensor with a dimension of size one inserted at the specified position.
  
  The returned tensor shares the same underlying data with this tensor.


Problem 2

After reviewing your code, I realized that you have added size_average param to MSELoss: 

criterion = torch.nn.MSELoss(size_average=False)



  size_average (bool, optional) – Deprecated (see reduction). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True


That's why 2 computed values not matched. This is sample code:

import torch
import torch.nn as nn

loss1 = nn.MSELoss()
loss2 = nn.MSELoss(size_average=False)
inputs = torch.randn(32, 1, requires_grad=True)
targets = torch.randn(32, 1)

output1 = loss1(inputs, targets)
output2 = loss2(inputs, targets)
output3 = torch.mean((inputs - targets) ** 2)

print(output1)  # tensor(1.0907)
print(output2)  # tensor(34.9021)
print(output3)  # tensor(1.0907)

