Very simple optim.SGD training loop not working as expected - PyTorch
I've been reading through the PyTorch documentation and I've been trying to figure out MSELoss and autograd. I tried creating a very simple training loop that takes two random tensors and updates the values in each tensor so that the sum all values in tensor1 plus the sum of all values in tensor2 add up to some target number. In my example I used 100. Here is how I implemented this:

import torch.nn as nn
import torch
import torch.optim as optim

loss = nn.MSELoss()
tensor1 = torch.randn(3, 5, requires_grad=True)
tensor2 = torch.randn(3, 5, requires_grad=True)
target_sum = torch.FloatTensor(torch.Tensor([100]))
optimizer = optim.SGD([nn.Parameter(tensor1), nn.Parameter(tensor2)], lr=1e-4)
print(str(tensor1.sum().item()) + str(tensor2.sum().item()))
for i in range(100):
    pred = tensor1.sum() + tensor2.sum()
    optimizer.zero_grad()
    loss(pred, target_sum).backward
    optimizer.step()
print(str(tensor1.sum().item()) + str(tensor2.sum().item()))


The sums of the tensors before the training loop and after the training loop are the same, but I should be seeing an increase in the sum and have it approach 100. I'm not sure what I'm missing here. I think it  might have something to do with my optimizer as the training loop is almost straight out of the documentation example. I'm new to PyTorch and I appreciate any help! 

Use abs(pred - target_sum) to calculate loss since pred and target_sum are scalars. Also, remove nn.Parameter() wrapper in optim definition.
Here is what I did. 

In [22]: import torch.nn as nn 
    ...: import torch 
    ...: import torch.optim as optim 
    ...:  
    ...: tensor1 = torch.randn(3, 5, requires_grad=True) 
    ...: tensor2 = torch.randn(3, 5, requires_grad=True) 
    ...: target_sum = 100 
    ...: optimizer = optim.SGD([tensor1, tensor2], lr=1e-2) 
    ...: print(tensor1.sum().item() + tensor2.sum().item()) 
    ...: for i in range(1000): 
    ...:     pred = tensor1.sum() + tensor2.sum() 
    ...:     optimizer.zero_grad() 
    ...:     loss = abs((pred-target_sum)) 
    ...:     loss.backward() 
    ...:     optimizer.step() 
    ...: print(tensor1.sum().item()+ tensor2.sum().item())


This results before and after the loop are as follows

-0.777163028717041
100.02290725708008

