Use PyTorch to adjust Tensor matrix values based on numbers I calculate from the Tensors?
I have two tensors (matrices) that I've initialized:

sm=Var(torch.randn(20,1),requires_grad=True)
sm = torch.mm(sm,sm.t()) 
freq_m=Var(torch.randn(12,20),requires_grad=True)


I am creating two lists from the data inside these 2 matrices, and I am using spearmanr to get a correlation value between these 2 lists. How I am creating the lists is not important, but the goal is to adjust the values inside the matrices so that the calculated correlation value is as close to 1 as possible.

If I were to solve this problem manually, I would tweak values in the matrices by .01 (or some small number) each time and recalculate the lists and correlation score. If the new correlation value is higher than the previous one, I would save the 2 matrices and tweak a different value until I get the 2 matrices that give me the highest correlation score possible.

Is PyTorch capable of doing this automatically? I know PyTorch can adjust based on an equation but the way I want to adjust the matrix values is not against an equation, it's against a correlation value that I calculate. Any guidance with this is greatly appreciated!

Pytorch has an autograd package, that means if you have variable and you pass them through differentiable functions and get a scalar result, you can perform a gradient descent to update the variable to lower or augment the scalar result.

So what you need to do is to define a function f that works on tensor level such that f(sm, freq_m) will give you the desired correlation.

Then, you should do something like:

lr = 1e-3
for i in range(100):
  # 100 updates
  loss = 1 - f(sm, freq_m)
  print(loss)
  loss.backward()
  with torch.no_grad():
    sm -= lr * sm.grad
    freq_m -= lr * freq_m.grad
    # Manually zero the gradients after updating weights
    sm.grad.zero_()
    freq_m.grad.zero_()


The learning rate is basically the size of the step you do, a learning rate too high will cause the loss to explode, and a learning rate too little will cause a slow convergence, I suggest you experiment.

Edit : To answer the comment on loss.backward : for any differentiable function f, f is a function of multiple tensors t1, ..., tn with requires_grad=True as a result, you can calculate the gradient of the loss with respect to each of those tensors. When you do loss.backward, it calculates those gradients and store those in t1.grad, ..., tn.grad. Then you update t1, ..., tn using gradient descent in order to lower the value of f. This update doesn't need a computational graph, so this is why you use with torch.no_grad().
At the end of the loop, you zero the gradients because .backward doesn't overwrite the gradients but rather add the new gradients to them. More on that here : https://discuss.pytorch.org/t/why-do-we-need-to-set-the-gradients-manually-to-zero-in-pytorch/4903
