Optimize input instead of network in pytorch
I am trying to optimize the inputs in some required task and I don't want to update my network as its freezed. I have written a minimal example but its not working as z has the same value in all iterations.  I am sure that I am doing some silly mistake in this process. Any guidance is highly appreciated.
thanks.
import torch

z = torch.rand((1,6))
z.requires_grad_(True)
optimizer = torch.optim.SGD([z], lr= 0.1)

criteria = torch.nn.MSELoss()


for i in range(5):
    optimizer.zero_grad()
    print(z)
    loss = criteria(z, z+torch.rand(1))
    #print(loss)
    loss.backward()
    optimizer.step()

##output
tensor([[0.1105, 0.8152, 0.2820, 0.1122, 0.6645, 0.7211]], requires_grad=True)
tensor([[0.1105, 0.8152, 0.2820, 0.1122, 0.6645, 0.7211]], requires_grad=True)
tensor([[0.1105, 0.8152, 0.2820, 0.1122, 0.6645, 0.7211]], requires_grad=True)
tensor([[0.1105, 0.8152, 0.2820, 0.1122, 0.6645, 0.7211]], requires_grad=True)
tensor([[0.1105, 0.8152, 0.2820, 0.1122, 0.6645, 0.7211]], requires_grad=True)


The gradients are always 0. This can be seen as follows:
import torch

z = torch.rand((1,6))
z.requires_grad_(True)
z.retain_grad()
optimizer = torch.optim.SGD([z], lr= 0.1)

criteria = torch.nn.MSELoss()


for i in range(5):
    optimizer.zero_grad()
    # print(z)
    loss = criteria(z, z+torch.rand(1))
    # print(loss)
    loss.backward()
    print(z.grad)
    optimizer.step()

tensor([[0., 0., 0., 0., 0., 0.]])
tensor([[0., 0., 0., 0., 0., 0.]])
...

I don't have the mathematical proof, but it probably comes from the definition of the target z+torch.rand(1), that directly depends on the input z by a simple addition.
