How to find input that maximizes output of a neural network using pytorch
I have a pytorch network, that have been trained and weights are updated (complete training).
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(1, H)
        self.fc2 = nn.Linear(1, H)
        self.fc3 = nn.Linear(H, 1)

    def forward(self, x, y):
        h1 = F.relu(self.fc1(x)+self.fc2(y))
        h2 = self.fc3(h1)
        return h2   

After training, I want to maximize the output of the network with respect to input. In other words, I want to optimize the input to maximize the neural network output, without changing weights. How can I achieve that.
My trial, but it doesn't make sense:
in     = torch.autograd.Variable(x)
out    = Net(in)
grad   = torch.autograd.grad(out, input)

      



Disable gradients for the network.
Set your input tensor as a parameter requiring grad.
Initialize an optimizer wrapping the input tensor.
Backprop with some loss function and a goal tensor
...
Profit!

import torch

f = torch.nn.Linear(10, 5)
f.requires_grad_(False)
x = torch.nn.Parameter(torch.rand(10), requires_grad=True)
optim = torch.optim.SGD([x], lr=1e-1)
mse = torch.nn.MSELoss()
y = torch.ones(5)  # the desired network response

num_steps = 5  # how many optim steps to take
for _ in range(num_steps):
   loss = mse(f(x), y)
   loss.backward()
   optim.step()
   optim.zero_grad()

But make sure that your goal tensor is well defined wrt. the network's monotonicity, otherwise you might end up with nans.
