title: How to transform output of neural network and still train?
body: <p>I have a neural network which outputs <code>output</code>. I want to transform <code>output</code> before the loss and backpropogation happen.</p>
  <p>Here is my general code:</p>
  <pre><code>with torch.set_grad_enabled(training):
                    outputs = net(x_batch[:, 0], x_batch[:, 1]) # the prediction of the NN
                    # My issue is here:
                    outputs = transform_torch(outputs)
                    loss = my_loss(outputs, y_batch)
  
                    if training:
                        scheduler.step()
                        loss.backward()
                        optimizer.step()
  </code></pre>
  <p>I have a transformation function which I put my output through:</p>
  <pre><code>def transform_torch(predictions):
      torch_dimensions = predictions.size()
      torch_grad = predictions.grad_fn
      cuda0 = torch.device('cuda:0')
      new_tensor = torch.ones(torch_dimensions, dtype=torch.float64, device=cuda0, requires_grad=True)
      for i in range(int(len(predictions))):
        a = predictions[i]
        # with torch.no_grad(): # Note: no training happens if this line is kept in
        new_tensor[i] = torch.flip(torch.cumsum(torch.flip(a, dims = [0]), dim = 0), dims = [0])
      return new_tensor
  </code></pre>
  <p>My problem is that I get an error on the next to last line:</p>
  <pre><code>RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.
  </code></pre>
  <p>Any suggestions? I have already tried using &quot;with torch.no_grad():&quot; (commented), but this results in very poor training and I believe that the gradients don't backpropogate properly after the transformation function.</p>
  <p>Thanks!</p>