How to flow gradient through parameter updates?
I have a parameter
import torch
import torch.nn as nn
x = nn.Parameter(torch.tensor([1.0]))

I need to update the value of this parameter while maintaining the gradient flow (for a meta learning problem).
PyTorch does not allow in-place operations on leaf-variable like
a = torch.tensor([2.0], requires_grad=True)
x.copy_(a)
x.backward()
&gt;&gt;&gt; RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

In addition, I also tried the following, but while working, the gradient didn't flow:
x = nn.Parameter(torch.tensor([1.0]))
a = torch.tensor([2.0], requires_grad=True)
x = nn.Parameter(a)
x.backward()
print(a.grad)
&gt;&gt;&gt; None

How can I update the value of a parameter while maintaining gradient flow?

What you are looking for is torch.Tensor.clone:

This function is differentiable, so gradients will flow back from the
result of this operation to input. To create a tensor without an
autograd relationship to input see detach().

&gt;&gt;&gt; a = torch.tensor([2.0], requires_grad=True)
&gt;&gt;&gt; x = a.clone()
&gt;&gt;&gt; x.backward()

&gt;&gt;&gt; a.grad
tensor([1.])

