title: Why does higher need to deep copy the parameters of the base model to create a functional model?
body: <p>I found <a href="https://github.com/facebookresearch/higher/blob/8f0716fb1663218324c02dabdba26b639959cfb6/higher/optim.py#L101" rel="nofollow noreferrer">this line of code</a> in the higher library:</p>
  
  <pre><code>self.param_groups = _copy.deepcopy(other.param_groups)
  </code></pre>
  
  <p>and I don't understand why that's needed.</p>
  
  <p>If anything I think it's harmful as I've outline <a href="https://github.com/facebookresearch/higher/issues/58" rel="nofollow noreferrer">here</a>. You can go to the issue to see my reasons but the gist is this:</p>
  
  <p>Wouldn't having that deep copy mean the (outer loop) optimizer would be computing the gradients with respect to parameters no present in the computation graph? Since:</p>
  
  <p>the parameters of the differentiable/inner optimizer are a deep copy compared to the initial parameters/weights
  the outer optimizer (e.g. Adam) would have the original/initial parameters, so the gradient of these should always be zero.
  That is the only explanation that I can think of to explain my issues in the past (gradients being zero unexpectedly) however it seems the higher MAML tutorial works, which should go against my theory. If my theory is right at the end of the inner loop of MAML and when the outer optimizer (usually Adam) computes the gradients, they should be zero (which I have observed sometimes). But I assume they are NOT zero, otherwise that tutorial wouldn't work.</p>
  
  <p>So I am inquiring about the need to use deep copy when creating inner optimizers. What is its purpose and why is it not causing the issues I describe in the original MAML tutorial in higher. How is it that the deep copy doesn't break the forward pass and thus the whole computation of gradient wrt the initialization that the outer optimizer would use?</p>
  
  <hr>
  
  <p>I think at the core of my confusion is that I don't understand why we need to do the <code>deepcopy</code> in the first place. Without all the other code (that seems convoluted to me) we even risk that the initialization we might want to train with the outer optimizer might not train, since the outer/meta optimizer has a pointer to the params of the original model and not a copy of the deep copy the inner optimizer could have had.</p>
  
  <p>Why would the developers go through all that by adding substantial code that seems to have high risks?</p>
  
  <hr>
  
  <p>Related question on how the copying of the initial parameters happens in higher: <a href="https://stackoverflow.com/questions/60311183/what-does-the-copy-initial-weights-documentation-mean-in-the-higher-library-for/62378264?noredirect=1#comment110457791_62378264">What does the copy_initial_weights documentation mean in the higher library for Pytorch?</a></p>