title: Pytorch customize weight
body: <p>I have a network</p>
  
  <pre><code>class Net(nn.Module)
  </code></pre>
  
  <p>and two different weights <code>w0</code> and <code>w1</code> (concatenate weights of all layers into a vector). Now I want to optimize the network on the line connecting <code>w0</code> and <code>w1</code>, which means that the weight will have the form <code>theta  * w0 + (1-theta) * w1</code>. So now the parameter I want to optimize is no longer the weight itself, but the <code>theta</code>. </p>
  
  <p>How can I implement this? In Pytorch, how can I define the parameter to be <code>theta</code>, and set the weight to be form I want. To be specific, if I create a new class </p>
  
  <pre><code>NetOnLine(nn.Module)
  </code></pre>
  
  <p>how should I write the <code>forward(self, X)</code> function?</p>