title: Correct way to register a parameter for model in Pytorch
body: <p>I tried to define a simple model in Pytorch. The model computes negative log prob for a gaussian distribution:</p>
  <pre><code>import torch
  import torch.nn as nn
  
  class GaussianModel(nn.Module):
  
      def __init__(self):
          super(GaussianModel, self).__init__()
  
          self.register_parameter('mean', nn.Parameter(torch.zeros(1),
                                                       requires_grad=True))
          
          self.pdf = torch.distributions.Normal(self.state_dict()['mean'],
                                                torch.tensor([1.0]))
      def forward(self, x):
          return -self.pdf.log_prob(x)
  
  model = GaussianModel()
  </code></pre>
  <p>Then I tried to optimize the <code>mean</code> parameter:</p>
  <pre><code>optimizer = torch.optim.SGD(model.parameters(), lr=0.002)
  for _ in range(5):
    optimizer.zero_grad()
    nll = model(torch.tensor([3.0], requires_grad=True))
    nll.backward()
    optimizer.step()
    print('mean : ', model.state_dict()['mean'],
                   ' - Negative Loglikelihood : ', nll.item())
  </code></pre>
  <p>But it seems the gradient is zero and <code>mean</code> does not change:</p>
  <pre><code>mean :  tensor([0.])  - Negative Loglikelihood :  5.418938636779785
  mean :  tensor([0.])  - Negative Loglikelihood :  5.418938636779785
  mean :  tensor([0.])  - Negative Loglikelihood :  5.418938636779785
  mean :  tensor([0.])  - Negative Loglikelihood :  5.418938636779785
  mean :  tensor([0.])  - Negative Loglikelihood :  5.418938636779785
  </code></pre>
  <p>Did I register and use the <code>mean</code> parameter correctly? can autograd compute the gradient for <code>torch.distributions.Normal.log_prob</code> or I should implement the <code>backward()</code> for the model?</p>