title: Is it mandatory in pytorch to add modules to ModuleList to access its parameters
body: <p>I read some posts about ModuleList and all of them said that adding modules to ModuleList gives access to parameters of the Neural Network but in “Training a classifier” example of <strong>60 mins blitz</strong> pytorch tutorial the modules are not added to any ModuleList and still the parameters could be accessed using</p>
  
  <p><code>optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)</code></p>
  
  <p>This is confusing. Please clarify how the parameters are accessible even though the modules have not been added to any ModuleList</p>
  
  <pre><code>class mmodel(nn.Module):
      def __init__(self):
          super(mmodel,self).__init__()
          self.lst=[]
          self.lst.append(nn.Linear(1,1))
          self.lst.append(nn.Linear(1,1))
          self.mlist = nn.ModuleList([nn.Linear(2,1,bias=False),nn.Linear(3,1)])
      def forward(self,x):
          for m in self.lst:
              print(type(m))
              x = m(x)
          for m in self.mlist:
              print(type(m))
  
  mm = mmodel()
  mm(X)
  print(list(mm.parameters())) 
  
  &lt;class 'torch.nn.modules.linear.Linear'&gt;
  &lt;class 'torch.nn.modules.linear.Linear'&gt;
  &lt;class 'torch.nn.modules.linear.Linear'&gt;
  &lt;class 'torch.nn.modules.linear.Linear'&gt;
  [Parameter containing:
  tensor([[0.2302, 0.3712]], requires_grad=True), Parameter containing:
  tensor([[-0.3451, -0.0274,  0.3990]], requires_grad=True), Parameter containing:
  tensor([0.3258], requires_grad=True)]
  </code></pre>
  
  <p>As evident from the above outputs the parameters of modules added in a python list are not visible in the forward method. Only the parameters of the modules added in the ModuleList are visible in the forward method</p>