title: Pytorch BatchNorm2d RuntimeError: running_mean should contain 64 elements not 0
body: <p>I'm using Octave Convolutions and have set up a BatchNorm2d adaptation that for some reasen is giving me</p>
  
  <p><code>RuntimeError: running_mean should contain 64 elements not 0</code> </p>
  
  <p>I've set up some debugging prints to check what was wrong with my Tensors' dimensions, but was unable to find it.
  Here is my class:</p>
  
  <pre><code>class _BatchNorm2d(nn.Module):
    def __init__(self, num_features, alpha_in=0, alpha_out=0, eps=1e-5, momentum=0.1, affine=True,
                 track_running_stats=True):
      super(_BatchNorm2d, self).__init__()
      hf_ch = int(num_features * (1 - alpha_out))
      lf_ch = num_features - hf_ch
      self.bnh = nn.BatchNorm2d(hf_ch)
      self.bnl = nn.BatchNorm2d(lf_ch)
    def forward(self, x):
      if isinstance(x, tuple):
          hf, lf = x
          print("IN ON BN: ",lf.shape if lf is not None else None) #DEBUGGING PRINT
          print(self.bnl)  #DEBUGGING PRINT
          hf = self.bnh(hf) if type(hf) == torch.Tensor else hf
          lf = self.bnh(lf) if type(lf) == torch.Tensor else lf #THIS IS THE LINE ACCUSING THE ERROR
          print("ENDED BN")
          return hf, lf
      else:
          return self.bnh(x)
  </code></pre>
  
  <p>Here is the printing error:</p>
  
  <pre><code>IN ON BN:  torch.Size([32, 64, 3, 3])
  BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  </code></pre>
  
  <p>It seems to me the function should have worked, since x has 64 channels and bn expects 64 channels.</p>
  
  <p>EDIT:
  May also be important to mention the error only happens on alpha value of 1. However, I do not understand it, since the volumes are still the same.</p>