title: layer Normalization in pytorch?
body: <p>shouldn't the layer normalization of <code>x = torch.tensor([[1.5,0,0,0,0]])</code> be <code>[[1.5,-0.5,-0.5,-0.5]]</code> ? according to this paper <a href="https://arxiv.org/pdf/1607.06450.pdf" rel="noreferrer">paper</a> and the equation from the <a href="https://pytorch.org/docs/stable/nn.html#layernorm" rel="noreferrer">pytorch doc</a>. But the <code>torch.nn.LayerNorm</code> gives <code>[[ 1.7320, -0.5773, -0.5773, -0.5773]]</code></p>
  
  <p>Here is the example code:</p>
  
  <pre><code>x = torch.tensor([[1.5,.0,.0,.0]])
  layerNorm = torch.nn.LayerNorm(4, elementwise_affine = False)
  
  y1 = layerNorm(x)
  
  mean = x.mean(-1, keepdim = True)
  var = x.var(-1, keepdim = True)
  y2 = (x-mean)/torch.sqrt(var+layerNorm.eps)
  </code></pre>
  
  <p>where:</p>
  
  <pre><code>y1 == tensor([[ 1.7320, -0.5773, -0.5773, -0.5773]])
  y2 == tensor([[ 1.5000, -0.5000, -0.5000, -0.5000]])
  </code></pre>