class_docstr: A wrapper around the embedding layer, which can be randomly initialized or
  loaded from a .npy file. Also supports normalization of embeddings to have
  zero mean and unit variance (weighted by token frequency) - this is useful
  for example when creating adversarial perturbations of the embeddings that
  should have norms relative to the embeddings' norms.
class_docstr_tok: ["A","wrapper","around","the","embedding","layer",",","which","can","be","randomly","initialized","or","loaded","from","a",".npy","file",".","Also","supports","normalization","of","embeddings","to","have","zero","mean","and","unit","variance","(","weighted","by","token","frequency",")","-","this","is","useful","for","example","when","creating","adversarial","perturbations","of","the","embeddings","that","should","have","norms","relative","to","the","embeddings","'","norms","."]
docstr: Updates the observed mean and variance of the token embeddings. Note
  that these will be weighted by the empirical frequency of each token
  (i.e. common tokens will be more heavily weighted in the params).
docstr_tok: ["Updates","the","observed","mean","and","variance","of","the","token","embeddings",".","Note","that","these","will","be","weighted","by","the","empirical","frequency","of","each","token","(","i.e",".","common","tokens","will","be","more","heavily","weighted","in","the","params",")","."]
code: def _update_normalize_params(self, x):
      """
      Updates the observed mean and variance of the token embeddings. Note
      that these will be weighted by the empirical frequency of each token
      (i.e. common tokens will be more heavily weighted in the params).
      """
      # Flatten x to be a tensor of embeddings.
      assert x.size()[-1:] == self.mean.size()
      x_flattened = x.view(-1, x.size(-1))
  
      # Update mean.
      x_mean = x_flattened.mean(dim=0)
      self.mean.data = (
          self.normalize_decay_rate * self.mean.data
          + (1.0 - self.normalize_decay_rate) * x_mean
      )
  
      # Update variance.
      x_var = ((x_flattened - self.mean) ** 2).mean(dim=0)
      self.var.data = (
          self.normalize_decay_rate * self.var.data
          + (1.0 - self.normalize_decay_rate) * x_var
      )
  
imports: import abc
  import math
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq import utils
  from fairseq.models import FairseqIncrementalDecoder, transformer
  from pytorch_translate import rnn_cell, utils, vocab_reduction
  from pytorch_translate.research.lexical_choice import lexical_translation
label_desc: Use image node_id 4 for calling the Embedding obj's underlying member method code with example usage: obj._update_normalize_params(x) without return types