class_docstr: A more featureful embedding module than the default in Pytorch.  Adds the ability to:
  
      1. embed higher-order inputs
      2. pre-specify the weight matrix
      3. use a non-trainable embedding
      4. project the resultant embeddings to some other dimension (which only makes sense with
         non-trainable embeddings).
  
  Note that if you are using our data API and are trying to embed a
  [`TextField`](../../data/fields/text_field.md), you should use a
  [`TextFieldEmbedder`](../text_field_embedders/text_field_embedder.md) instead of using this directly.
  
  Registered as a `TokenEmbedder` with name "embedding".
  
  # Parameters
  
  num_embeddings : `int`
      Size of the dictionary of embeddings (vocabulary size).
  embedding_dim : `int`
      The size of each embedding vector.
  projection_dim : `int`, optional (default=`None`)
      If given, we add a projection layer after the embedding layer.  This really only makes
      sense if `trainable` is `False`.
  weight : `torch.FloatTensor`, optional (default=`None`)
      A pre-initialised weight matrix for the embedding lookup, allowing the use of
      pretrained vectors.
  padding_index : `int`, optional (default=`None`)
      If given, pads the output with zeros whenever it encounters the index.
  trainable : `bool`, optional (default=`True`)
      Whether or not to optimize the embedding parameters.
  max_norm : `float`, optional (default=`None`)
      If given, will renormalize the embeddings to always have a norm lesser than this
  norm_type : `float`, optional (default=`2`)
      The p of the p-norm to compute for the max_norm option
  scale_grad_by_freq : `bool`, optional (default=`False`)
      If given, this will scale gradients by the frequency of the words in the mini-batch.
  sparse : `bool`, optional (default=`False`)
      Whether or not the Pytorch backend should use a sparse representation of the embedding weight.
  vocab_namespace : `str`, optional (default=`None`)
      In case of fine-tuning/transfer learning, the model's embedding matrix needs to be
      extended according to the size of extended-vocabulary. To be able to know how much to
      extend the embedding-matrix, it's necessary to know which vocab_namspace was used to
      construct it in the original training. We store vocab_namespace used during the original
      training as an attribute, so that it can be retrieved during fine-tuning.
  pretrained_file : `str`, optional (default=`None`)
      Path to a file of word vectors to initialize the embedding matrix. It can be the
      path to a local file or a URL of a (cached) remote file. Two formats are supported:
          * hdf5 file - containing an embedding matrix in the form of a torch.Tensor;
          * text file - an utf-8 encoded text file with space separated fields.
  vocab : `Vocabulary`, optional (default = `None`)
      Used to construct an embedding from a pretrained file.
  
      In a typical AllenNLP configuration file, this parameter does not get an entry under the
      "embedding", it gets specified as a top-level parameter, then is passed in to this module
      separately.
  
  # Returns
  
  An Embedding module.
class_docstr_tok: ["A","more","featureful","embedding","module","than","the","default","in","Pytorch",".","Adds","the","ability","to",":","1.","embed","higher-order","inputs","2.","pre-specify","the","weight","matrix","3.","use","a","non-trainable","embedding","4.","project","the","resultant","embeddings","to","some","other","dimension","(","which","only","makes","sense","with","non-trainable","embeddings",")",".","Note","that","if","you","are","using","our","data","API","and","are","trying","to","embed","a","[","`","TextField","`","]","(","..","\/","..","\/data\/fields\/text_field.md",")",",","you","should","use","a","[","`","TextFieldEmbedder","`","]","(","..","\/text_field_embedders\/text_field_embedder.md",")","instead","of","using","this","directly",".","Registered","as","a","`","TokenEmbedder","`","with","name","``","embedding","''",".","#","Parameters","num_embeddings",":","`","int","`","Size","of","the","dictionary","of","embeddings","(","vocabulary","size",")",".","embedding_dim",":","`","int","`","The","size","of","each","embedding","vector",".","projection_dim",":","`","int","`",",","optional","(","default=","`","None","`",")","If","given",",","we","add","a","projection","layer","after","the","embedding","layer",".","This","really","only","makes","sense","if","`","trainable","`","is","`","False","`",".","weight",":","`","torch.FloatTensor","`",",","optional","(","default=","`","None","`",")","A","pre-initialised","weight","matrix","for","the","embedding","lookup",",","allowing","the","use","of","pretrained","vectors",".","padding_index",":","`","int","`",",","optional","(","default=","`","None","`",")","If","given",",","pads","the","output","with","zeros","whenever","it","encounters","the","index",".","trainable",":","`","bool","`",",","optional","(","default=","`","True","`",")","Whether","or","not","to","optimize","the","embedding","parameters",".","max_norm",":","`","float","`",",","optional","(","default=","`","None","`",")","If","given",",","will","renormalize","the","embeddings","to","always","have","a","norm","lesser","than","this","norm_type",":","`","float","`",",","optional","(","default=","`","2","`",")","The","p","of","the","p-norm","to","compute","for","the","max_norm","option","scale_grad_by_freq",":","`","bool","`",",","optional","(","default=","`","False","`",")","If","given",",","this","will","scale","gradients","by","the","frequency","of","the","words","in","the","mini-batch",".","sparse",":","`","bool","`",",","optional","(","default=","`","False","`",")","Whether","or","not","the","Pytorch","backend","should","use","a","sparse","representation","of","the","embedding","weight",".","vocab_namespace",":","`","str","`",",","optional","(","default=","`","None","`",")","In","case","of","fine-tuning\/transfer","learning",",","the","model","'s","embedding","matrix","needs","to","be","extended","according","to","the","size","of","extended-vocabulary",".","To","be","able","to","know","how","much","to","extend","the","embedding-matrix",",","it","'s","necessary","to","know","which","vocab_namspace","was","used","to","construct","it","in","the","original","training",".","We","store","vocab_namespace","used","during","the","original","training","as","an","attribute",",","so","that","it","can","be","retrieved","during","fine-tuning",".","pretrained_file",":","`","str","`",",","optional","(","default=","`","None","`",")","Path","to","a","file","of","word","vectors","to","initialize","the","embedding","matrix",".","It","can","be","the","path","to","a","local","file","or","a","URL","of","a","(","cached",")","remote","file",".","Two","formats","are","supported",":","*","hdf5","file","-","containing","an","embedding","matrix","in","the","form","of","a","torch.Tensor",";","*","text","file","-","an","utf-8","encoded","text","file","with","space","separated","fields",".","vocab",":","`","Vocabulary","`",",","optional","(","default","=","`","None","`",")","Used","to","construct","an","embedding","from","a","pretrained","file",".","In","a","typical","AllenNLP","configuration","file",",","this","parameter","does","not","get","an","entry","under","the","``","embedding","''",",","it","gets","specified","as","a","top-level","parameter",",","then","is","passed","in","to","this","module","separately",".","#","Returns","An","Embedding","module","."]
docstr: Extends the embedding matrix according to the extended vocabulary.
  If extension_pretrained_file is available, it will be used for initializing the new words
  embeddings in the extended vocabulary; otherwise we will check if _pretrained_file attribute
  is already available. If none is available, they will be initialized with xavier uniform.
  
  # Parameters
  
  extended_vocab : `Vocabulary`
      Vocabulary extended from original vocabulary used to construct
      this `Embedding`.
  vocab_namespace : `str`, (optional, default=`None`)
      In case you know what vocab_namespace should be used for extension, you
      can pass it. If not passed, it will check if vocab_namespace used at the
      time of `Embedding` construction is available. If so, this namespace
      will be used or else extend_vocab will be a no-op.
  extension_pretrained_file : `str`, (optional, default=`None`)
      A file containing pretrained embeddings can be specified here. It can be
      the path to a local file or an URL of a (cached) remote file. Check format
      details in `from_params` of `Embedding` class.
  model_path : `str`, (optional, default=`None`)
      Path traversing the model attributes upto this embedding module.
      Eg. "_text_field_embedder.token_embedder_tokens". This is only useful
      to give a helpful error message when extend_vocab is implicitly called
      by train or any other command.
docstr_tok: ["Extends","the","embedding","matrix","according","to","the","extended","vocabulary",".","If","extension_pretrained_file","is","available",",","it","will","be","used","for","initializing","the","new","words","embeddings","in","the","extended","vocabulary",";","otherwise","we","will","check","if","_pretrained_file","attribute","is","already","available",".","If","none","is","available",",","they","will","be","initialized","with","xavier","uniform",".","#","Parameters","extended_vocab",":","`","Vocabulary","`","Vocabulary","extended","from","original","vocabulary","used","to","construct","this","`","Embedding","`",".","vocab_namespace",":","`","str","`",",","(","optional",",","default=","`","None","`",")","In","case","you","know","what","vocab_namespace","should","be","used","for","extension",",","you","can","pass","it",".","If","not","passed",",","it","will","check","if","vocab_namespace","used","at","the","time","of","`","Embedding","`","construction","is","available",".","If","so",",","this","namespace","will","be","used","or","else","extend_vocab","will","be","a","no-op",".","extension_pretrained_file",":","`","str","`",",","(","optional",",","default=","`","None","`",")","A","file","containing","pretrained","embeddings","can","be","specified","here",".","It","can","be","the","path","to","a","local","file","or","an","URL","of","a","(","cached",")","remote","file",".","Check","format","details","in","`","from_params","`","of","`","Embedding","`","class",".","model_path",":","`","str","`",",","(","optional",",","default=","`","None","`",")","Path","traversing","the","model","attributes","upto","this","embedding","module",".","Eg",".","``","_text_field_embedder.token_embedder_tokens","''",".","This","is","only","useful","to","give","a","helpful","error","message","when","extend_vocab","is","implicitly","called","by","train","or","any","other","command","."]
code: def extend_vocab(
      self,
      extended_vocab: Vocabulary,
      vocab_namespace: str = None,
      extension_pretrained_file: str = None,
      model_path: str = None,
  ):
      """
      Extends the embedding matrix according to the extended vocabulary.
      If extension_pretrained_file is available, it will be used for initializing the new words
      embeddings in the extended vocabulary; otherwise we will check if _pretrained_file attribute
      is already available. If none is available, they will be initialized with xavier uniform.
  
      # Parameters
  
      extended_vocab : `Vocabulary`
          Vocabulary extended from original vocabulary used to construct
          this `Embedding`.
      vocab_namespace : `str`, (optional, default=`None`)
          In case you know what vocab_namespace should be used for extension, you
          can pass it. If not passed, it will check if vocab_namespace used at the
          time of `Embedding` construction is available. If so, this namespace
          will be used or else extend_vocab will be a no-op.
      extension_pretrained_file : `str`, (optional, default=`None`)
          A file containing pretrained embeddings can be specified here. It can be
          the path to a local file or an URL of a (cached) remote file. Check format
          details in `from_params` of `Embedding` class.
      model_path : `str`, (optional, default=`None`)
          Path traversing the model attributes upto this embedding module.
          Eg. "_text_field_embedder.token_embedder_tokens". This is only useful
          to give a helpful error message when extend_vocab is implicitly called
          by train or any other command.
      """
      # Caveat: For allennlp v0.8.1 and below, we weren't storing vocab_namespace as an attribute,
      # knowing which is necessary at time of embedding vocab extension. So old archive models are
      # currently unextendable.
  
      vocab_namespace = vocab_namespace or self._vocab_namespace
      if not vocab_namespace:
          # It's not safe to default to "tokens" or any other namespace.
          logger.info(
              "Loading a model trained before embedding extension was implemented; "
              "pass an explicit vocab namespace if you want to extend the vocabulary."
          )
          return
  
      extended_num_embeddings = extended_vocab.get_vocab_size(
          vocab_namespace
      )
      if extended_num_embeddings == self.num_embeddings:
          # It's already been extended. No need to initialize / read pretrained file in first place (no-op)
          return
  
      if extended_num_embeddings < self.num_embeddings:
          raise ConfigurationError(
              f"Size of namespace, {vocab_namespace} for extended_vocab is smaller than "
              f"embedding. You likely passed incorrect vocab or namespace for extension."
          )
  
      # Case 1: user passed extension_pretrained_file and it's available.
      if extension_pretrained_file and is_url_or_existing_file(
          extension_pretrained_file
      ):
          # Don't have to do anything here, this is the happy case.
          pass
      # Case 2: user passed extension_pretrained_file and it's not available
      elif extension_pretrained_file:
          raise ConfigurationError(
              f"You passed pretrained embedding file {extension_pretrained_file} "
              f"for model_path {model_path} but it's not available."
          )
      # Case 3: user didn't pass extension_pretrained_file, but pretrained_file attribute was
      # saved during training and is available.
      elif is_url_or_existing_file(self._pretrained_file):
          extension_pretrained_file = self._pretrained_file
      # Case 4: no file is available, hope that pretrained embeddings weren't used in the first place and warn
      elif self._pretrained_file is not None:
          # Warn here instead of an exception to allow a fine-tuning even without the original pretrained_file
          logger.warning(
              f"Embedding at model_path, {model_path} cannot locate the pretrained_file. "
              f"Originally pretrained_file was at '{self._pretrained_file}'."
          )
      else:
          # When loading a model from archive there is no way to distinguish between whether a pretrained-file
          # was or wasn't used during the original training. So we leave an info here.
          logger.info(
              "If you are fine-tuning and want to use a pretrained_file for "
              "embedding extension, please pass the mapping by --embedding-sources argument."
          )
  
      embedding_dim = self.weight.data.shape[-1]
      if not extension_pretrained_file:
          extra_num_embeddings = (
              extended_num_embeddings - self.num_embeddings
          )
          extra_weight = torch.FloatTensor(
              extra_num_embeddings, embedding_dim
          )
          torch.nn.init.xavier_uniform_(extra_weight)
      else:
          # It's easiest to just reload the embeddings for the entire vocab,
          # then only keep the ones we need.
          whole_weight = _read_pretrained_embeddings_file(
              extension_pretrained_file,
              embedding_dim,
              extended_vocab,
              vocab_namespace,
          )
          extra_weight = whole_weight[self.num_embeddings :, :]
  
      device = self.weight.data.device
      extended_weight = torch.cat(
          [self.weight.data, extra_weight.to(device)], dim=0
      )
      self.weight = torch.nn.Parameter(
          extended_weight, requires_grad=self.weight.requires_grad
      )
      self.num_embeddings = extended_num_embeddings
  
imports: import io
  import itertools
  import logging
  import re
  import tarfile
  import warnings
  import zipfile
  from typing import Any, cast, Iterator, NamedTuple, Optional, Sequence, Tuple, BinaryIO
  import numpy
  import torch
  from torch.nn.functional import embedding
  from allennlp.common import Tqdm
  from allennlp.common.checks import ConfigurationError
  from allennlp.common.file_utils import cached_path, get_file_extension, is_url_or_existing_file
  from allennlp.data.vocabulary import Vocabulary
  from allennlp.modules.time_distributed import TimeDistributed
  from allennlp.modules.token_embedders.token_embedder import TokenEmbedder
  from allennlp.nn import util
label_desc: Use image node_id 4 for calling the Embedding obj's underlying member method code with example usage: obj.extend_vocab(extended_vocab, vocab_namespace, extension_pretrained_file, model_path) without return types