class_docstr: Run a pre-trained bidirectional language model, outputting the activations at each
  layer for weighting together into an ELMo representation (with
  `allennlp.modules.seq2seq_encoders.Elmo`).  This is a lower level class, useful
  for advanced uses, but most users should use `allennlp.modules.Elmo` directly.
  
  # Parameters
  
  options_file : `str`
      ELMo JSON options file
  weight_file : `str`
      ELMo hdf5 weight file
  requires_grad : `bool`, optional, (default = `False`).
      If True, compute gradient of ELMo parameters for fine tuning.
  vocab_to_cache : `List[str]`, optional, (default = `None`).
      A list of words to pre-compute and cache character convolutions
      for. If you use this option, _ElmoBiLm expects that you pass word
      indices of shape (batch_size, timesteps) to forward, instead
      of character indices. If you use this option and pass a word which
      wasn't pre-cached, this will break.
class_docstr_tok: ["Run","a","pre-trained","bidirectional","language","model",",","outputting","the","activations","at","each","layer","for","weighting","together","into","an","ELMo","representation","(","with","`","allennlp.modules.seq2seq_encoders.Elmo","`",")",".","This","is","a","lower","level","class",",","useful","for","advanced","uses",",","but","most","users","should","use","`","allennlp.modules.Elmo","`","directly",".","#","Parameters","options_file",":","`","str","`","ELMo","JSON","options","file","weight_file",":","`","str","`","ELMo","hdf5","weight","file","requires_grad",":","`","bool","`",",","optional",",","(","default","=","`","False","`",")",".","If","True",",","compute","gradient","of","ELMo","parameters","for","fine","tuning",".","vocab_to_cache",":","`","List","[","str","]","`",",","optional",",","(","default","=","`","None","`",")",".","A","list","of","words","to","pre-compute","and","cache","character","convolutions","for",".","If","you","use","this","option",",","_ElmoBiLm","expects","that","you","pass","word","indices","of","shape","(","batch_size",",","timesteps",")","to","forward",",","instead","of","character","indices",".","If","you","use","this","option","and","pass","a","word","which","was","n't","pre-cached",",","this","will","break","."]
docstr: # Parameters
  
  inputs : `torch.Tensor`, required.
      Shape `(batch_size, timesteps, 50)` of character ids representing the current batch.
  word_inputs : `torch.Tensor`, required.
      If you passed a cached vocab, you can in addition pass a tensor of shape `(batch_size, timesteps)`,
      which represent word ids which have been pre-cached.
  
  # Returns
  
  Dict with keys:
  
  `'activations'` : `List[torch.Tensor]`
      A list of activations at each layer of the network, each of shape
      `(batch_size, timesteps + 2, embedding_dim)`
  `'mask'`:  `torch.BoolTensor`
      Shape `(batch_size, timesteps + 2)` long tensor with sequence mask.
  
  Note that the output tensors all include additional special begin and end of sequence
  markers.
code: def forward(
      self, inputs: torch.Tensor, word_inputs: torch.Tensor = None
  ) -> Dict[str, Union[torch.Tensor, List[torch.Tensor]]]:
      """
      # Parameters
  
      inputs : `torch.Tensor`, required.
          Shape `(batch_size, timesteps, 50)` of character ids representing the current batch.
      word_inputs : `torch.Tensor`, required.
          If you passed a cached vocab, you can in addition pass a tensor of shape `(batch_size, timesteps)`,
          which represent word ids which have been pre-cached.
  
      # Returns
  
      Dict with keys:
  
      `'activations'` : `List[torch.Tensor]`
          A list of activations at each layer of the network, each of shape
          `(batch_size, timesteps + 2, embedding_dim)`
      `'mask'`:  `torch.BoolTensor`
          Shape `(batch_size, timesteps + 2)` long tensor with sequence mask.
  
      Note that the output tensors all include additional special begin and end of sequence
      markers.
      """
      if self._word_embedding is not None and word_inputs is not None:
          try:
              mask_without_bos_eos = word_inputs > 0
              # The character cnn part is cached - just look it up.
              embedded_inputs = self._word_embedding(word_inputs)  # type: ignore
              # shape (batch_size, timesteps + 2, embedding_dim)
              (
                  type_representation,
                  mask,
              ) = add_sentence_boundary_token_ids(
                  embedded_inputs,
                  mask_without_bos_eos,
                  self._bos_embedding,
                  self._eos_embedding,
              )
          except (RuntimeError, IndexError):
              # Back off to running the character convolutions,
              # as we might not have the words in the cache.
              token_embedding = self._token_embedder(inputs)
              mask = token_embedding["mask"]
              type_representation = token_embedding["token_embedding"]
      else:
          token_embedding = self._token_embedder(inputs)
          mask = token_embedding["mask"]
          type_representation = token_embedding["token_embedding"]
      lstm_outputs = self._elmo_lstm(type_representation, mask)
  
      # Prepare the output.  The first layer is duplicated.
      # Because of minor differences in how masking is applied depending
      # on whether the char cnn layers are cached, we'll be defensive and
      # multiply by the mask here. It's not strictly necessary, as the
      # mask passed on is correct, but the values in the padded areas
      # of the char cnn representations can change.
      output_tensors = [
          torch.cat([type_representation, type_representation], dim=-1)
          * mask.unsqueeze(-1)
      ]
      for layer_activations in torch.chunk(
          lstm_outputs, lstm_outputs.size(0), dim=0
      ):
          output_tensors.append(layer_activations.squeeze(0))
  
      return {"activations": output_tensors, "mask": mask}
  
imports: import json
  import logging
  import warnings
  from typing import Any, Dict, List, Union
  import numpy
  import torch
  from torch.nn.modules import Dropout
  from allennlp.common import FromParams
  from allennlp.common.checks import ConfigurationError
  from allennlp.common.file_utils import cached_path
  from allennlp.common.util import lazy_groups_of
  from allennlp.data.instance import Instance
  from allennlp.data.tokenizers.token_class import Token
  from allennlp.data.vocabulary import Vocabulary
  from allennlp.data.batch import Batch
  from allennlp.data.fields import TextField
  from allennlp.data.token_indexers.elmo_indexer import ELMoCharacterMapper, ELMoTokenCharactersIndexer
  from allennlp.modules.elmo_lstm import ElmoLstm
  from allennlp.modules.highway import Highway
  from allennlp.modules.scalar_mix import ScalarMix
  from allennlp.nn.util import add_sentence_boundary_token_ids, get_device_of, remove_sentence_boundaries
label_desc: Use image node_id 3 for calling the _ElmoBiLm obj's underlying member method code with example usage: obj.forward(inputs, word_inputs) and returns: dict