class_docstr: Run a pre-trained bidirectional language model, outputing the activations at each
  layer for weighting together into an ELMo representation (with
  ``allennlp.modules.seq2seq_encoders.Elmo``).  This is a lower level class, useful
  for advanced uses, but most users should use ``allennlp.modules.seq2seq_encoders.Elmo``
  directly.
  
  Parameters
  ----------
  options_file : ``str``
      ELMo JSON options file
  weight_file : ``str``
      ELMo hdf5 weight file
  requires_grad: ``bool``, optional
      If True, compute gradient of ELMo parameters for fine tuning.
  vocab_to_cache : ``List[str]``, optional, (default = 0.5).
      A list of words to pre-compute and cache character convolutions
      for. If you use this option, _ElmoBiLm expects that you pass word
      indices of shape (batch_size, timesteps) to forward, instead
      of character indices. If you use this option and pass a word which
      wasn't pre-cached, this will break.
class_docstr_tok: ["Run","a","pre-trained","bidirectional","language","model",",","outputing","the","activations","at","each","layer","for","weighting","together","into","an","ELMo","representation","(","with","``","allennlp.modules.seq2seq_encoders.Elmo","``",")",".","This","is","a","lower","level","class",",","useful","for","advanced","uses",",","but","most","users","should","use","``","allennlp.modules.seq2seq_encoders.Elmo","``","directly",".","Parameters","--","--","--","--","--","options_file",":","``","str","``","ELMo","JSON","options","file","weight_file",":","``","str","``","ELMo","hdf5","weight","file","requires_grad",":","``","bool","``",",","optional","If","True",",","compute","gradient","of","ELMo","parameters","for","fine","tuning",".","vocab_to_cache",":","``","List","[","str","]","``",",","optional",",","(","default","=","0.5",")",".","A","list","of","words","to","pre-compute","and","cache","character","convolutions","for",".","If","you","use","this","option",",","_ElmoBiLm","expects","that","you","pass","word","indices","of","shape","(","batch_size",",","timesteps",")","to","forward",",","instead","of","character","indices",".","If","you","use","this","option","and","pass","a","word","which","was","n't","pre-cached",",","this","will","break","."]
docstr: Parameters
  ----------
  inputs: ``torch.Tensor``, required.
      Shape ``(batch_size, timesteps, 50)`` of character ids representing the current batch.
  word_inputs : ``torch.Tensor``, required.
      If you passed a cached vocab, you can in addition pass a tensor of shape ``(batch_size, timesteps)``,
      which represent word ids which have been pre-cached.
  
  Returns
  -------
  Dict with keys:
  
  ``'activations'``: ``List[torch.Tensor]``
      A list of activations at each layer of the network, each of shape
      ``(batch_size, timesteps + 2, embedding_dim)``
  ``'mask'``:  ``torch.Tensor``
      Shape ``(batch_size, timesteps + 2)`` long tensor with sequence mask.
  
  Note that the output tensors all include additional special begin and end of sequence
  markers.
code: def forward(
      self, inputs, word_inputs=None  # pylint: disable=arguments-differ
  ):
      """
      Parameters
      ----------
      inputs: ``torch.Tensor``, required.
          Shape ``(batch_size, timesteps, 50)`` of character ids representing the current batch.
      word_inputs : ``torch.Tensor``, required.
          If you passed a cached vocab, you can in addition pass a tensor of shape ``(batch_size, timesteps)``,
          which represent word ids which have been pre-cached.
  
      Returns
      -------
      Dict with keys:
  
      ``'activations'``: ``List[torch.Tensor]``
          A list of activations at each layer of the network, each of shape
          ``(batch_size, timesteps + 2, embedding_dim)``
      ``'mask'``:  ``torch.Tensor``
          Shape ``(batch_size, timesteps + 2)`` long tensor with sequence mask.
  
      Note that the output tensors all include additional special begin and end of sequence
      markers.
      """
      if self._word_embedding is not None and word_inputs is not None:
          try:
              mask_without_bos_eos = (word_inputs > 0).long()
              # The character cnn part is cached - just look it up.
              embedded_inputs = self._word_embedding(word_inputs)  # type: ignore
              # shape (batch_size, timesteps + 2, embedding_dim)
              (
                  type_representation,
                  mask,
              ) = add_sentence_boundary_token_ids(
                  embedded_inputs,
                  mask_without_bos_eos,
                  self._bos_embedding,
                  self._eos_embedding,
              )
          except RuntimeError:
              # Back off to running the character convolutions,
              # as we might not have the words in the cache.
              token_embedding = self._token_embedder(inputs)
              mask = token_embedding["mask"]
              type_representation = token_embedding["token_embedding"]
      else:
          token_embedding = self._token_embedder(inputs)
          mask = token_embedding["mask"]
          type_representation = token_embedding["token_embedding"]
      lstm_outputs = self._elmo_lstm(type_representation, mask)
  
      # Prepare the output.  The first layer is duplicated.
      # Because of minor differences in how masking is applied depending
      # on whether the char cnn layers are cached, we'll be defensive and
      # multiply by the mask here. It's not strictly necessary, as the
      # mask passed on is correct, but the values in the padded areas
      # of the char cnn representations can change.
      output_tensors = [
          torch.cat([type_representation, type_representation], dim=-1)
          * mask.float().unsqueeze(-1)
      ]
      for layer_activations in torch.chunk(
          lstm_outputs, lstm_outputs.size(0), dim=0
      ):
          output_tensors.append(layer_activations.squeeze(0))
  
      return {
          "activations": output_tensors,
          "mask": mask,
      }
  
imports: from __future__ import with_statement
  from __future__ import absolute_import
  import json
  import logging
  import warnings
  import torch
  from torch.nn.modules import Dropout
  import numpy
  from io import open
  from allennlp.common.file_utils import cached_path
  from allennlp.common.checks import ConfigurationError
  from allennlp.common import Params
  from allennlp.common.util import lazy_groups_of
  from allennlp.modules.elmo_lstm import ElmoLstm
  from allennlp.modules.highway import Highway
  from allennlp.modules.scalar_mix import ScalarMix
  from allennlp.nn.util import remove_sentence_boundaries, add_sentence_boundary_token_ids, get_device_of
  from allennlp.data.token_indexers.elmo_indexer import ELMoCharacterMapper, ELMoTokenCharactersIndexer
  from allennlp.data.dataset import Batch
  from allennlp.data import Token, Vocabulary, Instance
  from allennlp.data.fields import TextField
label_desc: Use image node_id 3 for calling the _ElmoBiLm obj's underlying member method code with example usage: obj.forward(inputs, word_inputs) and returns: dict