class_docstr: A stacked, bidirectional LSTM which uses
  :class:`~allennlp.modules.lstm_cell_with_projection.LstmCellWithProjection`'s
  with highway layers between the inputs to layers.
  The inputs to the forward and backward directions are independent - forward and backward
  states are not concatenated between layers.
  Additionally, this LSTM maintains its `own` state, which is updated every time
  ``forward`` is called. It is dynamically resized for different batch sizes and is
  designed for use with non-continuous inputs (i.e inputs which aren't formatted as a stream,
  such as text used for a language modelling task, which is how stateful RNNs are typically used).
  This is non-standard, but can be thought of as having an "end of sentence" state, which is
  carried across different sentences.
  Parameters
  ----------
  input_size : ``int``, required
      The dimension of the inputs to the LSTM.
  hidden_size : ``int``, required
      The dimension of the outputs of the LSTM.
  cell_size : ``int``, required.
      The dimension of the memory cell of the
      :class:`~allennlp.modules.lstm_cell_with_projection.LstmCellWithProjection`.
  num_layers : ``int``, required
      The number of bidirectional LSTMs to use.
  requires_grad: ``bool``, optional
      If True, compute gradient of ELMo parameters for fine tuning.
  recurrent_dropout_probability: ``float``, optional (default = 0.0)
      The dropout probability to be used in a dropout scheme as stated in
      `A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
      <https://arxiv.org/abs/1512.05287>`_ .
  state_projection_clip_value: ``float``, optional, (default = None)
      The magnitude with which to clip the hidden_state after projecting it.
  memory_cell_clip_value: ``float``, optional, (default = None)
      The magnitude with which to clip the memory cell.
docstr: Parameters
  ----------
  inputs : ``torch.Tensor``, required.
      A Tensor of shape ``(batch_size, sequence_length, hidden_size)``.
  mask : ``torch.LongTensor``, required.
      A binary mask of shape ``(batch_size, sequence_length)`` representing the
      non-padded elements in each sequence in the batch.
  Returns
  -------
  A ``torch.Tensor`` of shape (num_layers, batch_size, sequence_length, hidden_size),
  where the num_layers dimension represents the LSTM output from that layer.
docstr_tok: ["Parameters","--","--","--","--","--","inputs",":","``","torch.Tensor","``",",","required",".","A","Tensor","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``",".","mask",":","``","torch.LongTensor","``",",","required",".","A","binary","mask","of","shape","``","(","batch_size",",","sequence_length",")","``","representing","the","non-padded","elements","in","each","sequence","in","the","batch",".","Returns","--","--","--","-","A","``","torch.Tensor","``","of","shape","(","num_layers",",","batch_size",",","sequence_length",",","hidden_size",")",",","where","the","num_layers","dimension","represents","the","LSTM","output","from","that","layer","."]
code: def forward(self, inputs, mask):  # pylint: disable=arguments-differ
      """
      Parameters
      ----------
      inputs : ``torch.Tensor``, required.
          A Tensor of shape ``(batch_size, sequence_length, hidden_size)``.
      mask : ``torch.LongTensor``, required.
          A binary mask of shape ``(batch_size, sequence_length)`` representing the
          non-padded elements in each sequence in the batch.
      Returns
      -------
      A ``torch.Tensor`` of shape (num_layers, batch_size, sequence_length, hidden_size),
      where the num_layers dimension represents the LSTM output from that layer.
      """
      batch_size, total_sequence_length = mask.size()
      (
          stacked_sequence_output,
          final_states,
          restoration_indices,
      ) = self.sort_and_run_forward(self._lstm_forward, inputs, mask)
  
      (
          num_layers,
          num_valid,
          returned_timesteps,
          encoder_dim,
      ) = stacked_sequence_output.size()
      # Add back invalid rows which were removed in the call to sort_and_run_forward.
      if num_valid < batch_size:
          zeros = stacked_sequence_output.new_zeros(
              num_layers,
              batch_size - num_valid,
              returned_timesteps,
              encoder_dim,
          )
          stacked_sequence_output = torch.cat(
              [stacked_sequence_output, zeros], 1
          )
  
          # The states also need to have invalid rows added back.
          new_states = []
          for state in final_states:
              state_dim = state.size(-1)
              zeros = state.new_zeros(
                  num_layers, batch_size - num_valid, state_dim
              )
              new_states.append(torch.cat([state, zeros], 1))
          final_states = new_states
  
      # It's possible to need to pass sequences which are padded to longer than the
      # max length of the sequence to a Seq2StackEncoder. However, packing and unpacking
      # the sequences mean that the returned tensor won't include these dimensions, because
      # the RNN did not need to process them. We add them back on in the form of zeros here.
      sequence_length_difference = (
          total_sequence_length - returned_timesteps
      )
      if sequence_length_difference > 0:
          zeros = stacked_sequence_output.new_zeros(
              num_layers,
              batch_size,
              sequence_length_difference,
              stacked_sequence_output[0].size(-1),
          )
          stacked_sequence_output = torch.cat(
              [stacked_sequence_output, zeros], 2
          )
  
      self._update_states(final_states, restoration_indices)
  
      # Restore the original indices and return the sequence.
      # Has shape (num_layers, batch_size, sequence_length, hidden_size)
      return stacked_sequence_output.index_select(
          1, restoration_indices
      )
  
file_path: magnitude/pymagnitude/third_party/allennlp/modules/elmo_lstm.py
imports: import warnings
  import torch
  from torch.nn.utils.rnn import PackedSequence, pad_packed_sequence
  import numpy
  from allennlp.modules.lstm_cell_with_projection import LstmCellWithProjection
  from allennlp.common.checks import ConfigurationError
  from allennlp.modules.encoder_base import _EncoderBase
  from allennlp.common.file_utils import cached_path
label_desc: Use image node_id 2 for calling the ElmoLstm obj's underlying member method code with example usage: obj.forward(inputs, mask) and returns: stacked_sequence_output