class_docstr: This abstract class serves as a base for the 3 ``Encoder`` abstractions in AllenNLP.
  - :class:`~allennlp.modules.seq2seq_encoders.Seq2SeqEncoders`
  - :class:`~allennlp.modules.seq2vec_encoders.Seq2VecEncoders`
  
  Additionally, this class provides functionality for sorting sequences by length
  so they can be consumed by Pytorch RNN classes, which require their inputs to be
  sorted by length. Finally, it also provides optional statefulness to all of it's
  subclasses by allowing the caching and retrieving of the hidden states of RNNs.
docstr: Returns an initial state for use in an RNN. Additionally, this method handles
  the batch size changing across calls by mutating the state to append initial states
  for new elements in the batch. Finally, it also handles sorting the states
  with respect to the sequence lengths of elements in the batch and removing rows
  which are completely padded. Importantly, this `mutates` the state if the
  current batch size is larger than when it was previously called.
  
  Parameters
  ----------
  batch_size : ``int``, required.
      The batch size can change size across calls to stateful RNNs, so we need
      to know if we need to expand or shrink the states before returning them.
      Expanded states will be set to zero.
  num_valid : ``int``, required.
      The batch may contain completely padded sequences which get removed before
      the sequence is passed through the encoder. We also need to clip these off
      of the state too.
  sorting_indices ``torch.LongTensor``, required.
      Pytorch RNNs take sequences sorted by length. When we return the states to be
      used for a given call to ``module.forward``, we need the states to match up to
      the sorted sequences, so before returning them, we sort the states using the
      same indices used to sort the sequences.
  
  Returns
  -------
  This method has a complex return type because it has to deal with the first time it
  is called, when it has no state, and the fact that types of RNN have heterogeneous
  states.
  
  If it is the first time the module has been called, it returns ``None``, regardless
  of the type of the ``Module``.
  
  Otherwise, for LSTMs, it returns a tuple of ``torch.Tensors`` with shape
  ``(num_layers, num_valid, state_size)`` and ``(num_layers, num_valid, memory_size)``
  respectively, or for GRUs, it returns a single ``torch.Tensor`` of shape
  ``(num_layers, num_valid, state_size)``.
docstr_tok: ["Returns","an","initial","state","for","use","in","an","RNN",".","Additionally",",","this","method","handles","the","batch","size","changing","across","calls","by","mutating","the","state","to","append","initial","states","for","new","elements","in","the","batch",".","Finally",",","it","also","handles","sorting","the","states","with","respect","to","the","sequence","lengths","of","elements","in","the","batch","and","removing","rows","which","are","completely","padded",".","Importantly",",","this","`","mutates","`","the","state","if","the","current","batch","size","is","larger","than","when","it","was","previously","called",".","Parameters","--","--","--","--","--","batch_size",":","``","int","``",",","required",".","The","batch","size","can","change","size","across","calls","to","stateful","RNNs",",","so","we","need","to","know","if","we","need","to","expand","or","shrink","the","states","before","returning","them",".","Expanded","states","will","be","set","to","zero",".","num_valid",":","``","int","``",",","required",".","The","batch","may","contain","completely","padded","sequences","which","get","removed","before","the","sequence","is","passed","through","the","encoder",".","We","also","need","to","clip","these","off","of","the","state","too",".","sorting_indices","``","torch.LongTensor","``",",","required",".","Pytorch","RNNs","take","sequences","sorted","by","length",".","When","we","return","the","states","to","be","used","for","a","given","call","to","``","module.forward","``",",","we","need","the","states","to","match","up","to","the","sorted","sequences",",","so","before","returning","them",",","we","sort","the","states","using","the","same","indices","used","to","sort","the","sequences",".","Returns","--","--","--","-","This","method","has","a","complex","return","type","because","it","has","to","deal","with","the","first","time","it","is","called",",","when","it","has","no","state",",","and","the","fact","that","types","of","RNN","have","heterogeneous","states",".","If","it","is","the","first","time","the","module","has","been","called",",","it","returns","``","None","``",",","regardless","of","the","type","of","the","``","Module","``",".","Otherwise",",","for","LSTMs",",","it","returns","a","tuple","of","``","torch.Tensors","``","with","shape","``","(","num_layers",",","num_valid",",","state_size",")","``","and","``","(","num_layers",",","num_valid",",","memory_size",")","``","respectively",",","or","for","GRUs",",","it","returns","a","single","``","torch.Tensor","``","of","shape","``","(","num_layers",",","num_valid",",","state_size",")","``","."]
code: def _get_initial_states(self, batch_size, num_valid, sorting_indices):
      """
      Returns an initial state for use in an RNN. Additionally, this method handles
      the batch size changing across calls by mutating the state to append initial states
      for new elements in the batch. Finally, it also handles sorting the states
      with respect to the sequence lengths of elements in the batch and removing rows
      which are completely padded. Importantly, this `mutates` the state if the
      current batch size is larger than when it was previously called.
  
      Parameters
      ----------
      batch_size : ``int``, required.
          The batch size can change size across calls to stateful RNNs, so we need
          to know if we need to expand or shrink the states before returning them.
          Expanded states will be set to zero.
      num_valid : ``int``, required.
          The batch may contain completely padded sequences which get removed before
          the sequence is passed through the encoder. We also need to clip these off
          of the state too.
      sorting_indices ``torch.LongTensor``, required.
          Pytorch RNNs take sequences sorted by length. When we return the states to be
          used for a given call to ``module.forward``, we need the states to match up to
          the sorted sequences, so before returning them, we sort the states using the
          same indices used to sort the sequences.
  
      Returns
      -------
      This method has a complex return type because it has to deal with the first time it
      is called, when it has no state, and the fact that types of RNN have heterogeneous
      states.
  
      If it is the first time the module has been called, it returns ``None``, regardless
      of the type of the ``Module``.
  
      Otherwise, for LSTMs, it returns a tuple of ``torch.Tensors`` with shape
      ``(num_layers, num_valid, state_size)`` and ``(num_layers, num_valid, memory_size)``
      respectively, or for GRUs, it returns a single ``torch.Tensor`` of shape
      ``(num_layers, num_valid, state_size)``.
      """
      # We don't know the state sizes the first time calling forward,
      # so we let the module define what it's initial hidden state looks like.
      if self._states is None:
          return None
  
      # Otherwise, we have some previous states.
      if batch_size > self._states[0].size(1):
          # This batch is larger than the all previous states.
          # If so, resize the states.
          num_states_to_concat = batch_size - self._states[0].size(1)
          resized_states = []
          # state has shape (num_layers, batch_size, hidden_size)
          for state in self._states:
              # This _must_ be inside the loop because some
              # RNNs have states with different last dimension sizes.
              zeros = state.new_zeros(
                  state.size(0), num_states_to_concat, state.size(2)
              )
              resized_states.append(torch.cat([state, zeros], 1))
          self._states = tuple(resized_states)
          correctly_shaped_states = self._states
  
      elif batch_size < self._states[0].size(1):
          # This batch is smaller than the previous one.
          correctly_shaped_states = tuple(
              state[:, :batch_size, :] for state in self._states
          )
      else:
          correctly_shaped_states = self._states
  
      # At this point, our states are of shape (num_layers, batch_size, hidden_size).
      # However, the encoder uses sorted sequences and additionally removes elements
      # of the batch which are fully padded. We need the states to match up to these
      # sorted and filtered sequences, so we do that in the next two blocks before
      # returning the state/s.
      if len(self._states) == 1:
          # GRUs only have a single state. This `unpacks` it from the
          # tuple and returns the tensor directly.
          correctly_shaped_state = correctly_shaped_states[0]
          sorted_state = correctly_shaped_state.index_select(
              1, sorting_indices
          )
          return sorted_state[:, :num_valid, :]
      else:
          # LSTMs have a state tuple of (state, memory).
          sorted_states = [
              state.index_select(1, sorting_indices)
              for state in correctly_shaped_states
          ]
          return tuple(
              state[:, :num_valid, :] for state in sorted_states
          )
  
file_path: magnitude/pymagnitude/third_party/allennlp/modules/encoder_base.py
imports: from __future__ import absolute_import
  import torch
  from torch.nn.utils.rnn import pack_padded_sequence, PackedSequence
  from allennlp.nn.util import get_lengths_from_binary_sequence_mask, sort_batch_by_length
label_desc: Use image node_id 3 for calling the _EncoderBase obj's underlying member method code with example usage: obj._get_initial_states(batch_size, num_valid, sorting_indices) and returns: None, sorted_state, tuple