class_docstr: Base class for all RNN-like sequence models.
  
  DP-friendly drop-in replacement of the ``torch.nn.RNNBase`` module.
  After training this module can be exported and loaded by the original ``torch.nn``
  implementation for inference.
  
  This module implements multi-layer (Type-2, see
  [this issue](https://github.com/pytorch/pytorch/issues/4930#issuecomment-361851298))
  bi-directional sequential model based on abstract cell.
  Cell should be a subclass of ``DPRNNCellBase``.
  
  Limitations:
  - proj_size > 0 is not implemented
  - this implementation doesn't use cuDNN
class_docstr_tok: ["Base","class","for","all","RNN-like","sequence","models",".","DP-friendly","drop-in","replacement","of","the","``","torch.nn.RNNBase","``","module",".","After","training","this","module","can","be","exported","and","loaded","by","the","original","``","torch.nn","``","implementation","for","inference",".","This","module","implements","multi-layer","(","Type-2",",","see","[","this","issue","]","(","https",":","\/\/github.com\/pytorch\/pytorch\/issues\/4930","#","issuecomment-361851298",")",")","bi-directional","sequential","model","based","on","abstract","cell",".","Cell","should","be","a","subclass","of","``","DPRNNCellBase","``",".","Limitations",":","-","proj_size",">","0","is","not","implemented","-","this","implementation","does","n't","use","cuDNN"]
docstr: Forward pass of a single RNN layer (one direction). Implemented for an abstract cell type.
  
  Inputs: x, h_0, c_0
      x: Input sequence. Tensor of shape ``[T, B, D]`` or PackedSequence if `is_packed = True`.
      h_0: Initial hidden state. Tensor of shape ``[B, H]``.
      c_0: Initial cell state. Tensor of shape ``[B, H]``. Only for cells with additional
           state `c_t`, e.g. DPLSTMCell.
  
  Outputs: h_t, h_last, c_last
      h_t: Final hidden state, output features (``h_t``) for each timestep ``t``. Tensor of
          shape ``[T, B, H]`` or list of length ``T`` with tensors ``[B, H]`` if PackedSequence is used.
      h_last: The last hidden state. Tensor of shape ``[B, H]``.
      c_last: The last cell state. Tensor of shape ``[B, H]``. None if cell has no additional state.
  
  where
      T = sequence length
      B = batch size
      D = input_size (for this specific layer)
      H = hidden_size (output size, for this specific layer)
  
  Args:
      batch_sizes: Contains the batch sizes as stored in PackedSequence
      cell: Module implementing a single cell of the network, must be an instance of DPRNNCell
      max_batch_size: batch size
      seq_length: sequence length
      is_packed: whether PackedSequence is used as input
      reverse_layer: if True, it will run forward pass for a reversed layer
docstr_tok: ["Forward","pass","of","a","single","RNN","layer","(","one","direction",")",".","Implemented","for","an","abstract","cell","type",".","Inputs",":","x",",","h_0",",","c_0","x",":","Input","sequence",".","Tensor","of","shape","``","[","T",",","B",",","D","]","``","or","PackedSequence","if","`","is_packed","=","True","`",".","h_0",":","Initial","hidden","state",".","Tensor","of","shape","``","[","B",",","H","]","``",".","c_0",":","Initial","cell","state",".","Tensor","of","shape","``","[","B",",","H","]","``",".","Only","for","cells","with","additional","state","`","c_t","`",",","e.g",".","DPLSTMCell",".","Outputs",":","h_t",",","h_last",",","c_last","h_t",":","Final","hidden","state",",","output","features","(","``","h_t","``",")","for","each","timestep","``","t","``",".","Tensor","of","shape","``","[","T",",","B",",","H","]","``","or","list","of","length","``","T","``","with","tensors","``","[","B",",","H","]","``","if","PackedSequence","is","used",".","h_last",":","The","last","hidden","state",".","Tensor","of","shape","``","[","B",",","H","]","``",".","c_last",":","The","last","cell","state",".","Tensor","of","shape","``","[","B",",","H","]","``",".","None","if","cell","has","no","additional","state",".","where","T","=","sequence","length","B","=","batch","size","D","=","input_size","(","for","this","specific","layer",")","H","=","hidden_size","(","output","size",",","for","this","specific","layer",")","Args",":","batch_sizes",":","Contains","the","batch","sizes","as","stored","in","PackedSequence","cell",":","Module","implementing","a","single","cell","of","the","network",",","must","be","an","instance","of","DPRNNCell","max_batch_size",":","batch","size","seq_length",":","sequence","length","is_packed",":","whether","PackedSequence","is","used","as","input","reverse_layer",":","if","True",",","it","will","run","forward","pass","for","a","reversed","layer"]
code: def forward_layer(
      self,
      x: Union[Tensor, PackedSequence],
      h_0: Tensor,
      c_0: Optional[Tensor],
      batch_sizes: Tensor,
      cell: DPRNNCellBase,
      max_batch_size: int,
      seq_length: int,
      is_packed: bool,
      reverse_layer: bool,
  ) -> Tuple[Union[Tensor, List[Tensor]], Tensor, Tensor]:
      """
      Forward pass of a single RNN layer (one direction). Implemented for an abstract cell type.
  
      Inputs: x, h_0, c_0
          x: Input sequence. Tensor of shape ``[T, B, D]`` or PackedSequence if `is_packed = True`.
          h_0: Initial hidden state. Tensor of shape ``[B, H]``.
          c_0: Initial cell state. Tensor of shape ``[B, H]``. Only for cells with additional
               state `c_t`, e.g. DPLSTMCell.
  
      Outputs: h_t, h_last, c_last
          h_t: Final hidden state, output features (``h_t``) for each timestep ``t``. Tensor of
              shape ``[T, B, H]`` or list of length ``T`` with tensors ``[B, H]`` if PackedSequence is used.
          h_last: The last hidden state. Tensor of shape ``[B, H]``.
          c_last: The last cell state. Tensor of shape ``[B, H]``. None if cell has no additional state.
  
      where
          T = sequence length
          B = batch size
          D = input_size (for this specific layer)
          H = hidden_size (output size, for this specific layer)
  
      Args:
          batch_sizes: Contains the batch sizes as stored in PackedSequence
          cell: Module implementing a single cell of the network, must be an instance of DPRNNCell
          max_batch_size: batch size
          seq_length: sequence length
          is_packed: whether PackedSequence is used as input
          reverse_layer: if True, it will run forward pass for a reversed layer
  
      """
      if is_packed:
          if reverse_layer:
              x = tuple(reversed(x))
              batch_sizes = batch_sizes.flip(0)
      else:
          if reverse_layer:
              x = x.flip(0)
          x = torch.unbind(x, dim=0)
  
      h_n = [h_0]
      c_n = [c_0]
      c_next = c_0
      batch_size_prev = h_0.shape[0]
  
      for t in range(seq_length):
          if is_packed:
              batch_size_t = batch_sizes[t].item()
              delta = batch_size_t - batch_size_prev
              if delta > 0:
                  h_cat = torch.cat(
                      (h_n[t], h_0[batch_size_prev:batch_size_t, :]), 0
                  )
                  if self.has_cell_state:
                      c_cat = torch.cat(
                          (
                              c_n[t],
                              c_0[batch_size_prev:batch_size_t, :],
                          ),
                          0,
                      )
                      h_next, c_next = cell(
                          x[t], (h_cat, c_cat), batch_size_t
                      )
                  else:
                      h_next = cell(x[t], h_cat, batch_size_t)
              else:
                  if self.has_cell_state:
                      h_next, c_next = cell(
                          x[t], (h_n[t], c_n[t]), batch_size_t
                      )
                  else:
                      h_next = cell(x[t], h_n[t], batch_size_t)
          else:
              if self.has_cell_state:
                  h_next, c_next = cell(x[t], (h_n[t], c_n[t]))
              else:
                  h_next = cell(x[t], h_n[t])
  
          if self.dropout:
              h_next = self.dropout_layer(h_next)
  
          h_n.append(h_next)
          c_n.append(c_next)
          batch_size_prev = h_next.shape[0]
  
      if is_packed:
          h_temp = h_n[1:]  # list T x [B, H]
          c_temp = c_n[1:]
  
          # Collect last states for all sequences
          seq_lengths = compute_seq_lengths(batch_sizes)
          h_last = torch.zeros(
              max_batch_size, self.hidden_size
          )  # [B, H]
          c_last = (
              torch.zeros(max_batch_size, self.hidden_size)
              if self.has_cell_state
              else None
          )
          for i, seq_len in enumerate(seq_lengths):
              h_last[i, :] = h_temp[seq_len - 1][i, :]
              if self.has_cell_state:
                  c_last[i, :] = c_temp[seq_len - 1][i, :]
          if reverse_layer:
              h_temp = tuple(reversed(h_temp))
  
      else:
          h_n = torch.stack(
              h_n[1:], dim=0
          )  # [T, B, H], init step not part of output
          h_temp = (
              h_n if not reverse_layer else h_n.flip(0)
          )  # Flip the output...
          h_last = h_n[-1]  # ... But not the states
          c_last = c_n[-1]
  
      return h_temp, h_last, c_last
  
imports: import math
  import numbers
  import warnings
  from typing import List, Optional, Tuple, Type, Union
  import torch
  import torch.nn
  from torch import Tensor
  from torch.nn.utils.rnn import PackedSequence
  from ..utils.packed_sequences import compute_seq_lengths
  from .param_rename import RenameParamsMixin
label_desc: Use image node_id 3 for calling the DPRNNBase obj's underlying member method code with example usage: obj.forward_layer(x, h_0, c_0, batch_sizes, cell, max_batch_size, seq_length, is_packed, reverse_layer) and returns: h_temp,  h_last,  c_last