class_docstr: Base class for incremental decoders.
  
  Incremental decoding is a special mode at inference time where the Model
  only receives a single timestep of input corresponding to the previous
  output token (for teacher forcing) and must produce the next output
  *incrementally*. Thus the model must cache any long-term state that is
  needed about the sequence, e.g., hidden states, convolutional states, etc.
  
  Compared to the standard :class:`FairseqDecoder` interface, the incremental
  decoder interface allows :func:`forward` functions to take an extra keyword
  argument (*incremental_state*) that can be used to cache state across
  time-steps.
  
  The :class:`FairseqIncrementalDecoder` interface also defines the
  :func:`reorder_incremental_state` method, which is used during beam search
  to select and reorder the incremental state based on the selection of beams.
  
  To learn more about how incremental decoding works, refer to `this blog
  <http://www.telesens.co/2019/04/21/understanding-incremental-decoding-in-fairseq/>`_.
class_docstr_tok: ["Base","class","for","incremental","decoders",".","Incremental","decoding","is","a","special","mode","at","inference","time","where","the","Model","only","receives","a","single","timestep","of","input","corresponding","to","the","previous","output","token","(","for","teacher","forcing",")","and","must","produce","the","next","output","*","incrementally","*",".","Thus","the","model","must","cache","any","long-term","state","that","is","needed","about","the","sequence",",","e.g.",",","hidden","states",",","convolutional","states",",","etc",".","Compared","to","the","standard",":","class",":","`","FairseqDecoder","`","interface",",","the","incremental","decoder","interface","allows",":","func",":","`","forward","`","functions","to","take","an","extra","keyword","argument","(","*","incremental_state","*",")","that","can","be","used","to","cache","state","across","time-steps",".","The",":","class",":","`","FairseqIncrementalDecoder","`","interface","also","defines","the",":","func",":","`","reorder_incremental_state","`","method",",","which","is","used","during","beam","search","to","select","and","reorder","the","incremental","state","based","on","the","selection","of","beams",".","To","learn","more","about","how","incremental","decoding","works",",","refer","to","`","this","blog","<","http",":","\/\/www.telesens.co\/2019\/04\/21\/understanding-incremental-decoding-in-fairseq\/",">","`","_","."]
docstr: Args:
      prev_output_tokens (LongTensor): shifted output tokens of shape
          `(batch, tgt_len)`, for teacher forcing
      encoder_out (dict, optional): output from the encoder, used for
          encoder-side attention
      incremental_state (dict, optional): dictionary used for storing
          state during :ref:`Incremental decoding`
  
  Returns:
      tuple:
          - the decoder's output of shape `(batch, tgt_len, vocab)`
          - a dictionary with any model-specific outputs
docstr_tok: ["Args",":","prev_output_tokens","(","LongTensor",")",":","shifted","output","tokens","of","shape","`","(","batch",",","tgt_len",")","`",",","for","teacher","forcing","encoder_out","(","dict",",","optional",")",":","output","from","the","encoder",",","used","for","encoder-side","attention","incremental_state","(","dict",",","optional",")",":","dictionary","used","for","storing","state","during",":","ref",":","`","Incremental","decoding","`","Returns",":","tuple",":","-","the","decoder","'s","output","of","shape","`","(","batch",",","tgt_len",",","vocab",")","`","-","a","dictionary","with","any","model-specific","outputs"]
code: def forward(
      self,
      prev_output_tokens,
      encoder_out=None,
      incremental_state=None,
      **kwargs,
  ):
      """
      Args:
          prev_output_tokens (LongTensor): shifted output tokens of shape
              `(batch, tgt_len)`, for teacher forcing
          encoder_out (dict, optional): output from the encoder, used for
              encoder-side attention
          incremental_state (dict, optional): dictionary used for storing
              state during :ref:`Incremental decoding`
  
      Returns:
          tuple:
              - the decoder's output of shape `(batch, tgt_len, vocab)`
              - a dictionary with any model-specific outputs
      """
      raise NotImplementedError
  
file_path: fairseq/fairseq/models/fairseq_incremental_decoder.py
imports: import logging
  from typing import Dict, Optional
  from fairseq.incremental_decoding_utils import with_incremental_state
  from fairseq.models import FairseqDecoder
  from torch import Tensor
label_desc: Use image node_id 2 for calling the FairseqIncrementalDecoder obj's underlying member method code with example usage: obj.forward(prev_output_tokens, encoder_out, incremental_state) without return types