docstr: Args:
      input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
          Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you
          provide it.
  
          Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and
          [`PreTrainedTokenizer.__call__`] for details.
  
          [What are input IDs?](../glossary#input-ids)
      attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*):
          Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`:
  
          - 1 for tokens that are **not masked**,
          - 0 for tokens that are **masked**.
  
          [What are attention masks?](../glossary#attention-mask)
      encoder_hidden_states  (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):
          Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention
          if the model is configured as a decoder.
      encoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*):
          Mask to avoid performing attention on the padding token indices of the encoder input. This mask is used
          in the cross-attention if the model is configured as a decoder. Mask values selected in `[0, 1]`:
      head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*):
          Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`:
  
          - 1 indicates the head is **not masked**,
          - 0 indicates the head is **masked**.
  
      cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*):
          Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`:
  
          - 1 indicates the head is **not masked**,
          - 0 indicates the head is **masked**.
  
      past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):
          Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of
          shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of
          shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. The two additional
          tensors are only required when the model is used as a decoder in a Sequence to Sequence model.
  
          Contains pre-computed hidden-states (key and values in the self-attention blocks and in the
          cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.
  
          If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those
          that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of
          all `decoder_input_ids` of shape `(batch_size, sequence_length)`.
      labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
          Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
          config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
          (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
      use_cache (`bool`, *optional*):
          If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding
          (see `past_key_values`).
  
          - 1 for tokens that are **not masked**,
          - 0 for tokens that are **masked**.
      output_attentions (`bool`, *optional*):
          Whether or not to return the attentions tensors of all attention layers. See `attentions` under
          returned tensors for more detail.
      output_hidden_states (`bool`, *optional*):
          Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
          for more detail.
      return_dict (`bool`, *optional*):
          Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
  
  Returns:
  
  Example:
  
  ```python
  >>> from transformers import (
  ...     TrOCRConfig,
  ...     TrOCRProcessor,
  ...     TrOCRForCausalLM,
  ...     ViTConfig,
  ...     ViTModel,
  ...     VisionEncoderDecoderModel,
  ... )
  >>> import requests
  >>> from PIL import Image
  
  >>> # TrOCR is a decoder model and should be used within a VisionEncoderDecoderModel
  >>> # init vision2text model with random weights
  >>> encoder = ViTModel(ViTConfig())
  >>> decoder = TrOCRForCausalLM(TrOCRConfig())
  >>> model = VisionEncoderDecoderModel(encoder=encoder, decoder=decoder)
  
  >>> # If you want to start from the pretrained model, load the checkpoint with `VisionEncoderDecoderModel`
  >>> processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")
  >>> model = VisionEncoderDecoderModel.from_pretrained("microsoft/trocr-base-handwritten")
  
  >>> # load image from the IAM dataset
  >>> url = "https://fki.tic.heia-fr.ch/static/img/a01-122-02.jpg"
  >>> image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
  >>> pixel_values = processor(image, return_tensors="pt").pixel_values
  >>> text = "industry, ' Mr. Brown commented icily. ' Let us have a"
  
  >>> # training
  >>> model.config.decoder_start_token_id = processor.tokenizer.cls_token_id
  >>> model.config.pad_token_id = processor.tokenizer.pad_token_id
  >>> model.config.vocab_size = model.config.decoder.vocab_size
  
  >>> labels = processor.tokenizer(text, return_tensors="pt").input_ids
  >>> outputs = model(pixel_values, labels=labels)
  >>> loss = outputs.loss
  >>> round(loss.item(), 2)
  5.30
  
  >>> # inference
  >>> generated_ids = model.generate(pixel_values)
  >>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
  >>> generated_text
  'industry, " Mr. Brown commented icily. " Let us have a'
  ```
code: def forward(
      self,
      input_ids: Optional[torch.LongTensor] = None,
      attention_mask: Optional[torch.Tensor] = None,
      encoder_hidden_states: Optional[torch.FloatTensor] = None,
      encoder_attention_mask: Optional[torch.LongTensor] = None,
      head_mask: Optional[torch.Tensor] = None,
      cross_attn_head_mask: Optional[torch.Tensor] = None,
      past_key_values: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
      inputs_embeds: Optional[torch.FloatTensor] = None,
      labels: Optional[torch.LongTensor] = None,
      use_cache: Optional[bool] = None,
      output_attentions: Optional[bool] = None,
      output_hidden_states: Optional[bool] = None,
      return_dict: Optional[bool] = None,
  ) -> Union[Tuple, CausalLMOutputWithCrossAttentions]:
      r"""
      Args:
          input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
              Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you
              provide it.
  
              Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and
              [`PreTrainedTokenizer.__call__`] for details.
  
              [What are input IDs?](../glossary#input-ids)
          attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*):
              Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`:
  
              - 1 for tokens that are **not masked**,
              - 0 for tokens that are **masked**.
  
              [What are attention masks?](../glossary#attention-mask)
          encoder_hidden_states  (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):
              Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention
              if the model is configured as a decoder.
          encoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*):
              Mask to avoid performing attention on the padding token indices of the encoder input. This mask is used
              in the cross-attention if the model is configured as a decoder. Mask values selected in `[0, 1]`:
          head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*):
              Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`:
  
              - 1 indicates the head is **not masked**,
              - 0 indicates the head is **masked**.
  
          cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*):
              Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`:
  
              - 1 indicates the head is **not masked**,
              - 0 indicates the head is **masked**.
  
          past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):
              Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of
              shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of
              shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. The two additional
              tensors are only required when the model is used as a decoder in a Sequence to Sequence model.
  
              Contains pre-computed hidden-states (key and values in the self-attention blocks and in the
              cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.
  
              If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those
              that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of
              all `decoder_input_ids` of shape `(batch_size, sequence_length)`.
          labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
              Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
              config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
              (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
          use_cache (`bool`, *optional*):
              If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding
              (see `past_key_values`).
  
              - 1 for tokens that are **not masked**,
              - 0 for tokens that are **masked**.
          output_attentions (`bool`, *optional*):
              Whether or not to return the attentions tensors of all attention layers. See `attentions` under
              returned tensors for more detail.
          output_hidden_states (`bool`, *optional*):
              Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
              for more detail.
          return_dict (`bool`, *optional*):
              Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
  
      Returns:
  
      Example:
  
      ```python
      >>> from transformers import (
      ...     TrOCRConfig,
      ...     TrOCRProcessor,
      ...     TrOCRForCausalLM,
      ...     ViTConfig,
      ...     ViTModel,
      ...     VisionEncoderDecoderModel,
      ... )
      >>> import requests
      >>> from PIL import Image
  
      >>> # TrOCR is a decoder model and should be used within a VisionEncoderDecoderModel
      >>> # init vision2text model with random weights
      >>> encoder = ViTModel(ViTConfig())
      >>> decoder = TrOCRForCausalLM(TrOCRConfig())
      >>> model = VisionEncoderDecoderModel(encoder=encoder, decoder=decoder)
  
      >>> # If you want to start from the pretrained model, load the checkpoint with `VisionEncoderDecoderModel`
      >>> processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")
      >>> model = VisionEncoderDecoderModel.from_pretrained("microsoft/trocr-base-handwritten")
  
      >>> # load image from the IAM dataset
      >>> url = "https://fki.tic.heia-fr.ch/static/img/a01-122-02.jpg"
      >>> image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
      >>> pixel_values = processor(image, return_tensors="pt").pixel_values
      >>> text = "industry, ' Mr. Brown commented icily. ' Let us have a"
  
      >>> # training
      >>> model.config.decoder_start_token_id = processor.tokenizer.cls_token_id
      >>> model.config.pad_token_id = processor.tokenizer.pad_token_id
      >>> model.config.vocab_size = model.config.decoder.vocab_size
  
      >>> labels = processor.tokenizer(text, return_tensors="pt").input_ids
      >>> outputs = model(pixel_values, labels=labels)
      >>> loss = outputs.loss
      >>> round(loss.item(), 2)
      5.30
  
      >>> # inference
      >>> generated_ids = model.generate(pixel_values)
      >>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
      >>> generated_text
      'industry, " Mr. Brown commented icily. " Let us have a'
      ```"""
  
      output_attentions = (
          output_attentions
          if output_attentions is not None
          else self.config.output_attentions
      )
      output_hidden_states = (
          output_hidden_states
          if output_hidden_states is not None
          else self.config.output_hidden_states
      )
      return_dict = (
          return_dict
          if return_dict is not None
          else self.config.use_return_dict
      )
  
      # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
      outputs = self.model.decoder(
          input_ids=input_ids,
          attention_mask=attention_mask,
          encoder_hidden_states=encoder_hidden_states,
          encoder_attention_mask=encoder_attention_mask,
          head_mask=head_mask,
          cross_attn_head_mask=cross_attn_head_mask,
          past_key_values=past_key_values,
          inputs_embeds=inputs_embeds,
          use_cache=use_cache,
          output_attentions=output_attentions,
          output_hidden_states=output_hidden_states,
          return_dict=return_dict,
      )
  
      logits = self.output_projection(outputs[0])
  
      loss = None
      if labels is not None:
          loss_fct = CrossEntropyLoss()
          loss = loss_fct(
              logits.view(-1, self.config.vocab_size), labels.view(-1)
          )
  
      if not return_dict:
          output = (logits,) + outputs[1:]
          return (loss,) + output if loss is not None else output
  
      return CausalLMOutputWithCrossAttentions(
          loss=loss,
          logits=logits,
          past_key_values=outputs.past_key_values,
          hidden_states=outputs.hidden_states,
          attentions=outputs.attentions,
          cross_attentions=outputs.cross_attentions,
      )
  
imports: import copy
  import math
  from typing import Optional, Tuple, Union
  import torch
  from torch import nn
  from torch.nn import CrossEntropyLoss
  from ...activations import ACT2FN
  from ...modeling_attn_mask_utils import _prepare_4d_attention_mask, _prepare_4d_causal_attention_mask
  from ...modeling_outputs import BaseModelOutputWithPastAndCrossAttentions, CausalLMOutputWithCrossAttentions
  from ...modeling_utils import PreTrainedModel
  from ...utils import add_start_docstrings, logging, replace_return_docstrings
  from .configuration_trocr import TrOCRConfig
label_desc: Use image node_id 8 for calling the TrOCRForCausalLM obj's underlying member method code with example usage: obj.forward(input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, head_mask, cross_attn_head_mask, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict) and returns: CausalLMOutputWithCrossAttentions, unknown