docstr: past_key_values (`tuple(tuple(torch.FloatTensor))` of length `config.n_layers` with each tuple having 4 tensors of shape `(batch_size, num_heads, sequence_length - 1, embed_size_per_head)`):
      Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding.
      If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that
      don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all
      `decoder_input_ids` of shape `(batch_size, sequence_length)`.
  use_cache (`bool`, *optional*):
      If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see
      `past_key_values`).
  
  Returns:
  
  Example:
  
  ```python
  >>> from transformers import AutoTokenizer, GPTNeoXJapaneseModel
  >>> import torch
  
  >>> tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
  >>> model = GPTNeoXJapaneseModel.from_pretrained("abeja/gpt-neox-japanese-2.7b")
  
  >>> inputs = tokenizer("日本語のGPT-neoxがHugging Faceで使えます😀", return_tensors="pt")
  >>> outputs = model(**inputs)
  
  >>> last_hidden_states = outputs.last_hidden_state
  ```
docstr_tok: ["past_key_values","(","`","tuple","(","tuple","(","torch.FloatTensor",")",")","`","of","length","`","config.n_layers","`","with","each","tuple","having","4","tensors","of","shape","`","(","batch_size",",","num_heads",",","sequence_length","-","1",",","embed_size_per_head",")","`",")",":","Contains","precomputed","key","and","value","hidden","states","of","the","attention","blocks",".","Can","be","used","to","speed","up","decoding",".","If","`","past_key_values","`","are","used",",","the","user","can","optionally","input","only","the","last","`","decoder_input_ids","`","(","those","that","do","n't","have","their","past","key","value","states","given","to","this","model",")","of","shape","`","(","batch_size",",","1",")","`","instead","of","all","`","decoder_input_ids","`","of","shape","`","(","batch_size",",","sequence_length",")","`",".","use_cache","(","`","bool","`",",","*","optional","*",")",":","If","set","to","`","True","`",",","`","past_key_values","`","key","value","states","are","returned","and","can","be","used","to","speed","up","decoding","(","see","`","past_key_values","`",")",".","Returns",":","Example",":","``","`","python",">",">",">","from","transformers","import","AutoTokenizer",",","GPTNeoXJapaneseModel",">",">",">","import","torch",">",">",">","tokenizer","=","AutoTokenizer.from_pretrained","(","``","abeja\/gpt-neox-japanese-2.7b","''",")",">",">",">","model","=","GPTNeoXJapaneseModel.from_pretrained","(","``","abeja\/gpt-neox-japanese-2.7b","''",")",">",">",">","inputs","=","tokenizer","(","``","\u65e5\u672c\u8a9e\u306eGPT-neox\u304cHugging","Face\u3067\u4f7f\u3048\u307e\u3059\ud83d\ude00","''",",","return_tensors=","''","pt","''",")",">",">",">","outputs","=","model","(","*","*","inputs",")",">",">",">","last_hidden_states","=","outputs.last_hidden_state","``","`"]
code: def forward(
      self,
      input_ids: Optional[torch.LongTensor] = None,
      attention_mask: Optional[torch.FloatTensor] = None,
      head_mask: Optional[torch.FloatTensor] = None,
      inputs_embeds: Optional[torch.FloatTensor] = None,
      past_key_values: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
      use_cache: Optional[bool] = None,
      output_attentions: Optional[bool] = None,
      output_hidden_states: Optional[bool] = None,
      return_dict: Optional[bool] = None,
  ) -> Union[Tuple, BaseModelOutputWithPast]:
      r"""
      past_key_values (`tuple(tuple(torch.FloatTensor))` of length `config.n_layers` with each tuple having 4 tensors of shape `(batch_size, num_heads, sequence_length - 1, embed_size_per_head)`):
          Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding.
          If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that
          don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all
          `decoder_input_ids` of shape `(batch_size, sequence_length)`.
      use_cache (`bool`, *optional*):
          If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see
          `past_key_values`).
  
      Returns:
  
      Example:
  
      ```python
      >>> from transformers import AutoTokenizer, GPTNeoXJapaneseModel
      >>> import torch
  
      >>> tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
      >>> model = GPTNeoXJapaneseModel.from_pretrained("abeja/gpt-neox-japanese-2.7b")
  
      >>> inputs = tokenizer("日本語のGPT-neoxがHugging Faceで使えます😀", return_tensors="pt")
      >>> outputs = model(**inputs)
  
      >>> last_hidden_states = outputs.last_hidden_state
      ```
      """
      output_attentions = (
          output_attentions
          if output_attentions is not None
          else self.config.output_attentions
      )
      output_hidden_states = (
          output_hidden_states
          if output_hidden_states is not None
          else self.config.output_hidden_states
      )
      return_dict = (
          return_dict
          if return_dict is not None
          else self.config.use_return_dict
      )
      use_cache = (
          use_cache if use_cache is not None else self.config.use_cache
      )
  
      if input_ids is not None and inputs_embeds is not None:
          raise ValueError(
              "You cannot specify both input_ids and inputs_embeds at the same time"
          )
      elif input_ids is not None:
          self.warn_if_padding_and_no_attention_mask(
              input_ids, attention_mask
          )
          input_shape = input_ids.size()
      elif inputs_embeds is not None:
          input_shape = inputs_embeds.size()[:-1]
      else:
          raise ValueError(
              "You have to specify either input_ids or inputs_embeds"
          )
  
      batch_size, seq_length = input_shape
  
      if past_key_values is None:
          past_key_values = tuple(
              [None] * self.config.num_hidden_layers
          )
  
      # Attention mask.
      if attention_mask is not None:
          if not batch_size > 0:
              raise ValueError("batch_size has to be defined and > 0")
          attention_mask = attention_mask.view(batch_size, -1)
          # We create a 3D attention mask from a 2D tensor mask.
          # Sizes are [batch_size, 1, 1, to_seq_length]
          # So we can broadcast to [batch_size, num_heads, from_seq_length, to_seq_length]
          # this attention mask is more simple than the triangular masking of causal attention
          # used in OpenAI GPT, we just need to prepare the broadcast dimension here.
          attention_mask = attention_mask[:, None, None, :]
  
          # Since attention_mask is 1.0 for positions we want to attend and 0.0 for
          # masked positions, this operation will create a tensor which is 0.0 for
          # positions we want to attend and -10000.0 for masked positions.
          # Since we are adding it to the raw scores before the softmax, this is
          # effectively the same as removing these entirely.
          attention_mask = attention_mask.to(
              dtype=self.dtype
          )  # fp16 compatibility
          attention_mask = (1.0 - attention_mask) * torch.finfo(
              self.dtype
          ).min
  
      # Prepare head mask if needed
      # 1.0 in head_mask indicate we keep the head
      # attention_probs has shape bsz x n_heads x N x N
      # input head_mask has shape [num_heads] or [num_hidden_layers x num_heads]
      # and head_mask is converted to shape [num_hidden_layers x batch x num_heads x seq_length x seq_length]
      head_mask = self.get_head_mask(
          head_mask, self.config.num_hidden_layers
      )
  
      if inputs_embeds is None:
          inputs_embeds = self.embed_in(input_ids)
  
      hidden_states = inputs_embeds
  
      presents = () if use_cache else None
      all_attentions = () if output_attentions else None
      all_hidden_states = () if output_hidden_states else None
      for i, (layer, layer_past) in enumerate(
          zip(self.layers, past_key_values)
      ):
          if output_hidden_states:
              all_hidden_states = all_hidden_states + (hidden_states,)
          outputs = layer(
              hidden_states,
              attention_mask=attention_mask,
              head_mask=head_mask[i],
              layer_past=layer_past,
              use_cache=use_cache,
              output_attentions=output_attentions,
          )
          hidden_states = outputs[0]
          if use_cache is True:
              presents = presents + (outputs[1],)
          if output_attentions:
              all_attentions = all_attentions + (
                  outputs[2 if use_cache else 1],
              )
  
      hidden_states = self.final_layer_norm(hidden_states)
      # Add last hidden state
      if output_hidden_states:
          all_hidden_states = all_hidden_states + (hidden_states,)
  
      if not return_dict:
          return tuple(
              v
              for v in [
                  hidden_states,
                  presents,
                  all_hidden_states,
                  all_attentions,
              ]
              if v is not None
          )
  
      return BaseModelOutputWithPast(
          last_hidden_state=hidden_states,
          past_key_values=presents,
          hidden_states=all_hidden_states,
          attentions=all_attentions,
      )
  
imports: from typing import Optional, Tuple, Union
  import torch
  import torch.utils.checkpoint
  from torch import Tensor, nn
  from torch.nn import CrossEntropyLoss
  from ...activations import ACT2FN
  from ...file_utils import add_start_docstrings, add_start_docstrings_to_model_forward, replace_return_docstrings
  from ...modeling_outputs import BaseModelOutputWithPast, CausalLMOutputWithPast
  from ...modeling_utils import PreTrainedModel
  from ...utils import logging
  from .configuration_gpt_neox_japanese import GPTNeoXJapaneseConfig
label_desc: Use image node_id 4 for calling the GPTNeoXJapaneseModel obj's underlying member method code with example usage: obj.forward(input_ids, attention_mask, head_mask, inputs_embeds, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict) and returns: BaseModelOutputWithPast, tuple