class_docstr: A class containing all of the functions supporting generation, to be used as a mixin in [`TFPreTrainedModel`].
class_docstr_tok: ["A","class","containing","all","of","the","functions","supporting","generation",",","to","be","used","as","a","mixin","in","[","`","TFPreTrainedModel","`","]","."]
docstr: Generates sequences for models with a language modeling head. The method currently supports greedy decoding,
  beam-search decoding, sampling with temperature, sampling with top-k or nucleus sampling.
  
  Adapted in part from [Facebook's XLM beam search
  code](https://github.com/facebookresearch/XLM/blob/9e6f6814d17be4fe5b15f2e6c43eb2b2d76daeb4/src/model/transformer.py#L529).
  
  Apart from `input_ids` and `attention_mask`, all the arguments below will default to the value of the attribute
  of the same name inside the [`PretrainedConfig`] of the model. The default values indicated are the default
  values of those config.
  
  Most of these parameters are explained in more detail in [this blog
  post](https://huggingface.co/blog/how-to-generate).
  
  Parameters:
      input_ids (`tf.Tensor` of `dtype=tf.int32` and shape `(batch_size, sequence_length)`, *optional*):
          The sequence used as a prompt for the generation. If `None` the method initializes it with
          `bos_token_id` and a batch size of 1.
      max_length (`int`, *optional*, defaults to `model.config.max_length`):
          The maximum length the generated tokens can have. Corresponds to the length of the input prompt +
          `max_new_tokens`. In general, prefer the use of `max_new_tokens`, which ignores the number of tokens in
          the prompt.
      max_new_tokens (`int`, *optional*):
          The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.
      min_length (`int`, *optional*, defaults to 10):
          The minimum length of the sequence to be generated.
      do_sample (`bool`, *optional*, defaults to `False`):
          Whether or not to use sampling ; use greedy decoding otherwise.
      early_stopping (`bool`, *optional*, defaults to `False`):
          Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.
      num_beams (`int`, *optional*, defaults to 1):
          Number of beams for beam search. 1 means no beam search.
      temperature (`float`, *optional*, defaults to 1.0):
          The value used to module the next token probabilities.
      top_k (`int`, *optional*, defaults to 50):
          The number of highest probability vocabulary tokens to keep for top-k-filtering.
      top_p (`float`, *optional*, defaults to 1.0):
          If set to float < 1, only the most probable tokens with probabilities that add up to `top_p` or higher
          are kept for generation.
      repetition_penalty (`float`, *optional*, defaults to 1.0):
          The parameter for repetition penalty. 1.0 means no penalty. See [this
          paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
      pad_token_id (`int`, *optional*):
          The id of the *padding* token.
      bos_token_id (`int`, *optional*):
          The id of the *beginning-of-sequence* token.
      eos_token_id (`int`, *optional*):
          The id of the *end-of-sequence* token.
      length_penalty (`float`, *optional*, defaults to 1.0):
          Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent
          to the sequence length, which in turn is used to divide the score of the sequence. Since the score is
          the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences,
          while `length_penalty` < 0.0 encourages shorter sequences.
      no_repeat_ngram_size (`int`, *optional*, defaults to 0):
          If set to int > 0, all ngrams of that size can only occur once.
      bad_words_ids(`List[int]`, *optional*):
          List of token ids that are not allowed to be generated. In order to get the tokens of the words that
          should not appear in the generated text, use `tokenizer.encode(bad_word, add_prefix_space=True)`.
      num_return_sequences(`int`, *optional*, defaults to 1):
          The number of independently computed returned sequences for each element in the batch.
      attention_mask (`tf.Tensor` of `dtype=tf.int32` and shape `(batch_size, sequence_length)`, *optional*):
          Mask to avoid performing attention on padding token indices. Mask values are in `[0, 1]`, 1 for tokens
          that are not masked, and 0 for masked tokens.
  
          If not provided, will default to a tensor the same shape as `input_ids` that masks the pad token.
  
          [What are attention masks?](../glossary#attention-mask)
      decoder_start_token_id (`int`, *optional*):
          If an encoder-decoder model starts decoding with a different token than *bos*, the id of that token.
      use_cache (`bool`, *optional*, defaults to `True`):
          Whether or not the model should use the past last key/values attentions (if applicable to the model) to
          speed up decoding.
      seed (`List[int]`, *optional*):
          Random seed to control sampling, containing two integers, used when `do_sample` is `True`. See the
          `seed` argument from stateless functions in `tf.random`.
      output_attentions (`bool`, *optional*, defaults to `False`):
          Whether or not to return the attentions tensors of all attention layers. See `attentions` under
          returned tensors for more details.
      output_hidden_states (`bool`, *optional*, defaults to `False`):
          Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
          for more details.
      output_scores (`bool`, *optional*, defaults to `False`):
          Whether or not to return the prediction scores. See `scores` under returned tensors for more details.
      return_dict_in_generate (`bool`, *optional*, defaults to `False`):
          Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
      forced_bos_token_id (`int`, *optional*):
          The id of the token to force as the first generated token after the `decoder_start_token_id`. Useful
          for multilingual models like [mBART](../model_doc/mbart) where the first generated token needs to be
          the target language token.
      forced_eos_token_id (`int`, *optional*):
          The id of the token to force as the last generated token when `max_length` is reached.
      suppress_tokens  (`List[int]`, *optional*, defaults to `model.config.suppress_tokens`):
          A list of tokens that will be supressed at generation. The `SupressTokens` logit processor will set
          their log probs to `-inf` so that they are not sampled.
      begin_suppress_tokens  (`List[int]`, *optional*, defaults to `model.config.begin_suppress_tokens`):
          A list of tokens that will be supressed at the begining of the generation. The `SupressBeginTokens`
          logit processor will set their log probs to `-inf` so that they are not sampled.
      forced_decoder_ids (`List[List[int]]`, *optional*, defaults to `model.config.forced_decoder_ids`):
          A list of pairs of integers which indicates a mapping from generation indices to token indices that
          will be forced before sampling. For example, `[[1, 123]]` means the second generated token will always
          be a token of index 123.
      model_kwargs:
          Additional model specific kwargs will be forwarded to the `call` function of the model.
  
  Return:
      [`~utils.ModelOutput`] or `tf.Tensor`: A [`~utils.ModelOutput`] (if `return_dict_in_generate=True` or when
      `config.return_dict_in_generate=True`) or a `tf.Tensor`.
  
          If the model is *not* an encoder-decoder model (`model.config.is_encoder_decoder=False`), the possible
          [`~utils.ModelOutput`] types are:
  
              - [`~generation_tf_utils.TFGreedySearchDecoderOnlyOutput`],
              - [`~generation_tf_utils.TFSampleDecoderOnlyOutput`],
              - [`~generation_tf_utils.TFBeamSearchDecoderOnlyOutput`],
              - [`~generation_tf_utils.TFBeamSampleDecoderOnlyOutput`]
  
          If the model is an encoder-decoder model (`model.config.is_encoder_decoder=True`), the possible
          [`~utils.ModelOutput`] types are:
  
              - [`~generation_tf_utils.TFGreedySearchEncoderDecoderOutput`],
              - [`~generation_tf_utils.TFSampleEncoderDecoderOutput`],
              - [`~generation_tf_utils.TFBeamSearchEncoderDecoderOutput`],
              - [`~generation_tf_utils.TFBeamSampleEncoderDecoderOutput`]
  
  Examples:
  
  ```python
  tokenizer = AutoTokenizer.from_pretrained("distilgpt2")  # Initialize tokenizer
  model = TFAutoModelWithLMHead.from_pretrained("distilgpt2")
  # Greedy decoding
  outputs = model.generate(max_length=40)
  print(f"Generated: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")
  
  tokenizer = AutoTokenizer.from_pretrained("openai-gpt")
  model = TFAutoModelWithLMHead.from_pretrained("openai-gpt")
  input_context = "The dog"
  input_ids = tokenizer.encode(input_context, return_tensors="tf")  # encode input context
  # Generate 3 independent sequences using beam search decoding (5 beams) with sampling from initial context 'The dog'
  outputs = model.generate(input_ids=input_ids, num_beams=5, num_return_sequences=3, temperature=1.5)
  # 3 output sequences were generated
  for i in range(3):
      print(f"Generated {i}: {tokenizer.decode(outputs[i], skip_special_tokens=True)}")
  
  tokenizer = AutoTokenizer.from_pretrained("distilgpt2")
  model = TFAutoModelWithLMHead.from_pretrained("distilgpt2")
  input_context = "The dog"
  input_ids = tokenizer.encode(input_context, return_tensors="tf")
  # Generate 3 candidates using sampling
  outputs = model.generate(
      input_ids=input_ids, max_length=40, temperature=0.7, num_return_sequences=3, do_sample=True
  )
  #  3 output sequences were generated
  for i in range(3):
      print(f"Generated {i}: {tokenizer.decode(outputs[i], skip_special_tokens=True)}")
  
  tokenizer = AutoTokenizer.from_pretrained("ctrl")
  model = TFAutoModelWithLMHead.from_pretrained("ctrl")
  # "Legal" is one of the control codes for ctrl
  input_context = "Legal My neighbor is"
  input_ids = tokenizer.encode(input_context, return_tensors="tf")
  outputs = model.generate(input_ids=input_ids, max_length=50, temperature=0.7, repetition_penalty=1.2)
  print(f"Generated: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")
  
  tokenizer = AutoTokenizer.from_pretrained("gpt2")
  model = TFAutoModelWithLMHead.from_pretrained("gpt2")
  input_context = "My cute dog"
  bad_words_ids = [
      tokenizer.encode(bad_word, add_prefix_space=True) for bad_word in ["idiot", "stupid", "shut up"]
  ]
  input_ids = tokenizer.encode(input_context, return_tensors="tf")
  # generate sequences without allowing bad_words to be generated
  outputs = model.generate(input_ids=input_ids, max_length=100, do_sample=True, bad_words_ids=bad_words_ids)
  ```
code: def _generate(
      self,
      input_ids=None,
      max_length=None,
      max_new_tokens=None,
      min_length=None,
      do_sample=None,
      early_stopping=None,
      num_beams=None,
      temperature=None,
      top_k=None,
      top_p=None,
      repetition_penalty=None,
      bad_words_ids=None,
      bos_token_id=None,
      pad_token_id=None,
      eos_token_id=None,
      length_penalty=None,
      no_repeat_ngram_size=None,
      num_return_sequences=None,
      attention_mask=None,
      decoder_start_token_id=None,
      use_cache=None,
      seed=None,
      output_scores=None,
      output_attentions=None,
      output_hidden_states=None,
      return_dict_in_generate=None,
      forced_bos_token_id=None,
      forced_eos_token_id=None,
      suppress_tokens=None,
      begin_suppress_tokens=None,
      forced_decoder_ids=None,
      **model_kwargs,
  ) -> Union[
      TFGreedySearchOutput,
      TFSampleOutput,
      TFBeamSearchOutput,
      TFBeamSampleOutput,
      tf.Tensor,
  ]:
      r"""
      Generates sequences for models with a language modeling head. The method currently supports greedy decoding,
      beam-search decoding, sampling with temperature, sampling with top-k or nucleus sampling.
  
      Adapted in part from [Facebook's XLM beam search
      code](https://github.com/facebookresearch/XLM/blob/9e6f6814d17be4fe5b15f2e6c43eb2b2d76daeb4/src/model/transformer.py#L529).
  
      Apart from `input_ids` and `attention_mask`, all the arguments below will default to the value of the attribute
      of the same name inside the [`PretrainedConfig`] of the model. The default values indicated are the default
      values of those config.
  
      Most of these parameters are explained in more detail in [this blog
      post](https://huggingface.co/blog/how-to-generate).
  
      Parameters:
          input_ids (`tf.Tensor` of `dtype=tf.int32` and shape `(batch_size, sequence_length)`, *optional*):
              The sequence used as a prompt for the generation. If `None` the method initializes it with
              `bos_token_id` and a batch size of 1.
          max_length (`int`, *optional*, defaults to `model.config.max_length`):
              The maximum length the generated tokens can have. Corresponds to the length of the input prompt +
              `max_new_tokens`. In general, prefer the use of `max_new_tokens`, which ignores the number of tokens in
              the prompt.
          max_new_tokens (`int`, *optional*):
              The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.
          min_length (`int`, *optional*, defaults to 10):
              The minimum length of the sequence to be generated.
          do_sample (`bool`, *optional*, defaults to `False`):
              Whether or not to use sampling ; use greedy decoding otherwise.
          early_stopping (`bool`, *optional*, defaults to `False`):
              Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.
          num_beams (`int`, *optional*, defaults to 1):
              Number of beams for beam search. 1 means no beam search.
          temperature (`float`, *optional*, defaults to 1.0):
              The value used to module the next token probabilities.
          top_k (`int`, *optional*, defaults to 50):
              The number of highest probability vocabulary tokens to keep for top-k-filtering.
          top_p (`float`, *optional*, defaults to 1.0):
              If set to float < 1, only the most probable tokens with probabilities that add up to `top_p` or higher
              are kept for generation.
          repetition_penalty (`float`, *optional*, defaults to 1.0):
              The parameter for repetition penalty. 1.0 means no penalty. See [this
              paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
          pad_token_id (`int`, *optional*):
              The id of the *padding* token.
          bos_token_id (`int`, *optional*):
              The id of the *beginning-of-sequence* token.
          eos_token_id (`int`, *optional*):
              The id of the *end-of-sequence* token.
          length_penalty (`float`, *optional*, defaults to 1.0):
              Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent
              to the sequence length, which in turn is used to divide the score of the sequence. Since the score is
              the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences,
              while `length_penalty` < 0.0 encourages shorter sequences.
          no_repeat_ngram_size (`int`, *optional*, defaults to 0):
              If set to int > 0, all ngrams of that size can only occur once.
          bad_words_ids(`List[int]`, *optional*):
              List of token ids that are not allowed to be generated. In order to get the tokens of the words that
              should not appear in the generated text, use `tokenizer.encode(bad_word, add_prefix_space=True)`.
          num_return_sequences(`int`, *optional*, defaults to 1):
              The number of independently computed returned sequences for each element in the batch.
          attention_mask (`tf.Tensor` of `dtype=tf.int32` and shape `(batch_size, sequence_length)`, *optional*):
              Mask to avoid performing attention on padding token indices. Mask values are in `[0, 1]`, 1 for tokens
              that are not masked, and 0 for masked tokens.
  
              If not provided, will default to a tensor the same shape as `input_ids` that masks the pad token.
  
              [What are attention masks?](../glossary#attention-mask)
          decoder_start_token_id (`int`, *optional*):
              If an encoder-decoder model starts decoding with a different token than *bos*, the id of that token.
          use_cache (`bool`, *optional*, defaults to `True`):
              Whether or not the model should use the past last key/values attentions (if applicable to the model) to
              speed up decoding.
          seed (`List[int]`, *optional*):
              Random seed to control sampling, containing two integers, used when `do_sample` is `True`. See the
              `seed` argument from stateless functions in `tf.random`.
          output_attentions (`bool`, *optional*, defaults to `False`):
              Whether or not to return the attentions tensors of all attention layers. See `attentions` under
              returned tensors for more details.
          output_hidden_states (`bool`, *optional*, defaults to `False`):
              Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
              for more details.
          output_scores (`bool`, *optional*, defaults to `False`):
              Whether or not to return the prediction scores. See `scores` under returned tensors for more details.
          return_dict_in_generate (`bool`, *optional*, defaults to `False`):
              Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
          forced_bos_token_id (`int`, *optional*):
              The id of the token to force as the first generated token after the `decoder_start_token_id`. Useful
              for multilingual models like [mBART](../model_doc/mbart) where the first generated token needs to be
              the target language token.
          forced_eos_token_id (`int`, *optional*):
              The id of the token to force as the last generated token when `max_length` is reached.
          suppress_tokens  (`List[int]`, *optional*, defaults to `model.config.suppress_tokens`):
              A list of tokens that will be supressed at generation. The `SupressTokens` logit processor will set
              their log probs to `-inf` so that they are not sampled.
          begin_suppress_tokens  (`List[int]`, *optional*, defaults to `model.config.begin_suppress_tokens`):
              A list of tokens that will be supressed at the begining of the generation. The `SupressBeginTokens`
              logit processor will set their log probs to `-inf` so that they are not sampled.
          forced_decoder_ids (`List[List[int]]`, *optional*, defaults to `model.config.forced_decoder_ids`):
              A list of pairs of integers which indicates a mapping from generation indices to token indices that
              will be forced before sampling. For example, `[[1, 123]]` means the second generated token will always
              be a token of index 123.
          model_kwargs:
              Additional model specific kwargs will be forwarded to the `call` function of the model.
  
      Return:
          [`~utils.ModelOutput`] or `tf.Tensor`: A [`~utils.ModelOutput`] (if `return_dict_in_generate=True` or when
          `config.return_dict_in_generate=True`) or a `tf.Tensor`.
  
              If the model is *not* an encoder-decoder model (`model.config.is_encoder_decoder=False`), the possible
              [`~utils.ModelOutput`] types are:
  
                  - [`~generation_tf_utils.TFGreedySearchDecoderOnlyOutput`],
                  - [`~generation_tf_utils.TFSampleDecoderOnlyOutput`],
                  - [`~generation_tf_utils.TFBeamSearchDecoderOnlyOutput`],
                  - [`~generation_tf_utils.TFBeamSampleDecoderOnlyOutput`]
  
              If the model is an encoder-decoder model (`model.config.is_encoder_decoder=True`), the possible
              [`~utils.ModelOutput`] types are:
  
                  - [`~generation_tf_utils.TFGreedySearchEncoderDecoderOutput`],
                  - [`~generation_tf_utils.TFSampleEncoderDecoderOutput`],
                  - [`~generation_tf_utils.TFBeamSearchEncoderDecoderOutput`],
                  - [`~generation_tf_utils.TFBeamSampleEncoderDecoderOutput`]
  
      Examples:
  
      ```python
      tokenizer = AutoTokenizer.from_pretrained("distilgpt2")  # Initialize tokenizer
      model = TFAutoModelWithLMHead.from_pretrained("distilgpt2")
      # Greedy decoding
      outputs = model.generate(max_length=40)
      print(f"Generated: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")
  
      tokenizer = AutoTokenizer.from_pretrained("openai-gpt")
      model = TFAutoModelWithLMHead.from_pretrained("openai-gpt")
      input_context = "The dog"
      input_ids = tokenizer.encode(input_context, return_tensors="tf")  # encode input context
      # Generate 3 independent sequences using beam search decoding (5 beams) with sampling from initial context 'The dog'
      outputs = model.generate(input_ids=input_ids, num_beams=5, num_return_sequences=3, temperature=1.5)
      # 3 output sequences were generated
      for i in range(3):
          print(f"Generated {i}: {tokenizer.decode(outputs[i], skip_special_tokens=True)}")
  
      tokenizer = AutoTokenizer.from_pretrained("distilgpt2")
      model = TFAutoModelWithLMHead.from_pretrained("distilgpt2")
      input_context = "The dog"
      input_ids = tokenizer.encode(input_context, return_tensors="tf")
      # Generate 3 candidates using sampling
      outputs = model.generate(
          input_ids=input_ids, max_length=40, temperature=0.7, num_return_sequences=3, do_sample=True
      )
      #  3 output sequences were generated
      for i in range(3):
          print(f"Generated {i}: {tokenizer.decode(outputs[i], skip_special_tokens=True)}")
  
      tokenizer = AutoTokenizer.from_pretrained("ctrl")
      model = TFAutoModelWithLMHead.from_pretrained("ctrl")
      # "Legal" is one of the control codes for ctrl
      input_context = "Legal My neighbor is"
      input_ids = tokenizer.encode(input_context, return_tensors="tf")
      outputs = model.generate(input_ids=input_ids, max_length=50, temperature=0.7, repetition_penalty=1.2)
      print(f"Generated: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")
  
      tokenizer = AutoTokenizer.from_pretrained("gpt2")
      model = TFAutoModelWithLMHead.from_pretrained("gpt2")
      input_context = "My cute dog"
      bad_words_ids = [
          tokenizer.encode(bad_word, add_prefix_space=True) for bad_word in ["idiot", "stupid", "shut up"]
      ]
      input_ids = tokenizer.encode(input_context, return_tensors="tf")
      # generate sequences without allowing bad_words to be generated
      outputs = model.generate(input_ids=input_ids, max_length=100, do_sample=True, bad_words_ids=bad_words_ids)
      ```"""
  
      # 0. Validate the `.generate()` call
      self._validate_model_class()
      self._validate_model_kwargs(model_kwargs.copy())
  
      # 1. Cast input dtypes to tf.int32 unless they're floats (which happens for some image models)
      if input_ids is not None:
          if (
              isinstance(input_ids, tf.Tensor)
              and input_ids.dtype.is_floating
          ):
              pass
          elif isinstance(input_ids, np.ndarray) and np.issubdtype(
              input_ids.dtype, np.floating
          ):
              pass
          else:
              input_ids = tf.cast(input_ids, tf.int32)
      if attention_mask is not None:
          attention_mask = tf.cast(attention_mask, tf.int32)
      if "decoder_input_ids" in model_kwargs:
          if (
              isinstance(model_kwargs["decoder_input_ids"], tf.Tensor)
              and model_kwargs["decoder_input_ids"].dtype.is_floating
          ):
              pass
          elif isinstance(
              model_kwargs["decoder_input_ids"], np.ndarray
          ) and np.issubdtype(
              model_kwargs["decoder_input_ids"].dtype, np.floating
          ):
              pass
          else:
              model_kwargs["decoder_input_ids"] = tf.cast(
                  model_kwargs["decoder_input_ids"], tf.int32
              )
  
      # 2. Set generation parameters if not already defined
      length_penalty = (
          length_penalty
          if length_penalty is not None
          else self.config.length_penalty
      )
      early_stopping = (
          early_stopping
          if early_stopping is not None
          else self.config.early_stopping
      )
  
      bos_token_id = (
          bos_token_id
          if bos_token_id is not None
          else self.config.bos_token_id
      )
      pad_token_id = (
          pad_token_id
          if pad_token_id is not None
          else self.config.pad_token_id
      )
      eos_token_id = (
          eos_token_id
          if eos_token_id is not None
          else self.config.eos_token_id
      )
  
      forced_bos_token_id = (
          forced_bos_token_id
          if forced_bos_token_id is not None
          else self.config.forced_bos_token_id
      )
      forced_eos_token_id = (
          forced_eos_token_id
          if forced_eos_token_id is not None
          else self.config.forced_eos_token_id
      )
  
      output_scores = (
          output_scores
          if output_scores is not None
          else self.config.output_scores
      )
      output_attentions = (
          output_attentions
          if output_attentions is not None
          else self.config.output_attentions
      )
      output_hidden_states = (
          output_hidden_states
          if output_hidden_states is not None
          else self.config.output_hidden_states
      )
      return_dict_in_generate = (
          return_dict_in_generate
          if return_dict_in_generate is not None
          else self.config.return_dict_in_generate
      )
  
      num_beams = (
          num_beams if num_beams is not None else self.config.num_beams
      )
      do_sample = (
          do_sample if do_sample is not None else self.config.do_sample
      )
      num_return_sequences = (
          num_return_sequences
          if num_return_sequences is not None
          else self.config.num_return_sequences
      )
  
      if pad_token_id is None and eos_token_id is not None:
          if attention_mask is None:
              logger.warning(
                  "The attention mask and the pad token id were not set. As a consequence, you may observe "
                  "unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results."
              )
          logger.warning(
              f"Setting `pad_token_id` to {eos_token_id} (first `eos_token_id`) to generate sequence"
          )
          pad_token_id = eos_token_id
  
      use_xla = not tf.executing_eagerly()
      if use_xla and not self.supports_xla_generation:
          raise ValueError(
              "The selected model does not support Graph mode nor XLA generation (e.g. from tf.function())"
          )
  
      # 3. Define model inputs
      input_ids = self._prepare_model_inputs(input_ids, bos_token_id)
      # inputs_ids now has to be defined and cannot be None anymore
      batch_size = shape_list(input_ids)[0]
  
      # 4. Prepare other model kwargs
      if output_attentions is not None:
          model_kwargs["output_attentions"] = output_attentions
      if output_hidden_states is not None:
          model_kwargs["output_hidden_states"] = output_hidden_states
      if use_cache is not None:
          model_kwargs["use_cache"] = use_cache
      if attention_mask is not None:
          model_kwargs["attention_mask"] = attention_mask
  
      accepts_attention_mask = "attention_mask" in set(
          inspect.signature(self.call).parameters.keys()
      )
      requires_attention_mask = "encoder_outputs" not in model_kwargs
  
      if (
          model_kwargs.get("attention_mask", None) is None
          and requires_attention_mask
          and accepts_attention_mask
      ):
          model_kwargs[
              "attention_mask"
          ] = self._prepare_attention_mask_for_generation(
              input_ids, pad_token_id, eos_token_id
          )
  
      # decoder-only models should use left-padding for generation
      if not self.config.is_encoder_decoder:
          if pad_token_id is not None and tf.math.reduce_any(
              input_ids[:, -1] == pad_token_id
          ):
              logger.warning(
                  "A decoder-only architecture is being used, but right-padding was detected! For correct "
                  "generation results, please set `padding_side='left'` when initializing the tokenizer."
              )
  
      # 5. Prepare model inputs which will be used for auto-regressive generation
      if self.config.is_encoder_decoder:
          # if encoder-decoder, we create encoder_outputs and add to `model_kwargs`
          model_kwargs = (
              self._prepare_encoder_decoder_kwargs_for_generation(
                  input_ids, model_kwargs
              )
          )
          # if encoder-decoder then `input_ids` come from `decoder_start_token_id`
          input_ids = self._prepare_decoder_input_ids_for_generation(
              batch_size,
              decoder_start_token_id=decoder_start_token_id,
              bos_token_id=bos_token_id,
              model_kwargs=model_kwargs,
          )
  
      # 6. Prepare `max_length` depending on other stopping criteria.
      input_ids_seq_length = input_ids.shape[-1]
      if max_length is None and max_new_tokens is None:
          warnings.warn(
              "Neither `max_length` nor `max_new_tokens` have been set, `max_length` will default to "
              f"{self.config.max_length} (`self.config.max_length`). Controlling `max_length` via the config is "
              "deprecated and `max_length` will be removed from the config in v5 of Transformers -- we recommend "
              "using `max_new_tokens` to control the maximum length of the generation.",
              UserWarning,
          )
      elif max_length is None and max_new_tokens is not None:
          max_length = max_new_tokens + input_ids_seq_length
      elif max_length is not None and max_new_tokens is not None:
          raise ValueError(
              "Both `max_new_tokens` and `max_length` have been set but they serve the same purpose -- setting a"
              " limit to the generated output length. Remove one of those arguments. Please refer to the"
              " documentation for more information. "
              "(https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)"
          )
      # default to config if still None
      max_length = (
          max_length
          if max_length is not None
          else self.config.max_length
      )
      min_length = (
          min_length
          if min_length is not None
          else self.config.min_length
      )
  
      if min_length is not None and min_length > max_length:
          raise ValueError(
              f"Unfeasable length constraints: the minimum length ({min_length}) is larger than the maximum "
              f"length ({max_length})"
          )
      if input_ids_seq_length >= max_length:
          input_ids_string = (
              "decoder_input_ids"
              if self.config.is_encoder_decoder
              else "input_ids"
          )
          logger.warning(
              f"Input length of {input_ids_string} is {input_ids_seq_length}, but `max_length` is set to"
              f" {max_length}. This can lead to unexpected behavior. You should consider increasing"
              "`max_new_tokens`."
          )
  
      # 7. determine generation mode
      # TODO(Matt, Joao, Patrick) - add more use cases here
      is_greedy_gen_mode = (num_beams == 1) and do_sample is False
      is_sample_gen_mode = (num_beams == 1) and do_sample is True
      is_beam_gen_mode = (num_beams > 1) and do_sample is False
  
      # 8. prepare distribution pre_processing samplers
      logits_processor = self._get_logits_processor(
          repetition_penalty=repetition_penalty,
          no_repeat_ngram_size=no_repeat_ngram_size,
          input_ids_seq_length=input_ids_seq_length,
          bad_words_ids=bad_words_ids,
          min_length=min_length,
          max_length=max_length,
          eos_token_id=eos_token_id,
          forced_bos_token_id=forced_bos_token_id,
          forced_eos_token_id=forced_eos_token_id,
          suppress_tokens=suppress_tokens,
          begin_suppress_tokens=begin_suppress_tokens,
          forced_decoder_ids=forced_decoder_ids,
      )
  
      # 9. go into different generation modes
      if is_greedy_gen_mode:
          if num_return_sequences > 1:
              raise ValueError(
                  f"num_return_sequences has to be 1, but is {num_return_sequences} when doing greedy search."
              )
          # 9. run greedy search
          return self.greedy_search(
              input_ids,
              max_length=max_length,
              pad_token_id=pad_token_id,
              eos_token_id=eos_token_id,
              logits_processor=logits_processor,
              output_scores=output_scores,
              return_dict_in_generate=return_dict_in_generate,
              **model_kwargs,
          )
      elif is_sample_gen_mode:
          # 10. prepare logits warper
          logits_warper = self._get_logits_warper(
              top_k=top_k, top_p=top_p, temperature=temperature
          )
  
          # 11. expand input_ids with `num_return_sequences` additional sequences per batch
          input_ids, model_kwargs = self._expand_inputs_for_generation(
              input_ids,
              expand_size=num_return_sequences,
              is_encoder_decoder=self.config.is_encoder_decoder,
              **model_kwargs,
          )
  
          # 12. run sample
          return self.sample(
              input_ids,
              logits_processor=logits_processor,
              logits_warper=logits_warper,
              max_length=max_length,
              pad_token_id=pad_token_id,
              eos_token_id=eos_token_id,
              seed=seed,
              output_scores=output_scores,
              return_dict_in_generate=return_dict_in_generate,
              **model_kwargs,
          )
  
      elif is_beam_gen_mode:
          if num_beams < num_return_sequences:
              raise ValueError(
                  "Greedy beam search decoding cannot return more sequences than it has beams. Please set "
                  f"num_beams >= num_return_sequences, got {num_beams} and {num_return_sequences} (respectivelly)"
              )
  
          # 10. broadcast inputs to the desired number of beams
          input_ids = self._expand_to_num_beams(
              input_ids, num_beams=num_beams
          )
  
          if "encoder_outputs" in model_kwargs:
              model_kwargs["encoder_outputs"][
                  "last_hidden_state"
              ] = self._expand_to_num_beams(
                  model_kwargs["encoder_outputs"]["last_hidden_state"],
                  num_beams=num_beams,
              )
  
          if "attention_mask" in model_kwargs:
              model_kwargs[
                  "attention_mask"
              ] = self._expand_to_num_beams(
                  model_kwargs["attention_mask"], num_beams=num_beams
              )
  
          # 11. run beam search
          return self.beam_search(
              input_ids,
              max_length=max_length,
              pad_token_id=pad_token_id,
              eos_token_id=eos_token_id,
              length_penalty=length_penalty,
              early_stopping=early_stopping,
              logits_processor=logits_processor,
              return_dict_in_generate=return_dict_in_generate,
              num_return_sequences=num_return_sequences,
              **model_kwargs,
          )
  
      else:
          # TODO(Matt, Joao, Patrick) - add more sub-generation methods here
          raise NotImplementedError(
              "Beam sampling is currently not implemented."
          )
  
imports: import inspect
  import warnings
  from dataclasses import dataclass
  from typing import Any, Dict, List, Optional, Tuple, Union
  import numpy
  import tensorflow
  from tensorflow.compiler.tf2xla.python.xla import dynamic_update_slice
  from .generation_tf_logits_process import TFForcedBOSTokenLogitsProcessor, TFForcedEOSTokenLogitsProcessor, TFForceTokensLogitsProcessor, TFLogitsProcessorList, TFMinLengthLogitsProcessor, TFNoBadWordsLogitsProcessor, TFNoRepeatNGramLogitsProcessor, TFRepetitionPenaltyLogitsProcessor, TFSuppressTokensAtBeginLogitsProcessor, TFSuppressTokensLogitsProcessor, TFTemperatureLogitsWarper, TFTopKLogitsWarper, TFTopPLogitsWarper
  from .models.auto import TF_MODEL_FOR_CAUSAL_LM_MAPPING, TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING, TF_MODEL_FOR_SPEECH_SEQ_2_SEQ_MAPPING, TF_MODEL_FOR_VISION_2_SEQ_MAPPING
  from .tf_utils import shape_list, stable_softmax
  from .utils import ModelOutput, logging
label_desc: Use image node_id 9 for calling the TFGenerationMixin obj's underlying member method code with example usage: obj._generate(input_ids, max_length, max_new_tokens, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, attention_mask, decoder_start_token_id, use_cache, seed, output_scores, output_attentions, output_hidden_states, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, suppress_tokens, begin_suppress_tokens, forced_decoder_ids) and returns: self, self, self