class_docstr: The IterativeSFTTrainer can be used to finetune models with methods that requires some steps between optimization.
  
  Attributes:
      **model** (`PreTrainedModel`) -- Model to be optimized, either an 'AutoModelForCausalLM' or an 'AutoModelForSeq2SeqLM'.
          Check the documentation of `PreTrainedModel` for more details.
      **args** (`transformers.TrainingArguments`): -- The arguments to use for training.
      **tokenizer** (`PreTrainedTokenizerBase`) -- Tokenizer to be used for encoding the
          data. Check the documentation of `transformers.PreTrainedTokenizer` and
          `transformers.PreTrainedTokenizerFast` for more details.
      **optimizers** (`Tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): -- The optimizer and scheduler to use for training.
      **data_collator** (Union[DataCollatorForLanguageModeling, DataCollatorForSeq2Seq], *optional*) -- Data collator to be used for training and
          passed along the dataloader.
      **eval_dataset** (`datasets.Dataset`): The dataset to use for evaluation.
      **max_length** (`int`, defaults to `None`): -- The maximum length of the input.
      **truncation_mode** (`str`, defaults to `keep_end`): -- The truncation mode to use, either `keep_end` or `keep_start`.
      **preprocess_logits_for_metrics** (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): -- The function to use to preprocess the logits before computing the metrics.
      **compute_metrics** (`Callable[[EvalPrediction], Dict]`, *optional*): -- The function to use to compute the metrics. Must take a `EvalPrediction` and return a dictionary string to metric values.
      **optimize_device_cache ** (`bool`, *optional*, defaults to `False`) -- Optimize CUDA cache for slightly more memory-efficient training.
class_docstr_tok: ["The","IterativeSFTTrainer","can","be","used","to","finetune","models","with","methods","that","requires","some","steps","between","optimization",".","Attributes",":","*","*","model","*","*","(","`","PreTrainedModel","`",")","--","Model","to","be","optimized",",","either","an","'AutoModelForCausalLM","'","or","an","'AutoModelForSeq2SeqLM","'",".","Check","the","documentation","of","`","PreTrainedModel","`","for","more","details",".","*","*","args","*","*","(","`","transformers.TrainingArguments","`",")",":","--","The","arguments","to","use","for","training",".","*","*","tokenizer","*","*","(","`","PreTrainedTokenizerBase","`",")","--","Tokenizer","to","be","used","for","encoding","the","data",".","Check","the","documentation","of","`","transformers.PreTrainedTokenizer","`","and","`","transformers.PreTrainedTokenizerFast","`","for","more","details",".","*","*","optimizers","*","*","(","`","Tuple","[","torch.optim.Optimizer",",","torch.optim.lr_scheduler.LambdaLR","]","`",")",":","--","The","optimizer","and","scheduler","to","use","for","training",".","*","*","data_collator","*","*","(","Union","[","DataCollatorForLanguageModeling",",","DataCollatorForSeq2Seq","]",",","*","optional","*",")","--","Data","collator","to","be","used","for","training","and","passed","along","the","dataloader",".","*","*","eval_dataset","*","*","(","`","datasets.Dataset","`",")",":","The","dataset","to","use","for","evaluation",".","*","*","max_length","*","*","(","`","int","`",",","defaults","to","`","None","`",")",":","--","The","maximum","length","of","the","input",".","*","*","truncation_mode","*","*","(","`","str","`",",","defaults","to","`","keep_end","`",")",":","--","The","truncation","mode","to","use",",","either","`","keep_end","`","or","`","keep_start","`",".","*","*","preprocess_logits_for_metrics","*","*","(","`","Callable","[","[","torch.Tensor",",","torch.Tensor","]",",","torch.Tensor","]","`",")",":","--","The","function","to","use","to","preprocess","the","logits","before","computing","the","metrics",".","*","*","compute_metrics","*","*","(","`","Callable","[","[","EvalPrediction","]",",","Dict","]","`",",","*","optional","*",")",":","--","The","function","to","use","to","compute","the","metrics",".","Must","take","a","`","EvalPrediction","`","and","return","a","dictionary","string","to","metric","values",".","*","*","optimize_device_cache","*","*","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")","--","Optimize","CUDA","cache","for","slightly","more","memory-efficient","training","."]
code: def prepare_model_inputs(
      self,
      input_ids: torch.Tensor,
      attention_mask: torch.Tensor,
      labels: torch.Tensor,
  ):
      if attention_mask is None:
          attention_mask = [torch.ones_like(ids) for ids in input_ids]
  
      if self.is_encoder_decoder:
          input_data = self.data_collator(
              [
                  {
                      "input_ids": ids,
                      "attention_mask": att,
                      "labels": lab,
                  }
                  for ids, att, lab in zip(
                      input_ids, attention_mask, labels
                  )
              ]
          ).to(self.model.device)
  
          input_data.pop(
              "decoder_input_ids", None
          )  # This is directly computed inside the model
  
          input_data["labels"][
              input_data["labels"] == self.tokenizer.pad_token_id
          ] = -100
  
      else:
          input_data = self.data_collator(
              [
                  {"input_ids": ids, "attention_mask": att}
                  for ids, att in zip(input_ids, attention_mask)
              ]
          ).to(self.model.device)
  
      # truncate in case the user has provided input_ids, attention_mask and labels
      if self.max_length is not None:
          if self.truncation_mode == "keep_start":
              input_data = {
                  k: v[: self.max_length] for k, v in input_data.items()
              }
          elif self.truncation_mode == "keep_end":
              input_data = {
                  k: v[-self.max_length :]
                  for k, v in input_data.items()
              }
          else:
              raise ValueError(
                  f"Unknown truncation mode: {self.truncation_mode}"
              )
  
      return input_data
  
file_path: trl/trl/trainer/iterative_sft_trainer.py
imports: import warnings
  from typing import Callable, Dict, List, Optional, Tuple, Union
  import torch
  from datasets import Dataset
  from torch.utils.data import DataLoader
  from transformers import DataCollator, DataCollatorForLanguageModeling, DataCollatorForSeq2Seq, PreTrainedModel, PreTrainedTokenizerBase, Trainer, TrainingArguments
  from transformers.trainer_utils import EvalLoopOutput
  from ..core import PPODecorators
  from ..import_utils import is_peft_available
label_desc: Use image node_id 2 for calling the IterativeSFTTrainer obj's underlying member method code with example usage: obj.prepare_model_inputs(input_ids, attention_mask, labels) and returns: input_data