class_docstr: The IterativeSFTTrainer can be used to finetune models with methods that requires some steps between optimization.
  
  Attributes:
      **model** (`PreTrainedModel`) -- Model to be optimized, either an 'AutoModelForCausalLM' or an 'AutoModelForSeq2SeqLM'.
          Check the documentation of `PreTrainedModel` for more details.
      **args** (`transformers.TrainingArguments`): -- The arguments to use for training.
      **tokenizer** (`PreTrainedTokenizerBase`) -- Tokenizer to be used for encoding the
          data. Check the documentation of `transformers.PreTrainedTokenizer` and
          `transformers.PreTrainedTokenizerFast` for more details.
      **optimizers** (`Tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): -- The optimizer and scheduler to use for training.
      **data_collator** (Union[DataCollatorForLanguageModeling, DataCollatorForSeq2Seq], *optional*) -- Data collator to be used for training and
          passed along the dataloader.
      **eval_dataset** (`datasets.Dataset`): The dataset to use for evaluation.
      **max_length** (`int`, defaults to `None`): -- The maximum length of the input.
      **truncation_mode** (`str`, defaults to `keep_end`): -- The truncation mode to use, either `keep_end` or `keep_start`.
      **preprocess_logits_for_metrics** (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): -- The function to use to preprocess the logits before computing the metrics.
      **compute_metrics** (`Callable[[EvalPrediction], Dict]`, *optional*): -- The function to use to compute the metrics. Must take a `EvalPrediction` and return a dictionary string to metric values.
      **optimize_device_cache ** (`bool`, *optional*, defaults to `False`) -- Optimize CUDA cache for slightly more memory-efficient training.
class_docstr_tok: ["The","IterativeSFTTrainer","can","be","used","to","finetune","models","with","methods","that","requires","some","steps","between","optimization",".","Attributes",":","*","*","model","*","*","(","`","PreTrainedModel","`",")","--","Model","to","be","optimized",",","either","an","'AutoModelForCausalLM","'","or","an","'AutoModelForSeq2SeqLM","'",".","Check","the","documentation","of","`","PreTrainedModel","`","for","more","details",".","*","*","args","*","*","(","`","transformers.TrainingArguments","`",")",":","--","The","arguments","to","use","for","training",".","*","*","tokenizer","*","*","(","`","PreTrainedTokenizerBase","`",")","--","Tokenizer","to","be","used","for","encoding","the","data",".","Check","the","documentation","of","`","transformers.PreTrainedTokenizer","`","and","`","transformers.PreTrainedTokenizerFast","`","for","more","details",".","*","*","optimizers","*","*","(","`","Tuple","[","torch.optim.Optimizer",",","torch.optim.lr_scheduler.LambdaLR","]","`",")",":","--","The","optimizer","and","scheduler","to","use","for","training",".","*","*","data_collator","*","*","(","Union","[","DataCollatorForLanguageModeling",",","DataCollatorForSeq2Seq","]",",","*","optional","*",")","--","Data","collator","to","be","used","for","training","and","passed","along","the","dataloader",".","*","*","eval_dataset","*","*","(","`","datasets.Dataset","`",")",":","The","dataset","to","use","for","evaluation",".","*","*","max_length","*","*","(","`","int","`",",","defaults","to","`","None","`",")",":","--","The","maximum","length","of","the","input",".","*","*","truncation_mode","*","*","(","`","str","`",",","defaults","to","`","keep_end","`",")",":","--","The","truncation","mode","to","use",",","either","`","keep_end","`","or","`","keep_start","`",".","*","*","preprocess_logits_for_metrics","*","*","(","`","Callable","[","[","torch.Tensor",",","torch.Tensor","]",",","torch.Tensor","]","`",")",":","--","The","function","to","use","to","preprocess","the","logits","before","computing","the","metrics",".","*","*","compute_metrics","*","*","(","`","Callable","[","[","EvalPrediction","]",",","Dict","]","`",",","*","optional","*",")",":","--","The","function","to","use","to","compute","the","metrics",".","Must","take","a","`","EvalPrediction","`","and","return","a","dictionary","string","to","metric","values",".","*","*","optimize_device_cache","*","*","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")","--","Optimize","CUDA","cache","for","slightly","more","memory-efficient","training","."]
code: def __init__(
      self,
      model: PreTrainedModel = None,
      args: TrainingArguments = None,
      tokenizer: PreTrainedTokenizerBase = None,
      optimizers: Tuple[
          torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR
      ] = (
          None,
          None,
      ),
      data_collator: Optional[DataCollator] = None,
      eval_dataset: Optional[Union[Dataset, Dict[str, Dataset]]] = None,
      max_length: Optional[int] = None,
      truncation_mode: Optional[str] = "keep_end",
      preprocess_logits_for_metrics: Optional[
          Callable[[torch.Tensor, torch.Tensor], torch.Tensor]
      ] = None,
      compute_metrics: Optional[
          Callable[[EvalLoopOutput], Dict]
      ] = None,
      optimize_device_cache: Optional[bool] = False,
  ):
      # Step 0: check positional arguments validity
      if not isinstance(tokenizer, (PreTrainedTokenizerBase)):
          raise ValueError(
              f"tokenizer must be a PreTrainedTokenizerBase like a PreTrainedTokenizer or a PreTrainedTokenizerFast, got {type(tokenizer)}"
          )
      if not isinstance(model, PreTrainedModel):
          raise ValueError(
              f"model must be a PreTrainedModel, got {type(model)}"
          )
      if not model.can_generate():
          warnings.warn(
              f"The current model class {type(model)} is not compatible with `.generate()`"
              "Please make sure that this is intended."
          )
      if optimizers[1] is None and args.max_steps == -1:
          raise ValueError(
              "When no scheduler is provided, you need to set the total number of training steps to perform `max_steps`"
          )
  
      self.is_encoder_decoder = getattr(
          model.config, "is_encoder_decoder", False
      )
      self.is_peft_model = is_peft_available() and isinstance(
          model, PeftModel
      )
  
      self.tokenizer = tokenizer
  
      if data_collator is None:
          if self.is_encoder_decoder:
              warnings.warn(
                  "No data collator is provided. Using 'DataCollatorForSeq2Seq' with"
                  "'labels_pad_token_id' set to '-100' and 'pad_to_multiple_of' set to 8."
              )
              self.data_collator = DataCollatorForSeq2Seq(
                  tokenizer,
                  label_pad_token_id=-100,
                  pad_to_multiple_of=8,
              )
          else:
              warnings.warn(
                  "No data collator is provided. Using 'DataCollatorForLanguageModeling'"
              )
              self.data_collator = DataCollatorForLanguageModeling(
                  self.tokenizer, mlm=False
              )
      else:
          self.data_collator = data_collator
  
      self.max_length = max_length
      self.truncation_mode = truncation_mode
      self.optimize_device_cache = optimize_device_cache
  
      super().__init__(
          model=model,
          args=args,
          data_collator=self.data_collator,
          eval_dataset=eval_dataset,
          tokenizer=tokenizer,
          compute_metrics=compute_metrics,
          optimizers=optimizers,
          preprocess_logits_for_metrics=preprocess_logits_for_metrics,
      )
  
      self.create_optimizer_and_scheduler(self.args.max_steps)
  
      # prepare model, optimizer and lr_scheduler
      (
          self.model,
          self.optimizer,
          self.lr_scheduler,
      ) = self.accelerator.prepare(
          self.model, self.optimizer, self.lr_scheduler
      )
  
      self.tokenizer.truncation_side = (
          "left" if self.truncation_mode == "keep_end" else "right"
      )
  
      if not hasattr(self, "accelerator"):
          raise AttributeError(
              "Your `Trainer` does not have an `accelerator` object. Consider upgrading `transformers`."
          )
  
      PPODecorators.optimize_device_cache = self.optimize_device_cache
  
file_path: trl/trl/trainer/iterative_sft_trainer.py
imports: import warnings
  from typing import Callable, Dict, List, Optional, Tuple, Union
  import torch
  from datasets import Dataset
  from torch.utils.data import DataLoader
  from transformers import DataCollator, DataCollatorForLanguageModeling, DataCollatorForSeq2Seq, PreTrainedModel, PreTrainedTokenizerBase, Trainer, TrainingArguments
  from transformers.trainer_utils import EvalLoopOutput
  from ..core import PPODecorators
  from ..import_utils import is_peft_available
label_desc: Use image node_id 1 to create a new IterativeSFTTrainer object from inherited base classes: Trainer with example: obj = IterativeSFTTrainer(model, args, tokenizer, optimizers, data_collator, eval_dataset, max_length, truncation_mode, preprocess_logits_for_metrics, compute_metrics, optimize_device_cache)