class_docstr: The IterativeSFTTrainer can be used to finetune models with methods that requires some steps between optimization.
  
  Attributes:
      **model** (`PreTrainedModel`) -- Model to be optimized, either an 'AutoModelForCausalLM' or an 'AutoModelForSeq2SeqLM'.
          Check the documentation of `PreTrainedModel` for more details.
      **args** (`transformers.TrainingArguments`): -- The arguments to use for training.
      **tokenizer** (`PreTrainedTokenizerBase`) -- Tokenizer to be used for encoding the
          data. Check the documentation of `transformers.PreTrainedTokenizer` and
          `transformers.PreTrainedTokenizerFast` for more details.
      **optimizers** (`Tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): -- The optimizer and scheduler to use for training.
      **data_collator** (Union[DataCollatorForLanguageModeling, DataCollatorForSeq2Seq], *optional*) -- Data collator to be used for training and
          passed along the dataloader.
      **eval_dataset** (`datasets.Dataset`): The dataset to use for evaluation.
      **max_length** (`int`, defaults to `None`): -- The maximum length of the input.
      **truncation_mode** (`str`, defaults to `keep_end`): -- The truncation mode to use, either `keep_end` or `keep_start`.
      **preprocess_logits_for_metrics** (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): -- The function to use to preprocess the logits before computing the metrics.
      **compute_metrics** (`Callable[[EvalPrediction], Dict]`, *optional*): -- The function to use to compute the metrics. Must take a `EvalPrediction` and return a dictionary string to metric values.
      **optimize_device_cache ** (`bool`, *optional*, defaults to `False`) -- Optimize CUDA cache for slightly more memory-efficient training.
class_docstr_tok: ["The","IterativeSFTTrainer","can","be","used","to","finetune","models","with","methods","that","requires","some","steps","between","optimization",".","Attributes",":","*","*","model","*","*","(","`","PreTrainedModel","`",")","--","Model","to","be","optimized",",","either","an","'AutoModelForCausalLM","'","or","an","'AutoModelForSeq2SeqLM","'",".","Check","the","documentation","of","`","PreTrainedModel","`","for","more","details",".","*","*","args","*","*","(","`","transformers.TrainingArguments","`",")",":","--","The","arguments","to","use","for","training",".","*","*","tokenizer","*","*","(","`","PreTrainedTokenizerBase","`",")","--","Tokenizer","to","be","used","for","encoding","the","data",".","Check","the","documentation","of","`","transformers.PreTrainedTokenizer","`","and","`","transformers.PreTrainedTokenizerFast","`","for","more","details",".","*","*","optimizers","*","*","(","`","Tuple","[","torch.optim.Optimizer",",","torch.optim.lr_scheduler.LambdaLR","]","`",")",":","--","The","optimizer","and","scheduler","to","use","for","training",".","*","*","data_collator","*","*","(","Union","[","DataCollatorForLanguageModeling",",","DataCollatorForSeq2Seq","]",",","*","optional","*",")","--","Data","collator","to","be","used","for","training","and","passed","along","the","dataloader",".","*","*","eval_dataset","*","*","(","`","datasets.Dataset","`",")",":","The","dataset","to","use","for","evaluation",".","*","*","max_length","*","*","(","`","int","`",",","defaults","to","`","None","`",")",":","--","The","maximum","length","of","the","input",".","*","*","truncation_mode","*","*","(","`","str","`",",","defaults","to","`","keep_end","`",")",":","--","The","truncation","mode","to","use",",","either","`","keep_end","`","or","`","keep_start","`",".","*","*","preprocess_logits_for_metrics","*","*","(","`","Callable","[","[","torch.Tensor",",","torch.Tensor","]",",","torch.Tensor","]","`",")",":","--","The","function","to","use","to","preprocess","the","logits","before","computing","the","metrics",".","*","*","compute_metrics","*","*","(","`","Callable","[","[","EvalPrediction","]",",","Dict","]","`",",","*","optional","*",")",":","--","The","function","to","use","to","compute","the","metrics",".","Must","take","a","`","EvalPrediction","`","and","return","a","dictionary","string","to","metric","values",".","*","*","optimize_device_cache","*","*","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")","--","Optimize","CUDA","cache","for","slightly","more","memory-efficient","training","."]
docstr: Run an optimisation step given a list of input_ids, attention_mask, and labels or a list of text and text_labels.
  Args:
      input_ids (List[`torch.LongTensor`]):
          List of tensors containing the input_ids (if not provided, text will be used)
      attention_mask (List[`torch.LongTensor`], , *optional*):
          List of tensors containing the attention_mask
      labels (List[`torch.FloatTensor`], *optional*):
          List of tensors containing the labels (if set to None, will default to input_ids)
      texts (List[`str`], *optional*):
          List of strings containing the text input (if not provided, input_ids will directly be used)
      texts_labels (List[`str`], *optional*):
          List of strings containing the text labels (if set to None, will default to text)
  Returns:
      `dict[str, Any]`: A summary of the training statistics
docstr_tok: ["Run","an","optimisation","step","given","a","list","of","input_ids",",","attention_mask",",","and","labels","or","a","list","of","text","and","text_labels",".","Args",":","input_ids","(","List","[","`","torch.LongTensor","`","]",")",":","List","of","tensors","containing","the","input_ids","(","if","not","provided",",","text","will","be","used",")","attention_mask","(","List","[","`","torch.LongTensor","`","]",",",",","*","optional","*",")",":","List","of","tensors","containing","the","attention_mask","labels","(","List","[","`","torch.FloatTensor","`","]",",","*","optional","*",")",":","List","of","tensors","containing","the","labels","(","if","set","to","None",",","will","default","to","input_ids",")","texts","(","List","[","`","str","`","]",",","*","optional","*",")",":","List","of","strings","containing","the","text","input","(","if","not","provided",",","input_ids","will","directly","be","used",")","texts_labels","(","List","[","`","str","`","]",",","*","optional","*",")",":","List","of","strings","containing","the","text","labels","(","if","set","to","None",",","will","default","to","text",")","Returns",":","`","dict","[","str",",","Any","]","`",":","A","summary","of","the","training","statistics"]
code: def step(
      self,
      input_ids: Optional[List[torch.LongTensor]] = None,
      attention_mask: Optional[List[torch.LongTensor]] = None,
      labels: Optional[List[torch.LongTensor]] = None,
      texts: Optional[List[str]] = None,
      texts_labels: Optional[List[str]] = None,
  ):
      """
      Run an optimisation step given a list of input_ids, attention_mask, and labels or a list of text and text_labels.
      Args:
          input_ids (List[`torch.LongTensor`]):
              List of tensors containing the input_ids (if not provided, text will be used)
          attention_mask (List[`torch.LongTensor`], , *optional*):
              List of tensors containing the attention_mask
          labels (List[`torch.FloatTensor`], *optional*):
              List of tensors containing the labels (if set to None, will default to input_ids)
          texts (List[`str`], *optional*):
              List of strings containing the text input (if not provided, input_ids will directly be used)
          texts_labels (List[`str`], *optional*):
              List of strings containing the text labels (if set to None, will default to text)
      Returns:
          `dict[str, Any]`: A summary of the training statistics
      """
      self.model.train()
  
      if self.state.global_step == 0:
          self.tr_loss = torch.tensor(0.0).to(self.args.device)
          self._globalstep_last_logged = self.state.global_step
  
      if input_ids is None and texts is None:
          raise ValueError(
              "Step should include `input_ids` or `texts` as keyword arguments."
          )
      elif input_ids is not None and texts is not None:
          warnings.warn(
              "Both 'input_ids' and 'texts' are provided. 'input_ids' will be overwritten using inputs provided by the 'texts' keyword argument."
          )
  
      if (
          labels is None
          and texts_labels is None
          and self.is_encoder_decoder
      ):
          raise ValueError(
              "No 'labels' or 'text_labels' are provided. When using an encoder-decoder architecture, 'labels' or 'text_labels' must be passed."
          )
  
      (
          input_ids,
          attention_mask,
          labels,
          texts,
          texts_labels,
      ) = self._step_safety_checker(
          input_ids, attention_mask, labels, texts, texts_labels
      )
  
      if texts is not None:
          model_inputs = self.tokenizer(
              texts,
              max_length=self.max_length,
              truncation=True,
              padding=True,
              return_tensors="pt",
          )
  
          input_ids, attention_mask = (
              model_inputs["input_ids"],
              model_inputs["attention_mask"],
          )
  
      if texts_labels is not None:
          labels = self.tokenizer(
              texts,
              max_length=self.max_length,
              truncation=True,
              padding=True,
              return_tensors="pt",
          )["input_ids"]
  
      if labels is None:
          warnings.warn(
              "No labels are provided. Setting labels to input_ids"
          )
          labels = input_ids
  
      model_inputs = self.prepare_model_inputs(
          input_ids, attention_mask, labels
      )
  
      model_inputs_names = list(model_inputs.keys())
  
      batch_dict = {}
      batch_dict.update(model_inputs)
  
      def collator(data):
          return_dict = dict()
          for key in data[0]:
              if key in ["input_ids", "attention_mask", "labels"]:
                  return_dict[key] = torch.stack(
                      [d[key] for d in data]
                  ).to(self.model.device)
          return return_dict
  
      batch_data = Dataset.from_dict(batch_dict)
      batch_data.set_format("torch")
  
      step_dataloader = DataLoader(
          batch_data,
          batch_size=self.args.per_device_train_batch_size,
          shuffle=True,
          collate_fn=collator,
      )
  
      for _, batch in enumerate(step_dataloader):
          with self.accelerator.accumulate(self.model):
              model_inputs = {k: batch[k] for k in model_inputs_names}
              loss = self.compute_loss(self.model, model_inputs)
  
              if self.args.n_gpu > 1:
                  loss = loss.mean()
  
              tr_loss_step = loss.detach()
  
              self.accelerator.backward(loss)
  
              if (
                  self.accelerator.sync_gradients
                  and self.args.max_grad_norm is not None
              ):
                  self.accelerator.clip_grad_norm_(
                      self.model.parameters(),
                      self.args.max_grad_norm,
                  )
  
              self.optimizer.step()
              self.optimizer.zero_grad()
              if self.lr_scheduler is not None:
                  self.lr_scheduler.step()
  
              self.state.global_step += 1
  
              # update stats etc
              self.tr_loss += tr_loss_step
  
              self._maybe_log_save_evaluate()
  
file_path: trl/trl/trainer/iterative_sft_trainer.py
imports: import warnings
  from typing import Callable, Dict, List, Optional, Tuple, Union
  import torch
  from datasets import Dataset
  from torch.utils.data import DataLoader
  from transformers import DataCollator, DataCollatorForLanguageModeling, DataCollatorForSeq2Seq, PreTrainedModel, PreTrainedTokenizerBase, Trainer, TrainingArguments
  from transformers.trainer_utils import EvalLoopOutput
  from ..core import PPODecorators
  from ..import_utils import is_peft_available
label_desc: Use image node_id 4 for calling the IterativeSFTTrainer obj's underlying member method code with example usage: obj.step(input_ids, attention_mask, labels, texts, texts_labels) and returns: return_dict