class_docstr: Class definition of the Supervised Finetuning Trainer (SFT Trainer).
  This class is a wrapper around the `transformers.Trainer` class and inherits all of its attributes and methods.
  The trainer takes care of properly initializing the PeftModel in case a user passes a `PeftConfig` object.
  
  Args:
      model (Union[`transformers.PreTrainedModel`, `nn.Module`, `str`]):
          The model to train, can be a `PreTrainedModel`, a `torch.nn.Module` or a string with the model name to
          load from cache or download. The model can be also converted to a `PeftModel` if a `PeftConfig` object is
          passed to the `peft_config` argument.
      args (Optional[`transformers.TrainingArguments`]):
          The arguments to tweak for training. Please refer to the official documentation of `transformers.TrainingArguments`
          for more information.
      data_collator (Optional[`transformers.DataCollator`]):
          The data collator to use for training.
      train_dataset (Optional[`datasets.Dataset`]):
          The dataset to use for training. We recommend users to use `trl.trainer.ConstantLengthDataset` to create their dataset.
      eval_dataset (Optional[Union[`datasets.Dataset`, Dict[`str`, `datasets.Dataset`]]]):
          The dataset to use for evaluation. We recommend users to use `trl.trainer.ConstantLengthDataset` to create their dataset.
      tokenizer (Optional[`transformers.PreTrainedTokenizer`]):
          The tokenizer to use for training. If not specified, the tokenizer associated to the model will be used.
      model_init (`Callable[[], transformers.PreTrainedModel]`):
          The model initializer to use for training. If None is specified, the default model initializer will be used.
      compute_metrics (`Callable[[transformers.EvalPrediction], Dict]`, *optional* defaults to None):
          The function used to compute metrics during evaluation. It should return a dictionary mapping metric names to metric values.
          If not specified, only the loss will be computed during evaluation.
      callbacks (`List[transformers.TrainerCallback]`):
          The callbacks to use for training.
      optimizers (`Tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`):
          The optimizer and scheduler to use for training.
      preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`):
          The function to use to preprocess the logits before computing the metrics.
      peft_config (`Optional[PeftConfig]`):
          The PeftConfig object to use to initialize the PeftModel.
      dataset_text_field (`Optional[str]`):
          The name of the text field of the dataset, in case this is passed by a user, the trainer will automatically create a
          `ConstantLengthDataset` based on the `dataset_text_field` argument.
      formatting_func (`Optional[Callable]`):
          The formatting function to be used for creating the `ConstantLengthDataset`.
      max_seq_length (`Optional[int]`):
          The maximum sequence length to use for the `ConstantLengthDataset` and for automatically creating the Dataset. Defaults to `512`.
      infinite (`Optional[bool]`):
          Whether to use an infinite dataset or not. Defaults to `False`.
      num_of_sequences (`Optional[int]`):
          The number of sequences to use for the `ConstantLengthDataset`. Defaults to `1024`.
      chars_per_token (`Optional[float]`):
          The number of characters per token to use for the `ConstantLengthDataset`. Defaults to `3.6`. You can check how this is computed in the
          stack-llama example: https://github.com/huggingface/trl/blob/08f550674c553c36c51d1027613c29f14f3676a5/examples/stack_llama/scripts/supervised_finetuning.py#L53.
      packing (`Optional[bool]`):
          Used only in case `dataset_text_field` is passed. This argument is used by the `ConstantLengthDataset` to pack the sequences
          of the dataset.
      dataset_num_proc (`Optional[int]`):
          The number of workers to use to tokenize the data. Only used when `packing=False`. Defaults to None.
      dataset_batch_size (`int`):
          The number of examples to tokenize per batch. If batch_size <= 0 or batch_size == None,
          tokenize the full dataset as a single batch. Defaults to 1000.
      neftune_noise_alpha (`Optional[float]`):
          If not `None`, this will activate NEFTune noise embeddings. This has been proven to drastically improve model performances for instruction
          fine-tuning. Check out the original paper here: https://arxiv.org/abs/2310.05914 and the original code here: https://github.com/neelsjain/NEFTune
      model_init_kwargs: (`Optional[Dict]`, *optional*):
          Dict of Optional kwargs to pass when instantiating the model from a string
      dataset_kwargs: (`Optional[Dict]`, *optional*):
          Dict of Optional kwargs to pass when creating packed or non-packed datasets
class_docstr_tok: ["Class","definition","of","the","Supervised","Finetuning","Trainer","(","SFT","Trainer",")",".","This","class","is","a","wrapper","around","the","`","transformers.Trainer","`","class","and","inherits","all","of","its","attributes","and","methods",".","The","trainer","takes","care","of","properly","initializing","the","PeftModel","in","case","a","user","passes","a","`","PeftConfig","`","object",".","Args",":","model","(","Union","[","`","transformers.PreTrainedModel","`",",","`","nn.Module","`",",","`","str","`","]",")",":","The","model","to","train",",","can","be","a","`","PreTrainedModel","`",",","a","`","torch.nn.Module","`","or","a","string","with","the","model","name","to","load","from","cache","or","download",".","The","model","can","be","also","converted","to","a","`","PeftModel","`","if","a","`","PeftConfig","`","object","is","passed","to","the","`","peft_config","`","argument",".","args","(","Optional","[","`","transformers.TrainingArguments","`","]",")",":","The","arguments","to","tweak","for","training",".","Please","refer","to","the","official","documentation","of","`","transformers.TrainingArguments","`","for","more","information",".","data_collator","(","Optional","[","`","transformers.DataCollator","`","]",")",":","The","data","collator","to","use","for","training",".","train_dataset","(","Optional","[","`","datasets.Dataset","`","]",")",":","The","dataset","to","use","for","training",".","We","recommend","users","to","use","`","trl.trainer.ConstantLengthDataset","`","to","create","their","dataset",".","eval_dataset","(","Optional","[","Union","[","`","datasets.Dataset","`",",","Dict","[","`","str","`",",","`","datasets.Dataset","`","]","]","]",")",":","The","dataset","to","use","for","evaluation",".","We","recommend","users","to","use","`","trl.trainer.ConstantLengthDataset","`","to","create","their","dataset",".","tokenizer","(","Optional","[","`","transformers.PreTrainedTokenizer","`","]",")",":","The","tokenizer","to","use","for","training",".","If","not","specified",",","the","tokenizer","associated","to","the","model","will","be","used",".","model_init","(","`","Callable","[","[","]",",","transformers.PreTrainedModel","]","`",")",":","The","model","initializer","to","use","for","training",".","If","None","is","specified",",","the","default","model","initializer","will","be","used",".","compute_metrics","(","`","Callable","[","[","transformers.EvalPrediction","]",",","Dict","]","`",",","*","optional","*","defaults","to","None",")",":","The","function","used","to","compute","metrics","during","evaluation",".","It","should","return","a","dictionary","mapping","metric","names","to","metric","values",".","If","not","specified",",","only","the","loss","will","be","computed","during","evaluation",".","callbacks","(","`","List","[","transformers.TrainerCallback","]","`",")",":","The","callbacks","to","use","for","training",".","optimizers","(","`","Tuple","[","torch.optim.Optimizer",",","torch.optim.lr_scheduler.LambdaLR","]","`",")",":","The","optimizer","and","scheduler","to","use","for","training",".","preprocess_logits_for_metrics","(","`","Callable","[","[","torch.Tensor",",","torch.Tensor","]",",","torch.Tensor","]","`",")",":","The","function","to","use","to","preprocess","the","logits","before","computing","the","metrics",".","peft_config","(","`","Optional","[","PeftConfig","]","`",")",":","The","PeftConfig","object","to","use","to","initialize","the","PeftModel",".","dataset_text_field","(","`","Optional","[","str","]","`",")",":","The","name","of","the","text","field","of","the","dataset",",","in","case","this","is","passed","by","a","user",",","the","trainer","will","automatically","create","a","`","ConstantLengthDataset","`","based","on","the","`","dataset_text_field","`","argument",".","formatting_func","(","`","Optional","[","Callable","]","`",")",":","The","formatting","function","to","be","used","for","creating","the","`","ConstantLengthDataset","`",".","max_seq_length","(","`","Optional","[","int","]","`",")",":","The","maximum","sequence","length","to","use","for","the","`","ConstantLengthDataset","`","and","for","automatically","creating","the","Dataset",".","Defaults","to","`","512","`",".","infinite","(","`","Optional","[","bool","]","`",")",":","Whether","to","use","an","infinite","dataset","or","not",".","Defaults","to","`","False","`",".","num_of_sequences","(","`","Optional","[","int","]","`",")",":","The","number","of","sequences","to","use","for","the","`","ConstantLengthDataset","`",".","Defaults","to","`","1024","`",".","chars_per_token","(","`","Optional","[","float","]","`",")",":","The","number","of","characters","per","token","to","use","for","the","`","ConstantLengthDataset","`",".","Defaults","to","`","3.6","`",".","You","can","check","how","this","is","computed","in","the","stack-llama","example",":","https",":","\/\/github.com\/huggingface\/trl\/blob\/08f550674c553c36c51d1027613c29f14f3676a5\/examples\/stack_llama\/scripts\/supervised_finetuning.py","#","L53",".","packing","(","`","Optional","[","bool","]","`",")",":","Used","only","in","case","`","dataset_text_field","`","is","passed",".","This","argument","is","used","by","the","`","ConstantLengthDataset","`","to","pack","the","sequences","of","the","dataset",".","dataset_num_proc","(","`","Optional","[","int","]","`",")",":","The","number","of","workers","to","use","to","tokenize","the","data",".","Only","used","when","`","packing=False","`",".","Defaults","to","None",".","dataset_batch_size","(","`","int","`",")",":","The","number","of","examples","to","tokenize","per","batch",".","If","batch_size","<","=","0","or","batch_size","==","None",",","tokenize","the","full","dataset","as","a","single","batch",".","Defaults","to","1000.","neftune_noise_alpha","(","`","Optional","[","float","]","`",")",":","If","not","`","None","`",",","this","will","activate","NEFTune","noise","embeddings",".","This","has","been","proven","to","drastically","improve","model","performances","for","instruction","fine-tuning",".","Check","out","the","original","paper","here",":","https",":","\/\/arxiv.org\/abs\/2310.05914","and","the","original","code","here",":","https",":","\/\/github.com\/neelsjain\/NEFTune","model_init_kwargs",":","(","`","Optional","[","Dict","]","`",",","*","optional","*",")",":","Dict","of","Optional","kwargs","to","pass","when","instantiating","the","model","from","a","string","dataset_kwargs",":","(","`","Optional","[","Dict","]","`",",","*","optional","*",")",":","Dict","of","Optional","kwargs","to","pass","when","creating","packed","or","non-packed","datasets"]
docstr: Activates the neftune as presented in this code: https://github.com/neelsjain/NEFTune and paper: https://arxiv.org/abs/2310.05914
  Since in transformers Trainer we do have an `_activate_neftune` method, we need to rename this method to avoid conflicts.
docstr_tok: ["Activates","the","neftune","as","presented","in","this","code",":","https",":","\/\/github.com\/neelsjain\/NEFTune","and","paper",":","https",":","\/\/arxiv.org\/abs\/2310.05914","Since","in","transformers","Trainer","we","do","have","an","`","_activate_neftune","`","method",",","we","need","to","rename","this","method","to","avoid","conflicts","."]
code: def _trl_activate_neftune(self, model):
      r"""
      Activates the neftune as presented in this code: https://github.com/neelsjain/NEFTune and paper: https://arxiv.org/abs/2310.05914
      Since in transformers Trainer we do have an `_activate_neftune` method, we need to rename this method to avoid conflicts.
      """
      unwrapped_model = unwrap_model(model)
      if is_peft_available() and isinstance(unwrapped_model, PeftModel):
          embeddings = (
              unwrapped_model.base_model.model.get_input_embeddings()
          )
      else:
          embeddings = unwrapped_model.get_input_embeddings()
  
      embeddings.neftune_noise_alpha = self.neftune_noise_alpha
      hook_handle = embeddings.register_forward_hook(
          neftune_post_forward_hook
      )
      self.neftune_hook_handle = hook_handle
      return model
  
imports: import dataclasses
  import inspect
  import warnings
  from functools import wraps
  from typing import Callable, Dict, List, Optional, Tuple, Union
  import torch
  import torch.nn
  from datasets import Dataset
  from datasets.arrow_writer import SchemaInferenceError
  from datasets.builder import DatasetGenerationError
  from transformers import AutoModelForCausalLM, AutoTokenizer, DataCollator, DataCollatorForLanguageModeling, PreTrainedModel, PreTrainedTokenizerBase, Trainer, TrainingArguments
  from transformers.modeling_utils import unwrap_model
  from transformers.trainer_callback import TrainerCallback
  from transformers.trainer_utils import EvalPrediction
  from ..extras.dataset_formatting import get_formatting_func_from_dataset
  from ..import_utils import is_peft_available
  from .utils import ConstantLengthDataset, DataCollatorForCompletionOnlyLM, neftune_post_forward_hook, peft_module_casting_to_bf16, trl_sanitze_kwargs_for_tagging
label_desc: Use image node_id 7 for calling the SFTTrainer obj's underlying member method code with example usage: obj._trl_activate_neftune(model) and returns: model