class_docstr: The PPOTrainer uses Proximal Policy Optimization to optimise language models.
  Note, this trainer is heavily inspired by the original OpenAI learning to summarize
  work here: https://github.com/openai/summarize-from-feedback
  
  Attributes:
      **cfg** (`LLM Studio Config`) -- Experiment configuration object. Check the
          documentation of `LLM Studio Config` for more details.
      **model** (`PreTrainedModelWrapper`) -- Model to be optimized, Hugging Face
          transformer model with a value head. Check the documentation of
          `PreTrainedModelWrapper` for more details.
      **tokenizer** (`Union[PreTrainedTokenizer, PreTrainedTokenizerFast]`)
          Tokenizer to be used for encoding the data. Check the documentation of
          `transformers.PreTrainedTokenizer` and
          `transformers.PreTrainedTokenizerFast` for more details.
      **optimizer** (`torch.optim.Optimizer`) -- Optimizer to be used for training.
      **lr_scheduler** (`torch.optim.lr_scheduler`) -- Learning rate scheduler to be
          used for training.
      **scaler** (`torch.cuda.amp.GradScaler`) -- Gradient scaler to be used for
          training.
class_docstr_tok: ["The","PPOTrainer","uses","Proximal","Policy","Optimization","to","optimise","language","models",".","Note",",","this","trainer","is","heavily","inspired","by","the","original","OpenAI","learning","to","summarize","work","here",":","https",":","\/\/github.com\/openai\/summarize-from-feedback","Attributes",":","*","*","cfg","*","*","(","`","LLM","Studio","Config","`",")","--","Experiment","configuration","object",".","Check","the","documentation","of","`","LLM","Studio","Config","`","for","more","details",".","*","*","model","*","*","(","`","PreTrainedModelWrapper","`",")","--","Model","to","be","optimized",",","Hugging","Face","transformer","model","with","a","value","head",".","Check","the","documentation","of","`","PreTrainedModelWrapper","`","for","more","details",".","*","*","tokenizer","*","*","(","`","Union","[","PreTrainedTokenizer",",","PreTrainedTokenizerFast","]","`",")","Tokenizer","to","be","used","for","encoding","the","data",".","Check","the","documentation","of","`","transformers.PreTrainedTokenizer","`","and","`","transformers.PreTrainedTokenizerFast","`","for","more","details",".","*","*","optimizer","*","*","(","`","torch.optim.Optimizer","`",")","--","Optimizer","to","be","used","for","training",".","*","*","lr_scheduler","*","*","(","`","torch.optim.lr_scheduler","`",")","--","Learning","rate","scheduler","to","be","used","for","training",".","*","*","scaler","*","*","(","`","torch.cuda.amp.GradScaler","`",")","--","Gradient","scaler","to","be","used","for","training","."]
docstr: Check if the input data is valid for training and move the data to the correct
  device.
  
  Args:
      queries (List[`torch.LongTensor`]):
          List of tensors containing the encoded queries of shape (`query_length`)
      responses (List[`torch.LongTensor`]):
          List of tensors containing the encoded responses of shape
          (`response_length`)
      scores (List[`torch.Tensor`]):
          List of tensors containing the scores.
  Returns:
      `tuple`: The input processed data.
code: def _step_safety_checker(
      self,
      queries: List[torch.LongTensor],
      responses: List[torch.LongTensor],
      scores: List[torch.Tensor],
  ):
      """
      Check if the input data is valid for training and move the data to the correct
      device.
  
      Args:
          queries (List[`torch.LongTensor`]):
              List of tensors containing the encoded queries of shape (`query_length`)
          responses (List[`torch.LongTensor`]):
              List of tensors containing the encoded responses of shape
              (`response_length`)
          scores (List[`torch.Tensor`]):
              List of tensors containing the scores.
      Returns:
          `tuple`: The input processed data.
      """
      for name, tensor_list in zip(
          ["queries", "responses", "scores"],
          [queries, responses, scores],
      ):
          if not isinstance(tensor_list, list):
              raise ValueError(
                  f"{name} must be a list of tensors - got {type(tensor_list)}"
              )
          if not isinstance(tensor_list[0], torch.Tensor):
              raise ValueError(
                  f"Elements in {name} must be tensors - got {type(tensor_list[0])}"
              )
          if self.cfg.training.batch_size is not None and len(
              tensor_list
          ) != (
              self.cfg.training.batch_size
              * self.cfg.training.rollout_steps
          ):
              raise ValueError(
                  f"Batch size ({self.cfg.training.batch_size}) does not match "
                  "number of examples"
                  f" - but got {len(tensor_list)} for: {name}"
              )
  
      # add queries, scores and responses on the correct device
      queries = [tensor.to(self.current_device) for tensor in queries]
      responses = [
          tensor.to(self.current_device) for tensor in responses
      ]
      scores = [tensor.to(self.current_device) for tensor in scores]
  
      # squeeze scores if needed
      for i, score in enumerate(scores):
          if score.dim() > 1:
              raise ValueError(
                  f"Scores must be 1-dimensional - got {score.dim()} for {score}"
              )
          elif score.dim() == 1:
              scores[i] = score.squeeze()
  
      return queries, responses, scores
  
imports: import math
  import time
  import warnings
  from collections.abc import Mapping
  from typing import Any, Dict, List, Optional, Union
  import numpy
  import torch
  import torch.nn.functional
  from datasets import Dataset
  from huggingface_hub import PyTorchModelHubMixin
  from torch.cuda.amp import autocast
  from torch.nn.utils.rnn import pad_sequence
  from torch.utils.data import DataLoader
  from transformers import DataCollatorForLanguageModeling, PreTrainedTokenizer, PreTrainedTokenizerFast
  from llm_studio.src.utils.modeling_utils import unwrap_model
label_desc: Use image node_id 2 for calling the PPOTrainer obj's underlying member method code with example usage: obj._step_safety_checker(queries, responses, scores) and returns: queries,  responses,  scores