docstr: Initializes a HuggingFace `AutoModel` and `AutoTokenizer` for evaluation.
  Args:
      pretrained (str):
          The HuggingFace Hub model ID name or the path to a pre-trained
          model to load. This is effectively the `pretrained_model_name_or_path`
          argument of `from_pretrained` in the HuggingFace `transformers` API.
      quantized (str or True, optional, defaults to None):
          File name of a GPTQ quantized model to load. Set to `True` to use the
          default name of the quantized model.
      add_special_tokens (bool, optional, defaults to True):
          Whether to add special tokens to the input sequences. If `None`, the
          default value will be set to `True` for seq2seq models (e.g. T5) and
          `False` for causal models.
          WARNING: Evaluating causal models with `add_special_tokens=True` is
          currently __not__ supported.
      > Large model loading `accelerate` arguments
      use_accelerate (bool, optional, defaults to False):
          If True, uses the `accelerate` library to load a large model across
          multiple devices.
      device_map_option (str, optional, defaults to "auto"):
          The device map option to use when loading the model with
          `accelerate`.
          Options:
              "auto", "balanced", "balanced_low_0", "sequential"
          See the `accelerate` docs for more details on these options:
          https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.device_map
      max_memory_per_gpu (Union[int, str], optional, defaults to None):
          The maximum memory available for each GPU in bytes as `int` or in
          the format f"{significand}{unit_symbol}" where {unit_symbol} is
          any of ["GB", "MB", "GIB", "MIB"]. Refer to the `max_memory` arg in
          the "Parameters for big model inference" section of the following
          docs:
          https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.max_memory
      max_cpu_memory (Union[int, str], optional, defaults to None):
          The maximum available CPU RAM in bytes as `int` or in the format
          f"{significand}{unit_symbol}" where {unit_symbol} is any of
          ["GB", "MB", "GIB", "MIB"]. Refer to the `max_memory` arg in the
          "Parameters for big model inference" section of the following docs:
          https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.max_memory
      offload_folder (str, optional, defaults to "./offload"):
          The folder to offload weights into if `device_map` contains any
          "disk" value.
      dtype (Union[str, torch.dtype], optional, defaults to None):):
          Converts the model weights to `dtype`, if specified. Strings get
          converted to `torch.dtype` objects (e.g. `float16` -> `torch.float16`).
          Use `dtype="auto"` to derive the type from the model’s weights.
      peft (str, optional, defaults to None):
          Path of the adapter weights to load from Huggingface. This will usually
          include a directory that includes the files `adapter_config.json` and
          `adapter_model.bin`. Compatible with [PEFT](https://github.com/huggingface/peft)
      load_in_8bit (bool, optional, defaults to False):
          If True, will convert the loaded model into mixed-8bit quantized model. See:
          https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.load_in_8bit
      trust_remote_code (bool, optional, defaults to False):
          If True, will trust the remote code when loading the model.
      use_fast (bool, optional, defaults to True):
          If True, will use the fast tokenizer when loading the model.
      gptq_use_triton (bool, optional, defaults to False):
          Use Triton for GPTQ inference.
docstr_tok: ["Initializes","a","HuggingFace","`","AutoModel","`","and","`","AutoTokenizer","`","for","evaluation",".","Args",":","pretrained","(","str",")",":","The","HuggingFace","Hub","model","ID","name","or","the","path","to","a","pre-trained","model","to","load",".","This","is","effectively","the","`","pretrained_model_name_or_path","`","argument","of","`","from_pretrained","`","in","the","HuggingFace","`","transformers","`","API",".","quantized","(","str","or","True",",","optional",",","defaults","to","None",")",":","File","name","of","a","GPTQ","quantized","model","to","load",".","Set","to","`","True","`","to","use","the","default","name","of","the","quantized","model",".","add_special_tokens","(","bool",",","optional",",","defaults","to","True",")",":","Whether","to","add","special","tokens","to","the","input","sequences",".","If","`","None","`",",","the","default","value","will","be","set","to","`","True","`","for","seq2seq","models","(","e.g",".","T5",")","and","`","False","`","for","causal","models",".","WARNING",":","Evaluating","causal","models","with","`","add_special_tokens=True","`","is","currently","__not__","supported",".",">","Large","model","loading","`","accelerate","`","arguments","use_accelerate","(","bool",",","optional",",","defaults","to","False",")",":","If","True",",","uses","the","`","accelerate","`","library","to","load","a","large","model","across","multiple","devices",".","device_map_option","(","str",",","optional",",","defaults","to","``","auto","''",")",":","The","device","map","option","to","use","when","loading","the","model","with","`","accelerate","`",".","Options",":","``","auto","''",",","``","balanced","''",",","``","balanced_low_0","''",",","``","sequential","''","See","the","`","accelerate","`","docs","for","more","details","on","these","options",":","https",":","\/\/huggingface.co\/docs\/transformers\/main\/en\/main_classes\/model","#","transformers.PreTrainedModel.from_pretrained.device_map","max_memory_per_gpu","(","Union","[","int",",","str","]",",","optional",",","defaults","to","None",")",":","The","maximum","memory","available","for","each","GPU","in","bytes","as","`","int","`","or","in","the","format","f","''","{","significand","}","{","unit_symbol","}","''","where","{","unit_symbol","}","is","any","of","[","``","GB","''",",","``","MB","''",",","``","GIB","''",",","``","MIB","''","]",".","Refer","to","the","`","max_memory","`","arg","in","the","``","Parameters","for","big","model","inference","''","section","of","the","following","docs",":","https",":","\/\/huggingface.co\/docs\/transformers\/main\/en\/main_classes\/model","#","transformers.PreTrainedModel.from_pretrained.max_memory","max_cpu_memory","(","Union","[","int",",","str","]",",","optional",",","defaults","to","None",")",":","The","maximum","available","CPU","RAM","in","bytes","as","`","int","`","or","in","the","format","f","''","{","significand","}","{","unit_symbol","}","''","where","{","unit_symbol","}","is","any","of","[","``","GB","''",",","``","MB","''",",","``","GIB","''",",","``","MIB","''","]",".","Refer","to","the","`","max_memory","`","arg","in","the","``","Parameters","for","big","model","inference","''","section","of","the","following","docs",":","https",":","\/\/huggingface.co\/docs\/transformers\/main\/en\/main_classes\/model","#","transformers.PreTrainedModel.from_pretrained.max_memory","offload_folder","(","str",",","optional",",","defaults","to","``",".\/offload","''",")",":","The","folder","to","offload","weights","into","if","`","device_map","`","contains","any","``","disk","''","value",".","dtype","(","Union","[","str",",","torch.dtype","]",",","optional",",","defaults","to","None",")",":",")",":","Converts","the","model","weights","to","`","dtype","`",",","if","specified",".","Strings","get","converted","to","`","torch.dtype","`","objects","(","e.g",".","`","float16","`","-",">","`","torch.float16","`",")",".","Use","`","dtype=","''","auto","''","`","to","derive","the","type","from","the","model","\u2019","s","weights",".","peft","(","str",",","optional",",","defaults","to","None",")",":","Path","of","the","adapter","weights","to","load","from","Huggingface",".","This","will","usually","include","a","directory","that","includes","the","files","`","adapter_config.json","`","and","`","adapter_model.bin","`",".","Compatible","with","[","PEFT","]","(","https",":","\/\/github.com\/huggingface\/peft",")","load_in_8bit","(","bool",",","optional",",","defaults","to","False",")",":","If","True",",","will","convert","the","loaded","model","into","mixed-8bit","quantized","model",".","See",":","https",":","\/\/huggingface.co\/docs\/transformers\/main\/en\/main_classes\/model","#","transformers.PreTrainedModel.from_pretrained.load_in_8bit","trust_remote_code","(","bool",",","optional",",","defaults","to","False",")",":","If","True",",","will","trust","the","remote","code","when","loading","the","model",".","use_fast","(","bool",",","optional",",","defaults","to","True",")",":","If","True",",","will","use","the","fast","tokenizer","when","loading","the","model",".","gptq_use_triton","(","bool",",","optional",",","defaults","to","False",")",":","Use","Triton","for","GPTQ","inference","."]
code: def __init__(
      self,
      pretrained: str,
      quantized: Optional[Union[bool, str]] = None,
      tokenizer: Optional[str] = None,
      subfolder: Optional[str] = None,
      revision: Optional[str] = "main",
      batch_size: Optional[int] = 1,
      max_gen_toks: Optional[int] = 256,
      max_length: Optional[int] = None,
      add_special_tokens: Optional[bool] = None,
      use_accelerate: Optional[bool] = False,
      device_map_option: Optional[str] = "auto",
      max_memory_per_gpu: Optional[Union[int, str]] = None,
      max_cpu_memory: Optional[Union[int, str]] = None,
      offload_folder: Optional[str] = "./offload",
      dtype: Optional[Union[str, torch.dtype]] = None,
      device: Optional[Union[int, str]] = "cuda",
      peft: str = None,
      load_in_8bit: Optional[bool] = False,
      trust_remote_code: Optional[bool] = False,
      use_fast: Optional[bool] = True,
      gptq_use_triton: Optional[bool] = False,
  ):
      """Initializes a HuggingFace `AutoModel` and `AutoTokenizer` for evaluation.
      Args:
          pretrained (str):
              The HuggingFace Hub model ID name or the path to a pre-trained
              model to load. This is effectively the `pretrained_model_name_or_path`
              argument of `from_pretrained` in the HuggingFace `transformers` API.
          quantized (str or True, optional, defaults to None):
              File name of a GPTQ quantized model to load. Set to `True` to use the
              default name of the quantized model.
          add_special_tokens (bool, optional, defaults to True):
              Whether to add special tokens to the input sequences. If `None`, the
              default value will be set to `True` for seq2seq models (e.g. T5) and
              `False` for causal models.
              WARNING: Evaluating causal models with `add_special_tokens=True` is
              currently __not__ supported.
          > Large model loading `accelerate` arguments
          use_accelerate (bool, optional, defaults to False):
              If True, uses the `accelerate` library to load a large model across
              multiple devices.
          device_map_option (str, optional, defaults to "auto"):
              The device map option to use when loading the model with
              `accelerate`.
              Options:
                  "auto", "balanced", "balanced_low_0", "sequential"
              See the `accelerate` docs for more details on these options:
              https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.device_map
          max_memory_per_gpu (Union[int, str], optional, defaults to None):
              The maximum memory available for each GPU in bytes as `int` or in
              the format f"{significand}{unit_symbol}" where {unit_symbol} is
              any of ["GB", "MB", "GIB", "MIB"]. Refer to the `max_memory` arg in
              the "Parameters for big model inference" section of the following
              docs:
              https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.max_memory
          max_cpu_memory (Union[int, str], optional, defaults to None):
              The maximum available CPU RAM in bytes as `int` or in the format
              f"{significand}{unit_symbol}" where {unit_symbol} is any of
              ["GB", "MB", "GIB", "MIB"]. Refer to the `max_memory` arg in the
              "Parameters for big model inference" section of the following docs:
              https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.max_memory
          offload_folder (str, optional, defaults to "./offload"):
              The folder to offload weights into if `device_map` contains any
              "disk" value.
          dtype (Union[str, torch.dtype], optional, defaults to None):):
              Converts the model weights to `dtype`, if specified. Strings get
              converted to `torch.dtype` objects (e.g. `float16` -> `torch.float16`).
              Use `dtype="auto"` to derive the type from the model’s weights.
          peft (str, optional, defaults to None):
              Path of the adapter weights to load from Huggingface. This will usually
              include a directory that includes the files `adapter_config.json` and
              `adapter_model.bin`. Compatible with [PEFT](https://github.com/huggingface/peft)
          load_in_8bit (bool, optional, defaults to False):
              If True, will convert the loaded model into mixed-8bit quantized model. See:
              https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.load_in_8bit
          trust_remote_code (bool, optional, defaults to False):
              If True, will trust the remote code when loading the model.
          use_fast (bool, optional, defaults to True):
              If True, will use the fast tokenizer when loading the model.
          gptq_use_triton (bool, optional, defaults to False):
              Use Triton for GPTQ inference.
      """
      super().__init__()
  
      assert isinstance(pretrained, str)
      assert isinstance(device, str)
      assert isinstance(batch_size, int)
      if (
          add_special_tokens is not None
          and self.AUTO_MODEL_CLASS is transformers.AutoModelForCausalLM
      ):
          # TODO: Support evaluating causal models with special tokens. Currently,
          # this is not possible because the `_loglikelihood_tokens()` method for
          # causal LMs makes a no-special-tokens assumption given that contexts
          # and labels/continuations are tokenized separately without special
          # tokens, concatenated, and then processed as inputs.
          assert (
              not add_special_tokens
          ), "Evaluating causal models with `add_special_tokens=True` is currently not supported."
  
      self._batch_size = batch_size  # TODO: Adaptive batch size
      self._max_gen_toks = max_gen_toks
      self._max_length = max_length
      self._config = self.AUTO_CONFIG_CLASS.from_pretrained(
          pretrained,
          trust_remote_code=trust_remote_code,
          revision=revision
          + ("/" + subfolder if subfolder is not None else ""),
      )
  
      self._add_special_tokens = add_special_tokens
      self.tokenizer = self._create_auto_tokenizer(
          pretrained=pretrained,
          revision=revision,
          subfolder=subfolder,
          tokenizer=tokenizer,
          use_fast=use_fast,
      )
      self.tokenizer.model_max_length = self.max_length
  
      model_kwargs = {}
      if use_accelerate:
          model_kwargs = _get_accelerate_args(
              device_map_option,
              max_memory_per_gpu,
              max_cpu_memory,
              offload_folder,
          )
      model_kwargs["load_in_8bit"] = load_in_8bit
      self.model = self._create_auto_model(
          pretrained=pretrained,
          quantized=quantized,
          trust_remote_code=trust_remote_code,
          revision=revision,
          subfolder=subfolder,
          torch_dtype=_get_dtype(dtype, self._config),
          gptq_use_triton=gptq_use_triton,
          **model_kwargs,
      )
      # note: peft_path can be different than pretrained model path
      if peft is not None:
          self.model = self._create_auto_model_peft(
              model=self.model,
              peft=peft,
              revision=revision,
              subfolder=subfolder,
              torch_dtype=_get_dtype(dtype, self._config),
              **model_kwargs,
          )
      self.model.eval()
      torch.set_grad_enabled(False)
  
      self._device = device
      if use_accelerate and "lm_head" in self.model.hf_device_map:
          # `accelerate` can place `lm_head` weights on a different device than
          # the user specified one so we force `self._device` to be the same as
          # `lm_head`'s.
          self._device = self.model.hf_device_map["lm_head"]
      if not use_accelerate and not load_in_8bit:
          try:
              self.model.to(self._device)
          except:  # noqa: E722
              print(
                  "Failed to place model onto specified device. This may be because the model is quantized via `bitsandbytes`. If the desired GPU is being used, this message is safe to ignore."
              )
  
imports: import math
  import torch
  import torch.nn.functional
  import transformers
  import peft
  from pathlib import Path
  from typing import List, Mapping, NewType, Optional, Tuple, Union
  from tqdm import tqdm
  from transformers import BatchEncoding
  from lm_eval import utils
  from lm_eval.base import BaseLM
label_desc: Use image node_id 1 to create a new HuggingFaceAutoLM object from inherited base classes: BaseLM with example: obj = HuggingFaceAutoLM(pretrained, quantized, tokenizer, subfolder, revision, batch_size, max_gen_toks, max_length, add_special_tokens, use_accelerate, device_map_option, max_memory_per_gpu, max_cpu_memory, offload_folder, dtype, device, peft, load_in_8bit, trust_remote_code, use_fast, gptq_use_triton)