chunk_content: e the padding token to use. If you’re using a GPT, OPT, or BLOOM model, you should set the padding_side to the left; otherwise it’ll be set to the right. Tokenize the sentence pairs and truncate them to the maximum length.
  
  
  	Copied
  if any(k in model_name_or_path for k in ("gpt", "opt", "bloom")):
      padding_side = "left"
  else:
      padding_side = "right"
  
  tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, padding_side=padding_side)
  