How to fine-tune gpt-j using Huggingface Trainer
I'm attempting to fine-tune gpt-j using the huggingface trainer and failing miserably. I followed the example that references bert, but of course, the gpt-j model isn't exactly like the bert model.
The error indicates that the model isn't producing a loss, which is great, except that I have no idea how to make it generate a loss or how to change what the trainer is expecting.
I'm using Transformers 4.22.2. I would like to get this working on a CPU before I try to do anything on Paperspace with a GPU. I did make an initial attempt there using a GPU that received the same error, with slightly different code to use cuda.
I suspect that my approach is entirely wrong. I found a very old example of fine-tuning gpt-j using 8-bit quantization, but even that repository says it is deprecated.
I'm unsure if my mistake is in using the compute_metrics() I found in the bert example or if it is something else. Any advice would be appreciated. Or, maybe it is an issue with the labels I provide the config, but I've tried different permutations.
I understand what a loss function is, but I don't know how it is supposed to be configured in this case.
My Code:
from transformers import Trainer, TrainingArguments, AutoModelForCausalLM
from transformers import GPTJForCausalLM, AutoTokenizer
from datasets import load_dataset
import time
import torch
import os
import numpy as np
import evaluate
import sklearn

start = time.time()

GPTJ_FINE_TUNED_FILE = &quot;./fine_tuned_models/gpt-j-6B&quot;

print(&quot;Loading model&quot;)
model = GPTJForCausalLM.from_pretrained(&quot;EleutherAI/gpt-j-6B&quot;, low_cpu_mem_usage=True)
model.config.pad_token_id = model.config.eos_token_id

print(&quot;Loading tokenizer&quot;)
tokenizer = AutoTokenizer.from_pretrained(&quot;EleutherAI/gpt-j-6B&quot;)
tokenizer.pad_token = tokenizer.eos_token

print(&quot;Loading dataset&quot;)
current_dataset = load_dataset(&quot;wikitext&quot;, 'wikitext-103-v1')
current_dataset['train'] = current_dataset['train'].select(range(1200))


def tokenize_function(examples):
    current_tokenizer_result = tokenizer(examples[&quot;text&quot;], padding=&quot;max_length&quot;, truncation=True)
    return current_tokenizer_result


print(&quot;Splitting and tokenizing dataset&quot;)
tokenized_datasets = current_dataset.map(tokenize_function, batched=True)
small_train_dataset = tokenized_datasets[&quot;train&quot;].select(range(100))

print(&quot;Preparing training arguments&quot;)

training_args = TrainingArguments(output_dir=GPTJ_FINE_TUNED_FILE,
                                  report_to='all',
                                  logging_dir='./logs',
                                  per_device_train_batch_size=1,
                                  label_names=['input_ids', 'attention_mask'],  # 'logits', 'past_key_values'
                                  num_train_epochs=1,
                                  no_cuda=True
                                  )

metric = evaluate.load(&quot;accuracy&quot;)


def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)


trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=small_train_dataset
)

print(&quot;Starting training&quot;)
trainer.train()
print(f&quot;Finished fine-tuning in {time.time() - start}&quot;)

Which leads to the error and stacktrace:
  File &quot;xxx\ft_v3.py&quot;, line 66, in &lt;module&gt;
  File &quot;xxx\venv\lib\site-packages\transformers\trainer.py&quot;, line 1521, in train
    return inner_training_loop(
  File &quot;xxx\venv\lib\site-packages\transformers\trainer.py&quot;, line 1763, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File &quot;xxx\venv\lib\site-packages\transformers\trainer.py&quot;, line 2499, in training_step
    loss = self.compute_loss(model, inputs)
  File &quot;xxx\venv\lib\site-packages\transformers\trainer.py&quot;, line 2544, in compute_loss
    raise ValueError(
ValueError: The model did not return a loss from the inputs, only the following keys: logits,past_key_values. For reference, the inputs it received are input_ids,attention_mask.


I found what appears to work, though now I'm running low on memory and working through ways of handling it.
The data_collator parameter seems to take care of the exact issue that I was having.
data_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=small_train_dataset,
    eval_dataset=small_eval_dataset,
    compute_metrics=compute_metrics,
    data_collator=data_collator,
)

