metadata

tags:
  - generated_from_trainer
model-index:
  - name: codebert-gpt2-commitgen
    results: []

codebert-gpt2-commitgen

This model is a fine-tuned version on dataset provided in the paper titled "Towards Automatic Generation of Short Summaries of Commits" by Siyuan Jiang and Collin McMillan. Heres is the link to the paper https://arxiv.org/abs/1703.09603

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 2000
num_epochs: 3

Training results

global_step=4521
training_loss=3.55994465065804
train_runtime: 3300.0492
train_samples_per_second: 21.919
train_steps_per_second: 1.37
total_flos: 1.062667587499776e+16
train_loss: 3.55994465065804

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Tokenizers 0.13.2