Hugging Face's logo

yalhessi
/
lemexp-task1-v2-lemma_object_small_nodefs-deepseek-coder-1.3b-base-ddp-8lr-v2

PEFT
Safetensors
Generated from Trainer
Model card Files Community
lemexp-task1-v2-lemma_object_small_nodefs-deepseek-coder-1.3b-base-ddp-8lr-v2 / README.md
yalhessi's picture
yalhessi
End of training
0e3fa05 verified
preview code
|
raw
history blame contribute delete
4.85 kB
metadata
library_name: peft
license: other
base_model: deepseek-ai/deepseek-coder-1.3b-base
tags:
  - generated_from_trainer
model-index:
  - name: >-
      lemexp-task1-v2-lemma_object_small_nodefs-deepseek-coder-1.3b-base-ddp-8lr-v2
    results: []

lemexp-task1-v2-lemma_object_small_nodefs-deepseek-coder-1.3b-base-ddp-8lr-v2

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2645

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0008
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 12
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.6349 0.2001 720 0.5111
0.5017 0.4001 1440 0.4620
0.4349 0.6002 2160 0.4331
0.417 0.8002 2880 0.4095
0.3957 1.0003 3600 0.4012
0.3607 1.2003 4320 0.3890
0.3585 1.4004 5040 0.3751
0.3561 1.6004 5760 0.3749
0.3505 1.8005 6480 0.3672
0.3438 2.0006 7200 0.3510
0.323 2.2006 7920 0.3524
0.3172 2.4007 8640 0.3435
0.3133 2.6007 9360 0.3368
0.3125 2.8008 10080 0.3331
0.312 3.0008 10800 0.3309
0.2852 3.2009 11520 0.3303
0.2916 3.4009 12240 0.3187
0.2876 3.6010 12960 0.3235
0.2868 3.8011 13680 0.3209
0.287 4.0011 14400 0.3150
0.2582 4.2012 15120 0.3147
0.2678 4.4012 15840 0.3117
0.2625 4.6013 16560 0.3082
0.2683 4.8013 17280 0.3010
0.2597 5.0014 18000 0.3030
0.2402 5.2014 18720 0.3017
0.2405 5.4015 19440 0.2975
0.2441 5.6016 20160 0.2950
0.2443 5.8016 20880 0.2945
0.2383 6.0017 21600 0.2912
0.2208 6.2017 22320 0.2862
0.2233 6.4018 23040 0.2856
0.2222 6.6018 23760 0.2818
0.2208 6.8019 24480 0.2808
0.2214 7.0019 25200 0.2786
0.2039 7.2020 25920 0.2844
0.1996 7.4021 26640 0.2802
0.1996 7.6021 27360 0.2814
0.2003 7.8022 28080 0.2721
0.2047 8.0022 28800 0.2717
0.1801 8.2023 29520 0.2722
0.1771 8.4023 30240 0.2736
0.1814 8.6024 30960 0.2716
0.1781 8.8024 31680 0.2663
0.1812 9.0025 32400 0.2668
0.1584 9.2026 33120 0.2684
0.1578 9.4026 33840 0.2635
0.1627 9.6027 34560 0.2666
0.1612 9.8027 35280 0.2601
0.1576 10.0028 36000 0.2623
0.14 10.2028 36720 0.2660
0.1403 10.4029 37440 0.2619
0.1407 10.6029 38160 0.2592
0.1399 10.8030 38880 0.2594
0.1424 11.0031 39600 0.2599
0.1275 11.2031 40320 0.2703
0.1253 11.4032 41040 0.2662
0.1242 11.6032 41760 0.2641
0.1234 11.8033 42480 0.2645

Framework versions

  • PEFT 0.14.0
  • Transformers 4.47.0
  • Pytorch 2.5.1+cu124
  • Datasets 3.2.0
  • Tokenizers 0.21.0