Hugging Face's logo

yalhessi
/
lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2

PEFT
Safetensors
Generated from Trainer
Model card Files Community
lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2 / README.md
yalhessi's picture
yalhessi
End of training
1a41b79 verified
preview code
|
raw
history blame
4.84 kB
metadata
library_name: peft
license: other
base_model: deepseek-ai/deepseek-coder-1.3b-base
tags:
  - generated_from_trainer
model-index:
  - name: lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2
    results: []

lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2436

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0008
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 12
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.6234 0.2002 721 0.4560
0.4715 0.4003 1442 0.4156
0.4128 0.6005 2163 0.3940
0.4006 0.8007 2884 0.3762
0.3801 1.0008 3605 0.3659
0.3524 1.2010 4326 0.3581
0.3464 1.4012 5047 0.3553
0.3381 1.6013 5768 0.3448
0.3394 1.8015 6489 0.3364
0.3344 2.0017 7210 0.3235
0.3158 2.2018 7931 0.3265
0.3101 2.4020 8652 0.3250
0.3068 2.6022 9373 0.3124
0.3043 2.8023 10094 0.3124
0.3069 3.0025 10815 0.3118
0.281 3.2027 11536 0.3095
0.2813 3.4028 12257 0.3043
0.2782 3.6030 12978 0.2970
0.2806 3.8032 13699 0.2969
0.2779 4.0033 14420 0.3016
0.2563 4.2035 15141 0.2922
0.2569 4.4037 15862 0.2853
0.2574 4.6038 16583 0.2868
0.2571 4.8040 17304 0.2862
0.2587 5.0042 18025 0.2851
0.2332 5.2043 18746 0.2777
0.235 5.4045 19467 0.2706
0.2341 5.6047 20188 0.2777
0.2369 5.8048 20909 0.2760
0.2376 6.0050 21630 0.2682
0.2166 6.2052 22351 0.2712
0.2159 6.4053 23072 0.2671
0.216 6.6055 23793 0.2641
0.2168 6.8057 24514 0.2650
0.2152 7.0058 25235 0.2621
0.2006 7.2060 25956 0.2604
0.1981 7.4062 26677 0.2534
0.1975 7.6063 27398 0.2504
0.196 7.8065 28119 0.2552
0.195 8.0067 28840 0.2576
0.1744 8.2068 29561 0.2524
0.1769 8.4070 30282 0.2472
0.1763 8.6072 31003 0.2470
0.1761 8.8073 31724 0.2454
0.1749 9.0075 32445 0.2493
0.1534 9.2077 33166 0.2450
0.1552 9.4078 33887 0.2438
0.1557 9.6080 34608 0.2457
0.1576 9.8082 35329 0.2439
0.1555 10.0083 36050 0.2442
0.1361 10.2085 36771 0.2490
0.1355 10.4087 37492 0.2447
0.1382 10.6088 38213 0.2399
0.1379 10.8090 38934 0.2413
0.1361 11.0092 39655 0.2444
0.1242 11.2093 40376 0.2459
0.1206 11.4095 41097 0.2451
0.1225 11.6097 41818 0.2465
0.122 11.8098 42539 0.2436

Framework versions

  • PEFT 0.14.0
  • Transformers 4.47.0
  • Pytorch 2.5.1+cu124
  • Datasets 3.2.0
  • Tokenizers 0.21.0