End of training

README.md

@@ -0,0 +1,122 @@
+---
+library_name: peft
+license: other
+base_model: deepseek-ai/deepseek-coder-1.3b-base
+tags:
+- generated_from_trainer
+model-index:
+- name: lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# lemexp-task1-v2-lemma_object_small-deepseek-coder-1.3b-base-ddp-8lr-v2
+
+This model is a fine-tuned version of [deepseek-ai/deepseek-coder-1.3b-base](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2436
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0008
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 8
+- total_train_batch_size: 16
+- total_eval_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 12
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch   | Step  | Validation Loss |
+|:-------------:|:-------:|:-----:|:---------------:|
+| 0.6234        | 0.2002  | 721   | 0.4560          |
+| 0.4715        | 0.4003  | 1442  | 0.4156          |
+| 0.4128        | 0.6005  | 2163  | 0.3940          |
+| 0.4006        | 0.8007  | 2884  | 0.3762          |
+| 0.3801        | 1.0008  | 3605  | 0.3659          |
+| 0.3524        | 1.2010  | 4326  | 0.3581          |
+| 0.3464        | 1.4012  | 5047  | 0.3553          |
+| 0.3381        | 1.6013  | 5768  | 0.3448          |
+| 0.3394        | 1.8015  | 6489  | 0.3364          |
+| 0.3344        | 2.0017  | 7210  | 0.3235          |
+| 0.3158        | 2.2018  | 7931  | 0.3265          |
+| 0.3101        | 2.4020  | 8652  | 0.3250          |
+| 0.3068        | 2.6022  | 9373  | 0.3124          |
+| 0.3043        | 2.8023  | 10094 | 0.3124          |
+| 0.3069        | 3.0025  | 10815 | 0.3118          |
+| 0.281         | 3.2027  | 11536 | 0.3095          |
+| 0.2813        | 3.4028  | 12257 | 0.3043          |
+| 0.2782        | 3.6030  | 12978 | 0.2970          |
+| 0.2806        | 3.8032  | 13699 | 0.2969          |
+| 0.2779        | 4.0033  | 14420 | 0.3016          |
+| 0.2563        | 4.2035  | 15141 | 0.2922          |
+| 0.2569        | 4.4037  | 15862 | 0.2853          |
+| 0.2574        | 4.6038  | 16583 | 0.2868          |
+| 0.2571        | 4.8040  | 17304 | 0.2862          |
+| 0.2587        | 5.0042  | 18025 | 0.2851          |
+| 0.2332        | 5.2043  | 18746 | 0.2777          |
+| 0.235         | 5.4045  | 19467 | 0.2706          |
+| 0.2341        | 5.6047  | 20188 | 0.2777          |
+| 0.2369        | 5.8048  | 20909 | 0.2760          |
+| 0.2376        | 6.0050  | 21630 | 0.2682          |
+| 0.2166        | 6.2052  | 22351 | 0.2712          |
+| 0.2159        | 6.4053  | 23072 | 0.2671          |
+| 0.216         | 6.6055  | 23793 | 0.2641          |
+| 0.2168        | 6.8057  | 24514 | 0.2650          |
+| 0.2152        | 7.0058  | 25235 | 0.2621          |
+| 0.2006        | 7.2060  | 25956 | 0.2604          |
+| 0.1981        | 7.4062  | 26677 | 0.2534          |
+| 0.1975        | 7.6063  | 27398 | 0.2504          |
+| 0.196         | 7.8065  | 28119 | 0.2552          |
+| 0.195         | 8.0067  | 28840 | 0.2576          |
+| 0.1744        | 8.2068  | 29561 | 0.2524          |
+| 0.1769        | 8.4070  | 30282 | 0.2472          |
+| 0.1763        | 8.6072  | 31003 | 0.2470          |
+| 0.1761        | 8.8073  | 31724 | 0.2454          |
+| 0.1749        | 9.0075  | 32445 | 0.2493          |
+| 0.1534        | 9.2077  | 33166 | 0.2450          |
+| 0.1552        | 9.4078  | 33887 | 0.2438          |
+| 0.1557        | 9.6080  | 34608 | 0.2457          |
+| 0.1576        | 9.8082  | 35329 | 0.2439          |
+| 0.1555        | 10.0083 | 36050 | 0.2442          |
+| 0.1361        | 10.2085 | 36771 | 0.2490          |
+| 0.1355        | 10.4087 | 37492 | 0.2447          |
+| 0.1382        | 10.6088 | 38213 | 0.2399          |
+| 0.1379        | 10.8090 | 38934 | 0.2413          |
+| 0.1361        | 11.0092 | 39655 | 0.2444          |
+| 0.1242        | 11.2093 | 40376 | 0.2459          |
+| 0.1206        | 11.4095 | 41097 | 0.2451          |
+| 0.1225        | 11.6097 | 41818 | 0.2465          |
+| 0.122         | 11.8098 | 42539 | 0.2436          |
+
+
+### Framework versions
+
+- PEFT 0.14.0
+- Transformers 4.47.0
+- Pytorch 2.5.1+cu124
+- Datasets 3.2.0
+- Tokenizers 0.21.0

adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "deepseek-ai/deepseek-coder-1.3b-base",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:210ec037bc571589e0d9f9f574baf351c38f313ebef10622996ddd4a564debfc
+size 268661312

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1b17d78865006b9eaad993512a32071c63d7db749880c40c74a04aed8b48d0
+size 5496