Hugging Face's logo

andrianangg
/
result

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
andrianangg commited on
Commit
0f6a03d
·
verified ·
1 Parent(s): 577667f

komodo-ner

Browse files
Files changed (30) hide show
  1. README.md +80 -71
  2. adapter_config.json +20 -31
  3. adapter_model.safetensors +2 -2
  4. runs/Aug27_23-58-55_DESKTOP-9PQ7MIK/events.out.tfevents.1724777946.DESKTOP-9PQ7MIK.4436.0 +3 -0
  5. runs/Aug27_23-58-55_DESKTOP-9PQ7MIK/events.out.tfevents.1724777971.DESKTOP-9PQ7MIK.4436.1 +3 -0
  6. runs/Aug28_00-46-12_DESKTOP-9PQ7MIK/events.out.tfevents.1724780775.DESKTOP-9PQ7MIK.12588.0 +3 -0
  7. runs/Aug28_00-47-17_DESKTOP-9PQ7MIK/events.out.tfevents.1724780841.DESKTOP-9PQ7MIK.12588.1 +3 -0
  8. runs/Aug28_00-48-08_DESKTOP-9PQ7MIK/events.out.tfevents.1724780890.DESKTOP-9PQ7MIK.12588.2 +3 -0
  9. runs/Aug28_00-50-43_DESKTOP-9PQ7MIK/events.out.tfevents.1724781045.DESKTOP-9PQ7MIK.12588.3 +3 -0
  10. runs/Aug28_00-51-52_DESKTOP-9PQ7MIK/events.out.tfevents.1724781119.DESKTOP-9PQ7MIK.12588.4 +3 -0
  11. runs/Aug28_00-52-47_DESKTOP-9PQ7MIK/events.out.tfevents.1724781169.DESKTOP-9PQ7MIK.12588.5 +3 -0
  12. runs/Aug28_01-00-20_DESKTOP-9PQ7MIK/events.out.tfevents.1724781633.DESKTOP-9PQ7MIK.10164.0 +3 -0
  13. runs/Aug28_01-01-39_DESKTOP-9PQ7MIK/events.out.tfevents.1724781702.DESKTOP-9PQ7MIK.10164.1 +3 -0
  14. runs/Aug28_01-01-39_DESKTOP-9PQ7MIK/events.out.tfevents.1724781714.DESKTOP-9PQ7MIK.10164.2 +3 -0
  15. runs/Aug28_01-05-05_DESKTOP-9PQ7MIK/events.out.tfevents.1724781911.DESKTOP-9PQ7MIK.10164.3 +3 -0
  16. runs/Aug28_01-05-50_DESKTOP-9PQ7MIK/events.out.tfevents.1724781953.DESKTOP-9PQ7MIK.10164.4 +3 -0
  17. runs/Aug28_01-11-59_DESKTOP-9PQ7MIK/events.out.tfevents.1724782360.DESKTOP-9PQ7MIK.14092.0 +3 -0
  18. runs/Aug28_01-13-28_DESKTOP-9PQ7MIK/events.out.tfevents.1724782410.DESKTOP-9PQ7MIK.14092.1 +3 -0
  19. runs/Aug28_01-14-15_DESKTOP-9PQ7MIK/events.out.tfevents.1724782457.DESKTOP-9PQ7MIK.14092.2 +3 -0
  20. runs/Aug28_01-14-44_DESKTOP-9PQ7MIK/events.out.tfevents.1724782486.DESKTOP-9PQ7MIK.14092.3 +3 -0
  21. runs/Aug28_01-16-24_DESKTOP-9PQ7MIK/events.out.tfevents.1724782588.DESKTOP-9PQ7MIK.10524.0 +3 -0
  22. runs/Aug28_01-33-29_DESKTOP-9PQ7MIK/events.out.tfevents.1724783612.DESKTOP-9PQ7MIK.13692.0 +3 -0
  23. runs/Aug28_01-35-51_DESKTOP-9PQ7MIK/events.out.tfevents.1724783759.DESKTOP-9PQ7MIK.10680.0 +3 -0
  24. runs/Aug28_01-49-00_DESKTOP-9PQ7MIK/events.out.tfevents.1724784602.DESKTOP-9PQ7MIK.10852.0 +3 -0
  25. runs/Aug28_01-53-30_DESKTOP-9PQ7MIK/events.out.tfevents.1724784832.DESKTOP-9PQ7MIK.2812.0 +3 -0
  26. runs/Aug28_01-55-22_DESKTOP-9PQ7MIK/events.out.tfevents.1724784932.DESKTOP-9PQ7MIK.6120.0 +3 -0
  27. special_tokens_map.json +24 -30
  28. tokenizer.json +0 -0
  29. tokenizer_config.json +0 -0
  30. training_args.bin +2 -2
README.md CHANGED
@@ -1,71 +1,80 @@
1
- ---
2
- base_model: NousResearch/Llama-2-7b-hf
3
- datasets:
4
- - conll2003
5
- library_name: peft
6
- metrics:
7
- - precision
8
- - recall
9
- - f1
10
- - accuracy
11
- tags:
12
- - generated_from_trainer
13
- model-index:
14
- - name: result
15
- results: []
16
- ---
17
-
18
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
19
- should probably proofread and complete it, then remove this comment. -->
20
-
21
- # result
22
-
23
- This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on the conll2003 dataset.
24
- It achieves the following results on the evaluation set:
25
- - Loss: nan
26
- - Precision: 0.0
27
- - Recall: 0.0
28
- - F1: 0.0
29
- - Accuracy: 0.8274
30
-
31
- ## Model description
32
-
33
- More information needed
34
-
35
- ## Intended uses & limitations
36
-
37
- More information needed
38
-
39
- ## Training and evaluation data
40
-
41
- More information needed
42
-
43
- ## Training procedure
44
-
45
- ### Training hyperparameters
46
-
47
- The following hyperparameters were used during training:
48
- - learning_rate: 2e-05
49
- - train_batch_size: 8
50
- - eval_batch_size: 8
51
- - seed: 42
52
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
53
- - lr_scheduler_type: linear
54
- - num_epochs: 3
55
-
56
- ### Training results
57
-
58
- | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
59
- |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:---:|:--------:|
60
- | 0.0 | 1.0 | 1756 | nan | 0.0 | 0.0 | 0.0 | 0.8339 |
61
- | 0.0 | 2.0 | 3512 | nan | 0.0 | 0.0 | 0.0 | 0.8339 |
62
- | 0.0 | 3.0 | 5268 | nan | 0.0 | 0.0 | 0.0 | 0.8339 |
63
-
64
-
65
- ### Framework versions
66
-
67
- - PEFT 0.12.0
68
- - Transformers 4.42.4
69
- - Pytorch 2.3.1+cu121
70
- - Datasets 2.20.0
71
- - Tokenizers 0.19.1
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ license: llama2
4
+ base_model: Yellow-AI-NLP/komodo-7b-base
5
+ tags:
6
+ - trl
7
+ - sft
8
+ - generated_from_trainer
9
+ datasets:
10
+ - id_nergrit_corpus
11
+ model-index:
12
+ - name: result
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # result
20
+
21
+ This model is a fine-tuned version of [Yellow-AI-NLP/komodo-7b-base](https://huggingface.co/Yellow-AI-NLP/komodo-7b-base) on the id_nergrit_corpus dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - eval_loss: 0.0598
24
+ - eval_runtime: 429.4851
25
+ - eval_samples_per_second: 1.469
26
+ - eval_steps_per_second: 0.368
27
+ - epoch: 0.8265
28
+ - step: 162
29
+
30
+ ## Model description
31
+
32
+ More information needed
33
+
34
+ ## Intended uses & limitations
35
+
36
+ More information needed
37
+
38
+ ## Training and evaluation data
39
+
40
+ More information needed
41
+
42
+ ## Training procedure
43
+
44
+
45
+ The following `bitsandbytes` quantization config was used during training:
46
+ - quant_method: bitsandbytes
47
+ - _load_in_8bit: False
48
+ - _load_in_4bit: True
49
+ - llm_int8_threshold: 6.0
50
+ - llm_int8_skip_modules: None
51
+ - llm_int8_enable_fp32_cpu_offload: False
52
+ - llm_int8_has_fp16_weight: False
53
+ - bnb_4bit_quant_type: nf4
54
+ - bnb_4bit_use_double_quant: True
55
+ - bnb_4bit_compute_dtype: float16
56
+ - bnb_4bit_quant_storage: uint8
57
+ - load_in_4bit: True
58
+ - load_in_8bit: False
59
+ ### Training hyperparameters
60
+
61
+ The following hyperparameters were used during training:
62
+ - learning_rate: 0.0001
63
+ - train_batch_size: 4
64
+ - eval_batch_size: 4
65
+ - seed: 42
66
+ - gradient_accumulation_steps: 4
67
+ - total_train_batch_size: 16
68
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
69
+ - lr_scheduler_type: cosine
70
+ - lr_scheduler_warmup_ratio: 0.05
71
+ - num_epochs: 2
72
+ - mixed_precision_training: Native AMP
73
+
74
+ ### Framework versions
75
+
76
+ - PEFT 0.4.0
77
+ - Transformers 4.44.1
78
+ - Pytorch 2.4.0+cu118
79
+ - Datasets 2.21.0
80
+ - Tokenizers 0.19.1
adapter_config.json CHANGED
@@ -1,32 +1,21 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "NousResearch/Llama-2-7b-hf",
5
- "bias": "none",
6
- "fan_in_fan_out": false,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layer_replication": null,
10
- "layers_pattern": null,
11
- "layers_to_transform": null,
12
- "loftq_config": {},
13
- "lora_alpha": 8,
14
- "lora_dropout": 0.1,
15
- "megatron_config": null,
16
- "megatron_core": "megatron.core",
17
- "modules_to_save": [
18
- "classifier",
19
- "score"
20
- ],
21
- "peft_type": "LORA",
22
- "r": 8,
23
- "rank_pattern": {},
24
- "revision": null,
25
- "target_modules": [
26
- "q_proj",
27
- "v_proj"
28
- ],
29
- "task_type": "TOKEN_CLS",
30
- "use_dora": false,
31
- "use_rslora": false
32
  }
 
1
+ {
2
+ "auto_mapping": null,
3
+ "base_model_name_or_path": "Yellow-AI-NLP/komodo-7b-base",
4
+ "bias": "none",
5
+ "fan_in_fan_out": false,
6
+ "inference_mode": true,
7
+ "init_lora_weights": true,
8
+ "layers_pattern": null,
9
+ "layers_to_transform": null,
10
+ "lora_alpha": 16,
11
+ "lora_dropout": 0.1,
12
+ "modules_to_save": null,
13
+ "peft_type": "LORA",
14
+ "r": 4,
15
+ "revision": null,
16
+ "target_modules": [
17
+ "q_proj",
18
+ "v_proj"
19
+ ],
20
+ "task_type": "CAUSAL_LM"
 
 
 
 
 
 
 
 
 
 
 
21
  }
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e46314a1713e92bc43fa5cab97dbbbb5d2b44c649af796bcdca9cbb3abec3f11
3
- size 16868138
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea25d0194248040d22deb90082710b78571ab0a93c747caf35dcd9efbfd8bced
3
+ size 8405472
runs/Aug27_23-58-55_DESKTOP-9PQ7MIK/events.out.tfevents.1724777946.DESKTOP-9PQ7MIK.4436.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8676dd898a7db25b8ac433612f0d331a495868ae4b0e4b61f166b58b1a3cae8
3
+ size 5735
runs/Aug27_23-58-55_DESKTOP-9PQ7MIK/events.out.tfevents.1724777971.DESKTOP-9PQ7MIK.4436.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95e0bc2be9002cb60ad706e281446403301ef59208d79abd9b9d8cbd7a008b3c
3
+ size 5735
runs/Aug28_00-46-12_DESKTOP-9PQ7MIK/events.out.tfevents.1724780775.DESKTOP-9PQ7MIK.12588.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a27035d21023e45c49ad6836aa9785861f0e024fd5807a9dacd3bffc7a645e3
3
+ size 5735
runs/Aug28_00-47-17_DESKTOP-9PQ7MIK/events.out.tfevents.1724780841.DESKTOP-9PQ7MIK.12588.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c69fad96937e6f6e25e82c072062857b3917b7566796f5d8262f7f7306d0a5b5
3
+ size 5735
runs/Aug28_00-48-08_DESKTOP-9PQ7MIK/events.out.tfevents.1724780890.DESKTOP-9PQ7MIK.12588.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83424caa84dae95450367d1c71d3b4dbafcba1109297692f300851c269693565
3
+ size 5735
runs/Aug28_00-50-43_DESKTOP-9PQ7MIK/events.out.tfevents.1724781045.DESKTOP-9PQ7MIK.12588.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:186cb1b9dc6e57710c4a62bd104e8d61bde21490fad86aba2dc958a94975ceb8
3
+ size 5735
runs/Aug28_00-51-52_DESKTOP-9PQ7MIK/events.out.tfevents.1724781119.DESKTOP-9PQ7MIK.12588.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b22c0d33fa46112e71bec6ccacb829906e0b3e797886014a0141e710ee29b06
3
+ size 5735
runs/Aug28_00-52-47_DESKTOP-9PQ7MIK/events.out.tfevents.1724781169.DESKTOP-9PQ7MIK.12588.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80e7eb74c9306939bcc70d1336ee6efc2a3ff07de6bad24f44eb2cdcc928a8bb
3
+ size 5735
runs/Aug28_01-00-20_DESKTOP-9PQ7MIK/events.out.tfevents.1724781633.DESKTOP-9PQ7MIK.10164.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7998918598a5230ef2e12b4d706b8551eec87dc3856046c1f160477ec89682d0
3
+ size 5735
runs/Aug28_01-01-39_DESKTOP-9PQ7MIK/events.out.tfevents.1724781702.DESKTOP-9PQ7MIK.10164.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b8bb08240d9629bf2b213befb384b77b3e8e5e55eacaa1e5d5429132fef0b33
3
+ size 5735
runs/Aug28_01-01-39_DESKTOP-9PQ7MIK/events.out.tfevents.1724781714.DESKTOP-9PQ7MIK.10164.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d6b112a15dfd9e8dbae4274aaf2905ddf108a5613dac321e1f0a2753771e57f
3
+ size 5735
runs/Aug28_01-05-05_DESKTOP-9PQ7MIK/events.out.tfevents.1724781911.DESKTOP-9PQ7MIK.10164.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29884167c3397ec9626ec322000f1ae6313cf4964d31241eab5d5ecd630f088a
3
+ size 5735
runs/Aug28_01-05-50_DESKTOP-9PQ7MIK/events.out.tfevents.1724781953.DESKTOP-9PQ7MIK.10164.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b56574956b2f678b09cb51c147bb9cdec9e22a93761f40f368bedb54f6d355cc
3
+ size 5735
runs/Aug28_01-11-59_DESKTOP-9PQ7MIK/events.out.tfevents.1724782360.DESKTOP-9PQ7MIK.14092.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c5192fa44fc42faf9e72091411ae852460db6d0c079be77694e1dd0d06aefd6
3
+ size 5735
runs/Aug28_01-13-28_DESKTOP-9PQ7MIK/events.out.tfevents.1724782410.DESKTOP-9PQ7MIK.14092.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0920397be21f85ef918e0c42b4e0d50097939adbd2138a9eb27f515c935f2ae
3
+ size 5735
runs/Aug28_01-14-15_DESKTOP-9PQ7MIK/events.out.tfevents.1724782457.DESKTOP-9PQ7MIK.14092.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9361886a41314a3a34ee83aba83b6defee48a316d293febf87103a9b4ba9c0ad
3
+ size 5735
runs/Aug28_01-14-44_DESKTOP-9PQ7MIK/events.out.tfevents.1724782486.DESKTOP-9PQ7MIK.14092.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3238e91cb5bcab2f7a07cdc3989982091bae58109255dd4bfde4b365470dbc0
3
+ size 5735
runs/Aug28_01-16-24_DESKTOP-9PQ7MIK/events.out.tfevents.1724782588.DESKTOP-9PQ7MIK.10524.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed2d944b1beb4b42efbea3179fb08f9cc91c1b9bf52e22c7991024a3a16b676a
3
+ size 6149
runs/Aug28_01-33-29_DESKTOP-9PQ7MIK/events.out.tfevents.1724783612.DESKTOP-9PQ7MIK.13692.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c03d8e5702e4fd4fa40fc7cc5e06975279826ac69f422d610abc39bc25b86f0
3
+ size 5735
runs/Aug28_01-35-51_DESKTOP-9PQ7MIK/events.out.tfevents.1724783759.DESKTOP-9PQ7MIK.10680.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60979a2c49b5a96a1e19a60a0418bb30e9bd4feb8784dbbfa1f3a85bb73be558
3
+ size 5735
runs/Aug28_01-49-00_DESKTOP-9PQ7MIK/events.out.tfevents.1724784602.DESKTOP-9PQ7MIK.10852.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f267dfdb18cdca122724f87dc0ee453e111edc69e3bf8479d25a9ff86e14dca6
3
+ size 5734
runs/Aug28_01-53-30_DESKTOP-9PQ7MIK/events.out.tfevents.1724784832.DESKTOP-9PQ7MIK.2812.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c343c91c2ffd2e9fb056b7dd114c716543791ca998bec2aa7166f3482b6d955b
3
+ size 5734
runs/Aug28_01-55-22_DESKTOP-9PQ7MIK/events.out.tfevents.1724784932.DESKTOP-9PQ7MIK.6120.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5dd5e368a3ac08b70d131b42bd8cf647532bd053486eddae4f0d7c5d6541f09
3
+ size 40216
special_tokens_map.json CHANGED
@@ -1,30 +1,24 @@
1
- {
2
- "bos_token": {
3
- "content": "<s>",
4
- "lstrip": false,
5
- "normalized": true,
6
- "rstrip": false,
7
- "single_word": false
8
- },
9
- "eos_token": {
10
- "content": "</s>",
11
- "lstrip": false,
12
- "normalized": true,
13
- "rstrip": false,
14
- "single_word": false
15
- },
16
- "pad_token": {
17
- "content": "<unk>",
18
- "lstrip": false,
19
- "normalized": true,
20
- "rstrip": false,
21
- "single_word": false
22
- },
23
- "unk_token": {
24
- "content": "<unk>",
25
- "lstrip": false,
26
- "normalized": true,
27
- "rstrip": false,
28
- "single_word": false
29
- }
30
- }
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": "</s>",
17
+ "unk_token": {
18
+ "content": "<unk>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ }
24
+ }
 
 
 
 
 
 
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ceb61393bfd1f690a71a05b202ce34e6eec4297f81d4d53cacf9ee0dd1632d9d
3
- size 5112
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5128a93f603947f91ff8c4ac900dad4390f1cde72523941f9634aecd3b74b45d
3
+ size 5432