File size: 16,157 Bytes
4f6582a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 |
---
base_model: bigcode/starencoder
tags:
- generated_from_trainer
metrics:
- precision
- recall
- accuracy
model-index:
- name: classifier-llama3-swift-500k
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# classifier-llama3-swift-500k
This model is a fine-tuned version of [bigcode/starencoder](https://huggingface.co/bigcode/starencoder) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3403
- Precision: 0.6596
- Recall: 0.3962
- F1 Macro: 0.4436
- Accuracy: 0.6112
- F1 Binary Minimum3: 0.7282
- F1 Binary Minimum2: 0.9673
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 256
- seed: 0
- distributed_type: multi-GPU
- num_devices: 8
- total_train_batch_size: 128
- total_eval_batch_size: 2048
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 30
### Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 Macro | Accuracy | F1 Binary Minimum3 | F1 Binary Minimum2 |
|:-------------:|:-------:|:------:|:---------------:|:---------:|:------:|:--------:|:--------:|:------------------:|:------------------:|
| No log | 0 | 0 | 7.9458 | 0.0159 | 0.2 | 0.0294 | 0.0794 | 0 | 0 |
| 0.3909 | 0.2932 | 1000 | 0.3869 | 0.6080 | 0.3401 | 0.3645 | 0.5781 | 0.7198 | 0.9645 |
| 0.3743 | 0.5863 | 2000 | 0.3686 | 0.6224 | 0.3523 | 0.3809 | 0.5910 | 0.7154 | 0.9657 |
| 0.3639 | 0.8795 | 3000 | 0.3666 | 0.6145 | 0.3529 | 0.3809 | 0.5920 | 0.7273 | 0.9657 |
| 0.3655 | 1.1727 | 4000 | 0.3594 | 0.6360 | 0.3623 | 0.3946 | 0.5962 | 0.7145 | 0.9661 |
| 0.3702 | 1.4658 | 5000 | 0.3585 | 0.6124 | 0.3566 | 0.3856 | 0.5960 | 0.7031 | 0.9663 |
| 0.3625 | 1.7590 | 6000 | 0.3558 | 0.6531 | 0.3717 | 0.4082 | 0.5999 | 0.7191 | 0.9665 |
| 0.3474 | 2.0522 | 7000 | 0.3542 | 0.6420 | 0.3696 | 0.4033 | 0.6004 | 0.7173 | 0.9666 |
| 0.3711 | 2.3454 | 8000 | 0.3536 | 0.6375 | 0.3618 | 0.3944 | 0.5999 | 0.7107 | 0.9665 |
| 0.3495 | 2.6385 | 9000 | 0.3522 | 0.6454 | 0.3736 | 0.4109 | 0.6021 | 0.7248 | 0.9667 |
| 0.3576 | 2.9317 | 10000 | 0.3536 | 0.6591 | 0.3664 | 0.4012 | 0.6015 | 0.7305 | 0.9666 |
| 0.3616 | 3.2249 | 11000 | 0.3558 | 0.6652 | 0.3813 | 0.4223 | 0.6011 | 0.7362 | 0.9666 |
| 0.3482 | 3.5180 | 12000 | 0.3504 | 0.6589 | 0.3773 | 0.4163 | 0.6046 | 0.7259 | 0.9671 |
| 0.3616 | 3.8112 | 13000 | 0.3550 | 0.6670 | 0.3889 | 0.4326 | 0.6036 | 0.7384 | 0.9666 |
| 0.36 | 4.1044 | 14000 | 0.3515 | 0.6646 | 0.3844 | 0.4262 | 0.6049 | 0.7346 | 0.9671 |
| 0.3553 | 4.3975 | 15000 | 0.3519 | 0.6779 | 0.3887 | 0.4336 | 0.6058 | 0.7369 | 0.9668 |
| 0.3457 | 4.6907 | 16000 | 0.3485 | 0.6668 | 0.3852 | 0.4275 | 0.6049 | 0.7199 | 0.9671 |
| 0.3544 | 4.9839 | 17000 | 0.3530 | 0.6510 | 0.3703 | 0.4043 | 0.5980 | 0.6881 | 0.9669 |
| 0.349 | 5.2770 | 18000 | 0.3472 | 0.6650 | 0.3785 | 0.4184 | 0.6049 | 0.7189 | 0.9671 |
| 0.3478 | 5.5702 | 19000 | 0.3479 | 0.6629 | 0.3709 | 0.4079 | 0.6038 | 0.7097 | 0.9669 |
| 0.3517 | 5.8634 | 20000 | 0.3470 | 0.6630 | 0.3830 | 0.4265 | 0.6051 | 0.7154 | 0.9669 |
| 0.3552 | 6.1566 | 21000 | 0.3475 | 0.6536 | 0.3767 | 0.4165 | 0.6041 | 0.7113 | 0.9667 |
| 0.3619 | 6.4497 | 22000 | 0.3472 | 0.6705 | 0.3739 | 0.4127 | 0.6071 | 0.7275 | 0.9667 |
| 0.3534 | 6.7429 | 23000 | 0.3626 | 0.6590 | 0.3978 | 0.4435 | 0.5977 | 0.7434 | 0.9667 |
| 0.3494 | 7.0361 | 24000 | 0.3479 | 0.6711 | 0.3905 | 0.4363 | 0.6078 | 0.7312 | 0.9668 |
| 0.3488 | 7.3292 | 25000 | 0.3455 | 0.6647 | 0.3836 | 0.4251 | 0.6075 | 0.7258 | 0.9671 |
| 0.3557 | 7.6224 | 26000 | 0.3457 | 0.6699 | 0.3890 | 0.4328 | 0.6066 | 0.7193 | 0.9671 |
| 0.3442 | 7.9156 | 27000 | 0.3461 | 0.6622 | 0.3726 | 0.4111 | 0.6060 | 0.7214 | 0.9671 |
| 0.3648 | 8.2087 | 28000 | 0.3463 | 0.6742 | 0.3834 | 0.4260 | 0.6087 | 0.7310 | 0.9667 |
| 0.35 | 8.5019 | 29000 | 0.3463 | 0.6577 | 0.3966 | 0.4434 | 0.6086 | 0.7317 | 0.9674 |
| 0.3539 | 8.7951 | 30000 | 0.3526 | 0.6478 | 0.3654 | 0.3975 | 0.5977 | 0.6838 | 0.9669 |
| 0.3475 | 9.0882 | 31000 | 0.3454 | 0.6761 | 0.3898 | 0.4352 | 0.6090 | 0.7326 | 0.9670 |
| 0.3535 | 9.3814 | 32000 | 0.3447 | 0.6719 | 0.3895 | 0.4346 | 0.6092 | 0.7291 | 0.9671 |
| 0.3514 | 9.6746 | 33000 | 0.3445 | 0.6660 | 0.3918 | 0.4380 | 0.6088 | 0.7281 | 0.9671 |
| 0.3504 | 9.9678 | 34000 | 0.3455 | 0.6698 | 0.3914 | 0.4363 | 0.6094 | 0.7349 | 0.9672 |
| 0.3629 | 10.2609 | 35000 | 0.3550 | 0.6620 | 0.3935 | 0.4389 | 0.6036 | 0.7441 | 0.9666 |
| 0.3436 | 10.5541 | 36000 | 0.3491 | 0.6607 | 0.3959 | 0.4422 | 0.6074 | 0.7403 | 0.9672 |
| 0.3516 | 10.8473 | 37000 | 0.3461 | 0.6641 | 0.3792 | 0.4166 | 0.6051 | 0.7062 | 0.9670 |
| 0.3483 | 11.1404 | 38000 | 0.3435 | 0.6705 | 0.3894 | 0.4348 | 0.6071 | 0.7214 | 0.9671 |
| 0.3506 | 11.4336 | 39000 | 0.3450 | 0.6595 | 0.3969 | 0.4447 | 0.6101 | 0.7344 | 0.9672 |
| 0.3484 | 11.7268 | 40000 | 0.3440 | 0.6641 | 0.3934 | 0.4380 | 0.6086 | 0.7273 | 0.9674 |
| 0.3491 | 12.0199 | 41000 | 0.3589 | 0.6601 | 0.3959 | 0.4408 | 0.6001 | 0.7454 | 0.9668 |
| 0.3468 | 12.3131 | 42000 | 0.3456 | 0.6731 | 0.3895 | 0.4345 | 0.6087 | 0.7371 | 0.9674 |
| 0.3461 | 12.6063 | 43000 | 0.3431 | 0.6730 | 0.3897 | 0.4357 | 0.6096 | 0.7274 | 0.9675 |
| 0.355 | 12.8994 | 44000 | 0.3440 | 0.6672 | 0.3812 | 0.4221 | 0.6069 | 0.7141 | 0.9672 |
| 0.347 | 13.1926 | 45000 | 0.3469 | 0.6574 | 0.3804 | 0.4180 | 0.6039 | 0.7030 | 0.9669 |
| 0.3453 | 13.4858 | 46000 | 0.3443 | 0.6736 | 0.3882 | 0.4326 | 0.6102 | 0.7350 | 0.9669 |
| 0.3463 | 13.7790 | 47000 | 0.3428 | 0.6640 | 0.3877 | 0.4308 | 0.6085 | 0.7186 | 0.9673 |
| 0.3485 | 14.0721 | 48000 | 0.3433 | 0.6667 | 0.3965 | 0.4438 | 0.6097 | 0.7308 | 0.9673 |
| 0.3539 | 14.3653 | 49000 | 0.3429 | 0.6570 | 0.3900 | 0.4343 | 0.6079 | 0.7214 | 0.9672 |
| 0.3541 | 14.6585 | 50000 | 0.3487 | 0.6623 | 0.3950 | 0.4412 | 0.6066 | 0.7417 | 0.9671 |
| 0.3523 | 14.9516 | 51000 | 0.3455 | 0.6512 | 0.3869 | 0.4283 | 0.6048 | 0.7027 | 0.9668 |
| 0.3372 | 15.2448 | 52000 | 0.3453 | 0.6706 | 0.3975 | 0.4449 | 0.6089 | 0.7375 | 0.9673 |
| 0.3522 | 15.5380 | 53000 | 0.3428 | 0.6680 | 0.3964 | 0.4441 | 0.6112 | 0.7326 | 0.9672 |
| 0.3457 | 15.8311 | 54000 | 0.3441 | 0.6738 | 0.3914 | 0.4382 | 0.6099 | 0.7363 | 0.9668 |
| 0.3403 | 16.1243 | 55000 | 0.3443 | 0.6643 | 0.3958 | 0.4432 | 0.6097 | 0.7364 | 0.9672 |
| 0.3448 | 16.4175 | 56000 | 0.3436 | 0.6703 | 0.3819 | 0.4262 | 0.6066 | 0.7182 | 0.9667 |
| 0.3316 | 16.7106 | 57000 | 0.3419 | 0.6640 | 0.3941 | 0.4405 | 0.6092 | 0.7223 | 0.9672 |
| 0.3436 | 17.0038 | 58000 | 0.3496 | 0.6626 | 0.4037 | 0.4526 | 0.6073 | 0.7431 | 0.9672 |
| 0.3507 | 17.2970 | 59000 | 0.3478 | 0.6624 | 0.4038 | 0.4530 | 0.6093 | 0.7424 | 0.9670 |
| 0.3407 | 17.5901 | 60000 | 0.3416 | 0.6654 | 0.3921 | 0.4380 | 0.6099 | 0.7268 | 0.9672 |
| 0.3483 | 17.8833 | 61000 | 0.3427 | 0.6679 | 0.3878 | 0.4317 | 0.6100 | 0.7336 | 0.9671 |
| 0.3436 | 18.1765 | 62000 | 0.3436 | 0.6702 | 0.3910 | 0.4377 | 0.6099 | 0.7340 | 0.9668 |
| 0.3474 | 18.4697 | 63000 | 0.3473 | 0.6560 | 0.4018 | 0.4501 | 0.6102 | 0.7429 | 0.9669 |
| 0.3429 | 18.7628 | 64000 | 0.3416 | 0.6609 | 0.3927 | 0.4375 | 0.6096 | 0.7244 | 0.9673 |
| 0.3533 | 19.0560 | 65000 | 0.3417 | 0.6696 | 0.3896 | 0.4348 | 0.6104 | 0.7284 | 0.9673 |
| 0.3375 | 19.3492 | 66000 | 0.3419 | 0.6621 | 0.3924 | 0.4377 | 0.6084 | 0.7154 | 0.9672 |
| 0.3428 | 19.6423 | 67000 | 0.3415 | 0.6700 | 0.3900 | 0.4357 | 0.6107 | 0.7301 | 0.9673 |
| 0.35 | 19.9355 | 68000 | 0.3424 | 0.6553 | 0.3817 | 0.4225 | 0.6074 | 0.7131 | 0.9673 |
| 0.3499 | 20.2287 | 69000 | 0.3478 | 0.6581 | 0.4031 | 0.4515 | 0.6090 | 0.7430 | 0.9674 |
| 0.3429 | 20.5218 | 70000 | 0.3567 | 0.6619 | 0.3969 | 0.4427 | 0.6030 | 0.7470 | 0.9661 |
| 0.3342 | 20.8150 | 71000 | 0.3415 | 0.6677 | 0.3920 | 0.4388 | 0.6107 | 0.7302 | 0.9674 |
| 0.3381 | 21.1082 | 72000 | 0.3482 | 0.6587 | 0.4022 | 0.4509 | 0.6078 | 0.7428 | 0.9668 |
| 0.3524 | 21.4013 | 73000 | 0.3439 | 0.6600 | 0.4036 | 0.4533 | 0.6116 | 0.7395 | 0.9672 |
| 0.3428 | 21.6945 | 74000 | 0.3423 | 0.6564 | 0.3889 | 0.4329 | 0.6074 | 0.7111 | 0.9671 |
| 0.3447 | 21.9877 | 75000 | 0.3416 | 0.6644 | 0.3929 | 0.4397 | 0.6108 | 0.7321 | 0.9671 |
| 0.3436 | 22.2809 | 76000 | 0.3412 | 0.6552 | 0.4009 | 0.4488 | 0.6098 | 0.7265 | 0.9674 |
| 0.3484 | 22.5740 | 77000 | 0.3430 | 0.6585 | 0.3976 | 0.4459 | 0.6110 | 0.7365 | 0.9670 |
| 0.3401 | 22.8672 | 78000 | 0.3423 | 0.6594 | 0.3920 | 0.4378 | 0.6080 | 0.7101 | 0.9671 |
| 0.3394 | 23.1604 | 79000 | 0.3416 | 0.6606 | 0.3901 | 0.4349 | 0.6088 | 0.7145 | 0.9674 |
| 0.3331 | 23.4535 | 80000 | 0.3413 | 0.6627 | 0.3844 | 0.4273 | 0.6098 | 0.7214 | 0.9672 |
| 0.3587 | 23.7467 | 81000 | 0.3413 | 0.6593 | 0.3904 | 0.4351 | 0.6087 | 0.7157 | 0.9673 |
| 0.3518 | 24.0399 | 82000 | 0.3419 | 0.6641 | 0.3962 | 0.4437 | 0.6112 | 0.7352 | 0.9672 |
| 0.3442 | 24.3330 | 83000 | 0.3406 | 0.6626 | 0.3959 | 0.4432 | 0.6105 | 0.7277 | 0.9673 |
| 0.3386 | 24.6262 | 84000 | 0.3452 | 0.6607 | 0.3984 | 0.4464 | 0.6100 | 0.7409 | 0.9669 |
| 0.3418 | 24.9194 | 85000 | 0.3409 | 0.6655 | 0.3919 | 0.4385 | 0.6098 | 0.7195 | 0.9673 |
| 0.3426 | 25.2125 | 86000 | 0.3420 | 0.6513 | 0.4021 | 0.4514 | 0.6109 | 0.7351 | 0.9672 |
| 0.3293 | 25.5057 | 87000 | 0.3406 | 0.6714 | 0.3926 | 0.4397 | 0.6101 | 0.7289 | 0.9672 |
| 0.3437 | 25.7989 | 88000 | 0.3413 | 0.6506 | 0.4038 | 0.4529 | 0.6104 | 0.7303 | 0.9673 |
| 0.341 | 26.0921 | 89000 | 0.3407 | 0.6663 | 0.3965 | 0.4449 | 0.6108 | 0.7298 | 0.9673 |
| 0.3284 | 26.3852 | 90000 | 0.3407 | 0.6615 | 0.3958 | 0.4435 | 0.6103 | 0.7293 | 0.9673 |
| 0.3463 | 26.6784 | 91000 | 0.3408 | 0.6625 | 0.3916 | 0.4375 | 0.6098 | 0.7180 | 0.9673 |
| 0.3423 | 26.9716 | 92000 | 0.3406 | 0.6652 | 0.3927 | 0.4393 | 0.6104 | 0.7222 | 0.9672 |
| 0.3408 | 27.2647 | 93000 | 0.3405 | 0.6669 | 0.3925 | 0.4393 | 0.6100 | 0.7274 | 0.9672 |
| 0.3431 | 27.5579 | 94000 | 0.3409 | 0.6686 | 0.3921 | 0.4392 | 0.6102 | 0.7305 | 0.9672 |
| 0.3424 | 27.8511 | 95000 | 0.3409 | 0.6664 | 0.3964 | 0.4445 | 0.6105 | 0.7318 | 0.9672 |
| 0.3438 | 28.1442 | 96000 | 0.3405 | 0.6650 | 0.3942 | 0.4414 | 0.6109 | 0.7290 | 0.9672 |
| 0.3328 | 28.4374 | 97000 | 0.3404 | 0.6640 | 0.3934 | 0.4401 | 0.6101 | 0.7220 | 0.9673 |
| 0.3341 | 28.7306 | 98000 | 0.3405 | 0.6602 | 0.3951 | 0.4425 | 0.6107 | 0.7288 | 0.9673 |
| 0.3373 | 29.0237 | 99000 | 0.3403 | 0.6602 | 0.3957 | 0.4428 | 0.6110 | 0.7243 | 0.9673 |
| 0.3377 | 29.3169 | 100000 | 0.3404 | 0.6614 | 0.3948 | 0.4422 | 0.6104 | 0.7278 | 0.9672 |
| 0.3376 | 29.6101 | 101000 | 0.3404 | 0.6622 | 0.3962 | 0.4437 | 0.6110 | 0.7288 | 0.9673 |
| 0.3404 | 29.9033 | 102000 | 0.3403 | 0.6596 | 0.3962 | 0.4436 | 0.6112 | 0.7282 | 0.9673 |
### Framework versions
- Transformers 4.43.4
- Pytorch 2.4.0+cu121
- Datasets 2.21.0
- Tokenizers 0.19.1
|