|
--- |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: mistral-7b-peptide-new-data |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# mistral-7b-peptide-new-data |
|
|
|
This model was trained from scratch on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.6867 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 3 |
|
- eval_batch_size: 3 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 4 |
|
- gradient_accumulation_steps: 4 |
|
- total_train_batch_size: 48 |
|
- total_eval_batch_size: 12 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- lr_scheduler_warmup_steps: 30 |
|
- training_steps: 8000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:------:|:----:|:---------------:| |
|
| 2.0847 | 0.0063 | 50 | 1.7965 | |
|
| 1.5136 | 0.0125 | 100 | 1.4393 | |
|
| 1.3045 | 0.0187 | 150 | 1.2247 | |
|
| 1.1527 | 0.025 | 200 | 1.1058 | |
|
| 1.0242 | 0.0312 | 250 | 0.9986 | |
|
| 0.6946 | 0.0375 | 300 | 0.9598 | |
|
| 0.5299 | 0.0437 | 350 | 0.9767 | |
|
| 0.5212 | 0.05 | 400 | 0.9270 | |
|
| 0.4852 | 0.0563 | 450 | 0.9116 | |
|
| 0.4739 | 0.0625 | 500 | 0.8924 | |
|
| 0.3936 | 0.0688 | 550 | 0.9344 | |
|
| 0.3514 | 0.075 | 600 | 0.9703 | |
|
| 0.3544 | 0.0813 | 650 | 0.9725 | |
|
| 0.3532 | 0.0875 | 700 | 0.9607 | |
|
| 0.3524 | 0.0938 | 750 | 0.9420 | |
|
| 0.3586 | 0.1 | 800 | 0.9656 | |
|
| 0.3171 | 0.1062 | 850 | 0.9821 | |
|
| 0.3218 | 0.1125 | 900 | 0.9767 | |
|
| 0.3264 | 0.1187 | 950 | 0.9796 | |
|
| 0.3211 | 0.125 | 1000 | 0.9690 | |
|
| 0.3279 | 0.1313 | 1050 | 0.9566 | |
|
| 0.2685 | 0.1375 | 1100 | 1.0596 | |
|
| 0.2714 | 0.1437 | 1150 | 1.0163 | |
|
| 0.2865 | 0.15 | 1200 | 1.0333 | |
|
| 0.2763 | 0.1562 | 1250 | 1.0295 | |
|
| 0.2823 | 0.1625 | 1300 | 1.0135 | |
|
| 0.2222 | 0.1688 | 1350 | 1.0825 | |
|
| 0.2091 | 0.175 | 1400 | 1.0671 | |
|
| 0.323 | 0.1812 | 1450 | 1.1212 | |
|
| 0.2035 | 0.1875 | 1500 | 1.0661 | |
|
| 0.1991 | 0.1938 | 1550 | 1.0609 | |
|
| 0.1775 | 0.2 | 1600 | 1.1045 | |
|
| 0.1624 | 0.2062 | 1650 | 1.1419 | |
|
| 0.1829 | 0.2125 | 1700 | 1.0643 | |
|
| 0.1879 | 0.2188 | 1750 | 1.1223 | |
|
| 0.1667 | 0.225 | 1800 | 1.1179 | |
|
| 0.1618 | 0.2313 | 1850 | 1.1347 | |
|
| 0.1469 | 0.2375 | 1900 | 1.1522 | |
|
| 0.151 | 0.2437 | 1950 | 1.1615 | |
|
| 0.1609 | 0.25 | 2000 | 1.1471 | |
|
| 0.1504 | 0.2562 | 2050 | 1.1457 | |
|
| 0.1452 | 0.2625 | 2100 | 1.1527 | |
|
| 0.1341 | 0.2687 | 2150 | 1.1743 | |
|
| 0.137 | 0.275 | 2200 | 1.1742 | |
|
| 0.1387 | 0.2812 | 2250 | 1.1652 | |
|
| 0.1357 | 0.2875 | 2300 | 1.1657 | |
|
| 0.1347 | 0.2938 | 2350 | 1.1545 | |
|
| 0.1217 | 0.3 | 2400 | 1.1933 | |
|
| 0.1226 | 0.3063 | 2450 | 1.1882 | |
|
| 0.1273 | 0.3125 | 2500 | 1.1999 | |
|
| 0.124 | 0.3187 | 2550 | 1.1980 | |
|
| 0.1197 | 0.325 | 2600 | 1.2027 | |
|
| 0.1134 | 0.3312 | 2650 | 1.2114 | |
|
| 0.112 | 0.3375 | 2700 | 1.2340 | |
|
| 0.1132 | 0.3438 | 2750 | 1.2302 | |
|
| 0.1127 | 0.35 | 2800 | 1.2177 | |
|
| 0.1088 | 0.3563 | 2850 | 1.2415 | |
|
| 0.1022 | 0.3625 | 2900 | 1.2502 | |
|
| 0.0988 | 0.3688 | 2950 | 1.2659 | |
|
| 0.0998 | 0.375 | 3000 | 1.2661 | |
|
| 0.1012 | 0.3812 | 3050 | 1.2714 | |
|
| 0.0974 | 0.3875 | 3100 | 1.2615 | |
|
| 0.0948 | 0.3937 | 3150 | 1.2465 | |
|
| 0.0903 | 0.4 | 3200 | 1.2662 | |
|
| 0.088 | 0.4062 | 3250 | 1.2820 | |
|
| 0.0895 | 0.4125 | 3300 | 1.2749 | |
|
| 0.0903 | 0.4188 | 3350 | 1.2479 | |
|
| 0.0871 | 0.425 | 3400 | 1.2638 | |
|
| 0.0733 | 0.4313 | 3450 | 1.3351 | |
|
| 0.0735 | 0.4375 | 3500 | 1.3046 | |
|
| 0.0818 | 0.4437 | 3550 | 1.3131 | |
|
| 0.08 | 0.45 | 3600 | 1.3224 | |
|
| 0.0795 | 0.4562 | 3650 | 1.3298 | |
|
| 0.0746 | 0.4625 | 3700 | 1.3175 | |
|
| 0.0706 | 0.4688 | 3750 | 1.3807 | |
|
| 0.0711 | 0.475 | 3800 | 1.3475 | |
|
| 0.0748 | 0.4813 | 3850 | 1.3502 | |
|
| 0.0705 | 0.4875 | 3900 | 1.3271 | |
|
| 0.0685 | 0.4938 | 3950 | 1.3551 | |
|
| 0.0663 | 0.5 | 4000 | 1.3735 | |
|
| 0.0663 | 0.5062 | 4050 | 1.3789 | |
|
| 0.0654 | 0.5125 | 4100 | 1.3495 | |
|
| 0.0658 | 0.5188 | 4150 | 1.3363 | |
|
| 0.0633 | 0.525 | 4200 | 1.3569 | |
|
| 0.0621 | 0.5312 | 4250 | 1.3798 | |
|
| 0.0636 | 0.5375 | 4300 | 1.3904 | |
|
| 0.0635 | 0.5437 | 4350 | 1.4183 | |
|
| 0.0597 | 0.55 | 4400 | 1.3955 | |
|
| 0.0574 | 0.5563 | 4450 | 1.3847 | |
|
| 0.0588 | 0.5625 | 4500 | 1.4347 | |
|
| 0.0575 | 0.5687 | 4550 | 1.4519 | |
|
| 0.0574 | 0.575 | 4600 | 1.4268 | |
|
| 0.056 | 0.5813 | 4650 | 1.4242 | |
|
| 0.0535 | 0.5875 | 4700 | 1.4149 | |
|
| 0.0523 | 0.5938 | 4750 | 1.4397 | |
|
| 0.0463 | 0.6 | 4800 | 1.4837 | |
|
| 0.0485 | 0.6062 | 4850 | 1.4928 | |
|
| 0.0472 | 0.6125 | 4900 | 1.4878 | |
|
| 0.0465 | 0.6188 | 4950 | 1.5182 | |
|
| 0.0391 | 0.625 | 5000 | 1.4831 | |
|
| 0.0389 | 0.6312 | 5050 | 1.4707 | |
|
| 0.0443 | 0.6375 | 5100 | 1.4903 | |
|
| 0.0367 | 0.6438 | 5150 | 1.5244 | |
|
| 0.033 | 0.65 | 5200 | 1.4586 | |
|
| 0.0352 | 0.6562 | 5250 | 1.4376 | |
|
| 0.0353 | 0.6625 | 5300 | 1.5125 | |
|
| 0.0309 | 0.6687 | 5350 | 1.5366 | |
|
| 0.0273 | 0.675 | 5400 | 1.4890 | |
|
| 0.0313 | 0.6813 | 5450 | 1.5407 | |
|
| 0.0243 | 0.6875 | 5500 | 1.5580 | |
|
| 0.0259 | 0.6937 | 5550 | 1.5675 | |
|
| 0.0247 | 0.7 | 5600 | 1.5824 | |
|
| 0.0212 | 0.7063 | 5650 | 1.5901 | |
|
| 0.0228 | 0.7125 | 5700 | 1.5499 | |
|
| 0.0248 | 0.7188 | 5750 | 1.5870 | |
|
| 0.0252 | 0.725 | 5800 | 1.5419 | |
|
| 0.0177 | 0.7312 | 5850 | 1.5714 | |
|
| 0.0239 | 0.7375 | 5900 | 1.5993 | |
|
| 0.0252 | 0.7438 | 5950 | 1.5668 | |
|
| 0.0243 | 0.75 | 6000 | 1.5898 | |
|
| 0.0219 | 0.7562 | 6050 | 1.5875 | |
|
| 0.0208 | 0.7625 | 6100 | 1.5930 | |
|
| 0.0245 | 0.7688 | 6150 | 1.5847 | |
|
| 0.0216 | 0.775 | 6200 | 1.6443 | |
|
| 0.0222 | 0.7812 | 6250 | 1.6116 | |
|
| 0.0175 | 0.7875 | 6300 | 1.6632 | |
|
| 0.0211 | 0.7937 | 6350 | 1.6293 | |
|
| 0.0218 | 0.8 | 6400 | 1.6341 | |
|
| 0.0212 | 0.8063 | 6450 | 1.6336 | |
|
| 0.0198 | 0.8125 | 6500 | 1.6720 | |
|
| 0.0217 | 0.8187 | 6550 | 1.6364 | |
|
| 0.0211 | 0.825 | 6600 | 1.6325 | |
|
| 0.0196 | 0.8313 | 6650 | 1.6860 | |
|
| 0.0231 | 0.8375 | 6700 | 1.6489 | |
|
| 0.0216 | 0.8438 | 6750 | 1.6443 | |
|
| 0.0229 | 0.85 | 6800 | 1.6406 | |
|
| 0.0204 | 0.8562 | 6850 | 1.6545 | |
|
| 0.0219 | 0.8625 | 6900 | 1.6468 | |
|
| 0.0235 | 0.8688 | 6950 | 1.6207 | |
|
| 0.022 | 0.875 | 7000 | 1.6522 | |
|
| 0.0188 | 0.8812 | 7050 | 1.6853 | |
|
| 0.0204 | 0.8875 | 7100 | 1.6584 | |
|
| 0.0197 | 0.8938 | 7150 | 1.6843 | |
|
| 0.0208 | 0.9 | 7200 | 1.7061 | |
|
| 0.0205 | 0.9062 | 7250 | 1.6769 | |
|
| 0.0235 | 0.9125 | 7300 | 1.6619 | |
|
| 0.0198 | 0.9187 | 7350 | 1.6702 | |
|
| 0.0216 | 0.925 | 7400 | 1.6880 | |
|
| 0.0221 | 0.9313 | 7450 | 1.6701 | |
|
| 0.0224 | 0.9375 | 7500 | 1.6614 | |
|
| 0.0193 | 0.9437 | 7550 | 1.6734 | |
|
| 0.0208 | 0.95 | 7600 | 1.6836 | |
|
| 0.0199 | 0.9563 | 7650 | 1.6981 | |
|
| 0.0223 | 0.9625 | 7700 | 1.6915 | |
|
| 0.0184 | 0.9688 | 7750 | 1.6524 | |
|
| 0.018 | 0.975 | 7800 | 1.7108 | |
|
| 0.0184 | 0.9812 | 7850 | 1.6707 | |
|
| 0.022 | 0.9875 | 7900 | 1.6735 | |
|
| 0.0229 | 0.9938 | 7950 | 1.6733 | |
|
| 0.0222 | 1.0 | 8000 | 1.6867 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.44.0.dev0 |
|
- Pytorch 2.1.0+cu121 |
|
- Datasets 2.20.0 |
|
- Tokenizers 0.19.1 |
|
|
