Hugging Face's logo

yimingzhang
/
zephyr-7b-dpo-full

Text Generation
Transformers
TensorBoard
Safetensors
mistral
Generated from Trainer
conversational
text-generation-inference
Model card Files Metrics Community
yimingzhang commited on
Commit
428274d
·
verified ·
1 Parent(s): 5826fc1

Model save

Browse files
Files changed (14) hide show
  1. README.md +67 -70
  2. all_results.json +17 -17
  3. config.json +1 -0
  4. eval_results.json +13 -13
  5. model-00001-of-00003.safetensors +1 -1
  6. model-00002-of-00003.safetensors +1 -1
  7. model-00003-of-00003.safetensors +1 -1
  8. runs/Jan09_01-55-21_babel-5-7/events.out.tfevents.1704783595.babel-5-7.3168828.0 +3 -0
  9. runs/Jan09_14-15-49_babel-5-7/events.out.tfevents.1704827798.babel-5-7.11663.0 +3 -0
  10. runs/Jan09_14-15-49_babel-5-7/events.out.tfevents.1704905056.babel-5-7.11663.1 +3 -0
  11. tokenizer_config.json +1 -1
  12. train_results.json +4 -4
  13. trainer_state.json +0 -0
  14. training_args.bin +1 -1
README.md CHANGED
@@ -15,15 +15,15 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.6578
19
- - Rewards/chosen: -2.4796
20
- - Rewards/rejected: -6.7098
21
- - Rewards/accuracies: 0.7849
22
- - Rewards/margins: 4.2302
23
- - Logps/rejected: -336.6310
24
- - Logps/chosen: -339.6573
25
- - Logits/rejected: -2.7799
26
- - Logits/chosen: -2.7857
27
 
28
  ## Model description
29
 
@@ -59,67 +59,64 @@ The following hyperparameters were used during training:
59
 
60
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
- | 0.6163 | 0.05 | 100 | 0.6102 | 0.4041 | 0.1032 | 0.6581 | 0.3009 | -268.5007 | -310.8200 | -2.9269 | -2.9344 |
63
- | 0.5771 | 0.1 | 200 | 0.5582 | 0.3265 | -0.3378 | 0.7243 | 0.6642 | -272.9109 | -311.5964 | -2.9032 | -2.9106 |
64
- | 0.586 | 0.15 | 300 | 0.5412 | 0.1738 | -0.5739 | 0.7059 | 0.7477 | -275.2723 | -313.1228 | -2.9433 | -2.9511 |
65
- | 0.5813 | 0.2 | 400 | 0.5903 | 0.6425 | -0.3995 | 0.7169 | 1.0420 | -273.5283 | -308.4358 | -2.9390 | -2.9467 |
66
- | 0.5532 | 0.24 | 500 | 0.5900 | 0.0572 | -0.9069 | 0.7132 | 0.9641 | -278.6020 | -314.2893 | -2.9779 | -2.9873 |
67
- | 0.8947 | 0.29 | 600 | 0.6950 | -0.1311 | -0.9980 | 0.7040 | 0.8669 | -279.5131 | -316.1719 | -2.9452 | -2.9646 |
68
- | 0.6725 | 0.34 | 700 | 0.6230 | 0.4265 | -0.8118 | 0.6930 | 1.2384 | -277.6516 | -310.5957 | -3.0431 | -3.0603 |
69
- | 0.5614 | 0.39 | 800 | 0.6195 | 0.3130 | -0.9523 | 0.7279 | 1.2653 | -279.0566 | -311.7314 | -3.0410 | -3.0701 |
70
- | 0.6018 | 0.44 | 900 | 0.5879 | 0.1986 | -1.1034 | 0.7316 | 1.3020 | -280.5670 | -312.8752 | -2.9048 | -2.9212 |
71
- | 0.5223 | 0.49 | 1000 | 0.5430 | 0.0003 | -1.3865 | 0.7463 | 1.3868 | -283.3980 | -314.8583 | -3.0152 | -3.0240 |
72
- | 0.5991 | 0.54 | 1100 | 0.5443 | 0.3420 | -1.1651 | 0.7298 | 1.5071 | -281.1841 | -311.4412 | -2.9692 | -2.9694 |
73
- | 0.6307 | 0.59 | 1200 | 0.5672 | 0.2669 | -1.3556 | 0.7188 | 1.6225 | -283.0891 | -312.1918 | -3.0108 | -3.0177 |
74
- | 0.5154 | 0.64 | 1300 | 0.5394 | 0.2294 | -1.4459 | 0.7482 | 1.6753 | -283.9921 | -312.5673 | -2.9960 | -2.9995 |
75
- | 0.6424 | 0.68 | 1400 | 0.5720 | 0.1759 | -1.3868 | 0.7261 | 1.5627 | -283.4010 | -313.1016 | -2.8801 | -2.8847 |
76
- | 0.587 | 0.73 | 1500 | 0.5491 | 0.0712 | -1.5333 | 0.7224 | 1.6045 | -284.8664 | -314.1490 | -2.8348 | -2.8401 |
77
- | 0.5185 | 0.78 | 1600 | 0.5475 | 0.5502 | -1.0613 | 0.7555 | 1.6115 | -280.1465 | -309.3589 | -2.8407 | -2.8521 |
78
- | 0.6767 | 0.83 | 1700 | 0.5471 | 0.2562 | -1.4091 | 0.7574 | 1.6652 | -283.6239 | -312.2993 | -2.8209 | -2.8255 |
79
- | 0.48 | 0.88 | 1800 | 0.5310 | 0.3013 | -1.4781 | 0.7555 | 1.7794 | -284.3137 | -311.8475 | -2.8333 | -2.8411 |
80
- | 0.5443 | 0.93 | 1900 | 0.5323 | 0.3830 | -1.5307 | 0.7702 | 1.9137 | -284.8400 | -311.0307 | -2.8435 | -2.8521 |
81
- | 0.5506 | 0.98 | 2000 | 0.5222 | -0.1696 | -2.0960 | 0.7537 | 1.9264 | -290.4926 | -316.5567 | -2.8470 | -2.8491 |
82
- | 0.1276 | 1.03 | 2100 | 0.5308 | -0.0587 | -2.3160 | 0.7537 | 2.2573 | -292.6928 | -315.4478 | -2.8374 | -2.8447 |
83
- | 0.1082 | 1.07 | 2200 | 0.5518 | -0.1785 | -2.5136 | 0.7647 | 2.3351 | -294.6690 | -316.6463 | -2.8418 | -2.8504 |
84
- | 0.1003 | 1.12 | 2300 | 0.5922 | -0.2226 | -2.7697 | 0.7610 | 2.5471 | -297.2297 | -317.0865 | -2.8248 | -2.8383 |
85
- | 0.129 | 1.17 | 2400 | 0.5731 | -0.1264 | -2.7407 | 0.7665 | 2.6143 | -296.9401 | -316.1251 | -2.8390 | -2.8419 |
86
- | 0.1052 | 1.22 | 2500 | 0.5872 | -0.1003 | -2.7871 | 0.7684 | 2.6868 | -297.4046 | -315.8643 | -2.8816 | -2.8835 |
87
- | 0.0895 | 1.27 | 2600 | 0.6246 | -0.1075 | -2.8772 | 0.7610 | 2.7697 | -298.3056 | -315.9363 | -2.8468 | -2.8509 |
88
- | 0.0801 | 1.32 | 2700 | 0.5785 | -0.8504 | -3.7473 | 0.7721 | 2.8969 | -307.0061 | -323.3646 | -2.7997 | -2.7980 |
89
- | 0.1012 | 1.37 | 2800 | 0.5710 | -0.4670 | -3.4414 | 0.7849 | 2.9745 | -303.9475 | -319.5309 | -2.7950 | -2.7899 |
90
- | 0.0899 | 1.42 | 2900 | 0.5645 | -0.9008 | -3.8628 | 0.7702 | 2.9621 | -308.1616 | -323.8686 | -2.8207 | -2.8211 |
91
- | 0.1434 | 1.47 | 3000 | 0.5710 | -0.6881 | -3.3126 | 0.7702 | 2.6245 | -302.6590 | -321.7423 | -2.8740 | -2.8758 |
92
- | 0.1172 | 1.51 | 3100 | 0.5433 | -1.1037 | -3.7443 | 0.7849 | 2.6405 | -306.9757 | -325.8981 | -2.8496 | -2.8471 |
93
- | 0.0997 | 1.56 | 3200 | 0.5483 | -0.8675 | -3.6998 | 0.7757 | 2.8323 | -306.5312 | -323.5364 | -2.8201 | -2.8260 |
94
- | 0.0793 | 1.61 | 3300 | 0.5521 | -0.3552 | -3.2334 | 0.7886 | 2.8782 | -301.8674 | -318.4130 | -2.8553 | -2.8636 |
95
- | 0.0706 | 1.66 | 3400 | 0.5406 | -0.4625 | -3.2643 | 0.7702 | 2.8017 | -302.1759 | -319.4865 | -2.8646 | -2.8763 |
96
- | 0.115 | 1.71 | 3500 | 0.5674 | -0.8059 | -3.5663 | 0.7739 | 2.7604 | -305.1960 | -322.9196 | -2.8838 | -2.8907 |
97
- | 0.1311 | 1.76 | 3600 | 0.5627 | -0.8274 | -3.7111 | 0.7721 | 2.8837 | -306.6445 | -323.1350 | -2.9513 | -2.9536 |
98
- | 0.1318 | 1.81 | 3700 | 0.5681 | -0.8221 | -3.7147 | 0.7757 | 2.8926 | -306.6800 | -323.0818 | -2.9496 | -2.9519 |
99
- | 0.0986 | 1.86 | 3800 | 0.5459 | -0.9712 | -3.7337 | 0.7868 | 2.7625 | -306.8705 | -324.5732 | -2.9532 | -2.9521 |
100
- | 0.1091 | 1.91 | 3900 | 0.5304 | -0.7304 | -3.5666 | 0.7978 | 2.8362 | -305.1989 | -322.1651 | -2.9126 | -2.9083 |
101
- | 0.0919 | 1.95 | 4000 | 0.5359 | -0.7072 | -3.6188 | 0.7941 | 2.9116 | -305.7208 | -321.9329 | -2.9641 | -2.9594 |
102
- | 0.0389 | 2.0 | 4100 | 0.5431 | -0.8382 | -3.8823 | 0.7960 | 3.0441 | -308.3558 | -323.2430 | -2.9283 | -2.9238 |
103
- | 0.0056 | 2.05 | 4200 | 0.5895 | -1.3165 | -4.8364 | 0.7996 | 3.5199 | -317.8972 | -328.0264 | -2.9077 | -2.9064 |
104
- | 0.0192 | 2.1 | 4300 | 0.6121 | -1.7383 | -5.4734 | 0.7868 | 3.7351 | -324.2670 | -332.2442 | -2.9048 | -2.9052 |
105
- | 0.0241 | 2.15 | 4400 | 0.6286 | -1.9652 | -5.8420 | 0.7812 | 3.8768 | -327.9529 | -334.5132 | -2.8892 | -2.8890 |
106
- | 0.0091 | 2.2 | 4500 | 0.6283 | -2.1645 | -6.2079 | 0.7849 | 4.0434 | -331.6122 | -336.5059 | -2.8420 | -2.8453 |
107
- | 0.0493 | 2.25 | 4600 | 0.6470 | -2.2950 | -6.4230 | 0.7776 | 4.1281 | -333.7635 | -337.8107 | -2.8227 | -2.8274 |
108
- | 0.0155 | 2.3 | 4700 | 0.6362 | -2.0410 | -6.0829 | 0.7849 | 4.0419 | -330.3618 | -335.2705 | -2.8404 | -2.8459 |
109
- | 0.0061 | 2.34 | 4800 | 0.6448 | -2.1377 | -6.2282 | 0.7868 | 4.0905 | -331.8155 | -336.2385 | -2.8279 | -2.8331 |
110
- | 0.0166 | 2.39 | 4900 | 0.6481 | -1.9867 | -6.0035 | 0.7868 | 4.0168 | -329.5686 | -334.7283 | -2.8678 | -2.8704 |
111
- | 0.0122 | 2.44 | 5000 | 0.6697 | -2.3651 | -6.6009 | 0.7757 | 4.2359 | -335.5422 | -338.5115 | -2.8242 | -2.8250 |
112
- | 0.0105 | 2.49 | 5100 | 0.6560 | -2.1710 | -6.3487 | 0.7812 | 4.1777 | -333.0203 | -336.5708 | -2.8010 | -2.8055 |
113
- | 0.0135 | 2.54 | 5200 | 0.6586 | -2.1484 | -6.3377 | 0.7794 | 4.1893 | -332.9100 | -336.3448 | -2.7855 | -2.7913 |
114
- | 0.0134 | 2.59 | 5300 | 0.6482 | -2.1512 | -6.2937 | 0.7757 | 4.1424 | -332.4699 | -336.3733 | -2.7916 | -2.7951 |
115
- | 0.0121 | 2.64 | 5400 | 0.6488 | -2.0332 | -6.2404 | 0.7776 | 4.2072 | -331.9375 | -335.1934 | -2.7945 | -2.7988 |
116
- | 0.0201 | 2.69 | 5500 | 0.6545 | -2.1552 | -6.3595 | 0.7739 | 4.2043 | -333.1280 | -336.4128 | -2.7999 | -2.8030 |
117
- | 0.0067 | 2.74 | 5600 | 0.6635 | -2.3473 | -6.5838 | 0.7757 | 4.2364 | -335.3709 | -338.3344 | -2.8061 | -2.8096 |
118
- | 0.0061 | 2.78 | 5700 | 0.6532 | -2.2098 | -6.4082 | 0.7812 | 4.1985 | -333.6156 | -336.9589 | -2.8122 | -2.8148 |
119
- | 0.0094 | 2.83 | 5800 | 0.6483 | -2.2624 | -6.4509 | 0.7794 | 4.1886 | -334.0426 | -337.4847 | -2.8032 | -2.8063 |
120
- | 0.0128 | 2.88 | 5900 | 0.6510 | -2.3776 | -6.5483 | 0.7831 | 4.1706 | -335.0157 | -338.6371 | -2.7903 | -2.7943 |
121
- | 0.0061 | 2.93 | 6000 | 0.6547 | -2.4427 | -6.6537 | 0.7812 | 4.2110 | -336.0696 | -339.2876 | -2.7825 | -2.7878 |
122
- | 0.0082 | 2.98 | 6100 | 0.6570 | -2.4834 | -6.7156 | 0.7812 | 4.2322 | -336.6893 | -339.6953 | -2.7804 | -2.7861 |
123
 
124
 
125
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.7656
19
+ - Rewards/chosen: -3.8106
20
+ - Rewards/rejected: -6.8888
21
+ - Rewards/accuracies: 0.7405
22
+ - Rewards/margins: 3.0782
23
+ - Logps/rejected: -327.0232
24
+ - Logps/chosen: -316.3568
25
+ - Logits/rejected: -2.4373
26
+ - Logits/chosen: -2.3640
27
 
28
  ## Model description
29
 
 
59
 
60
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
+ | 0.6612 | 0.05 | 100 | 0.6640 | 0.0464 | -0.0349 | 0.6565 | 0.0813 | -258.4842 | -277.7868 | -2.8352 | -2.7411 |
63
+ | 0.5924 | 0.1 | 200 | 0.6068 | 0.0778 | -0.2525 | 0.6927 | 0.3302 | -260.6598 | -277.4728 | -2.8153 | -2.7265 |
64
+ | 0.5488 | 0.15 | 300 | 0.5772 | 0.1688 | -0.4844 | 0.7385 | 0.6531 | -262.9787 | -276.5630 | -2.8364 | -2.7548 |
65
+ | 0.5144 | 0.2 | 400 | 0.5635 | 0.0609 | -0.7604 | 0.7347 | 0.8213 | -265.7392 | -277.6411 | -2.7890 | -2.7072 |
66
+ | 0.5399 | 0.25 | 500 | 0.5393 | 0.0316 | -0.9906 | 0.75 | 1.0221 | -268.0409 | -277.9347 | -2.8372 | -2.7565 |
67
+ | 0.5776 | 0.31 | 600 | 0.5706 | 0.0425 | -0.9799 | 0.7424 | 1.0224 | -267.9345 | -277.8257 | -2.8388 | -2.7569 |
68
+ | 0.5834 | 0.36 | 700 | 0.5596 | 0.0454 | -1.0216 | 0.7424 | 1.0670 | -268.3513 | -277.7964 | -2.7830 | -2.6941 |
69
+ | 0.5394 | 0.41 | 800 | 0.5358 | 0.0804 | -0.9536 | 0.7481 | 1.0341 | -267.6714 | -277.4460 | -2.6313 | -2.5480 |
70
+ | 0.5141 | 0.46 | 900 | 0.5412 | -0.2704 | -1.4309 | 0.7443 | 1.1605 | -272.4444 | -280.9546 | -2.6662 | -2.5832 |
71
+ | 0.51 | 0.51 | 1000 | 0.5350 | -0.2070 | -1.4043 | 0.7366 | 1.1973 | -272.1781 | -280.3206 | -2.7118 | -2.6217 |
72
+ | 0.5219 | 0.56 | 1100 | 0.5405 | -0.1673 | -1.3152 | 0.7290 | 1.1479 | -271.2871 | -279.9233 | -2.7451 | -2.6605 |
73
+ | 0.5391 | 0.61 | 1200 | 0.5320 | -0.2460 | -1.4452 | 0.7405 | 1.1992 | -272.5871 | -280.7106 | -2.7552 | -2.6692 |
74
+ | 0.536 | 0.66 | 1300 | 0.5502 | -0.1919 | -1.3564 | 0.7271 | 1.1645 | -271.6995 | -280.1697 | -2.7006 | -2.6126 |
75
+ | 0.6544 | 0.71 | 1400 | 0.5309 | -0.3757 | -1.6757 | 0.7080 | 1.3000 | -274.8926 | -282.0077 | -2.6970 | -2.6046 |
76
+ | 0.5697 | 0.76 | 1500 | 0.5662 | -0.2493 | -1.4791 | 0.7156 | 1.2297 | -272.9258 | -280.7440 | -2.7656 | -2.6730 |
77
+ | 0.5538 | 0.81 | 1600 | 0.5326 | -0.4658 | -1.6791 | 0.7214 | 1.2134 | -274.9264 | -282.9080 | -2.6934 | -2.5946 |
78
+ | 0.551 | 0.86 | 1700 | 0.5258 | -0.6217 | -1.8893 | 0.7137 | 1.2676 | -277.0278 | -284.4673 | -2.6535 | -2.5567 |
79
+ | 0.5708 | 0.92 | 1800 | 0.5639 | -0.5168 | -1.8962 | 0.7214 | 1.3794 | -277.0974 | -283.4186 | -2.6279 | -2.5564 |
80
+ | 0.5344 | 0.97 | 1900 | 0.5603 | -0.3788 | -1.8158 | 0.7271 | 1.4370 | -276.2931 | -282.0388 | -2.6680 | -2.5998 |
81
+ | 0.0925 | 1.02 | 2000 | 0.5587 | -0.4628 | -2.1277 | 0.7405 | 1.6648 | -279.4120 | -282.8788 | -2.6520 | -2.5825 |
82
+ | 0.112 | 1.07 | 2100 | 0.5731 | -0.6788 | -2.5908 | 0.7481 | 1.9120 | -284.0433 | -285.0383 | -2.5722 | -2.5094 |
83
+ | 0.0539 | 1.12 | 2200 | 0.5869 | -1.0820 | -2.9310 | 0.7366 | 1.8489 | -287.4448 | -289.0707 | -2.5937 | -2.5303 |
84
+ | 0.0811 | 1.17 | 2300 | 0.6306 | -0.8332 | -2.7204 | 0.7424 | 1.8872 | -285.3392 | -286.5822 | -2.5137 | -2.4560 |
85
+ | 0.0877 | 1.22 | 2400 | 0.5963 | -1.3075 | -3.3622 | 0.7481 | 2.0548 | -291.7576 | -291.3254 | -2.5925 | -2.5291 |
86
+ | 0.1114 | 1.27 | 2500 | 0.6126 | -1.3609 | -3.5524 | 0.7462 | 2.1915 | -293.6587 | -291.8594 | -2.4792 | -2.4142 |
87
+ | 0.0864 | 1.32 | 2600 | 0.6457 | -1.6093 | -3.7584 | 0.75 | 2.1491 | -295.7195 | -294.3440 | -2.5710 | -2.5058 |
88
+ | 0.0708 | 1.37 | 2700 | 0.6080 | -1.8094 | -3.7042 | 0.7462 | 1.8948 | -295.1769 | -296.3445 | -2.5394 | -2.4684 |
89
+ | 0.0794 | 1.42 | 2800 | 0.6010 | -1.7685 | -3.8603 | 0.7538 | 2.0918 | -296.7380 | -295.9354 | -2.5369 | -2.4663 |
90
+ | 0.1009 | 1.48 | 2900 | 0.6102 | -1.6050 | -3.5962 | 0.7347 | 1.9912 | -294.0973 | -294.3007 | -2.4834 | -2.4073 |
91
+ | 0.083 | 1.53 | 3000 | 0.6125 | -1.6395 | -3.6683 | 0.7424 | 2.0288 | -294.8184 | -294.6455 | -2.5306 | -2.4521 |
92
+ | 0.0871 | 1.58 | 3100 | 0.6392 | -1.7447 | -3.8250 | 0.75 | 2.0802 | -296.3850 | -295.6979 | -2.5032 | -2.4279 |
93
+ | 0.1168 | 1.63 | 3200 | 0.5973 | -1.6226 | -3.5602 | 0.7443 | 1.9376 | -293.7374 | -294.4764 | -2.5372 | -2.4606 |
94
+ | 0.0699 | 1.68 | 3300 | 0.5816 | -1.6383 | -3.5364 | 0.7424 | 1.8982 | -293.4994 | -294.6331 | -2.5287 | -2.4527 |
95
+ | 0.1082 | 1.73 | 3400 | 0.5895 | -1.8055 | -3.7976 | 0.7424 | 1.9920 | -296.1109 | -296.3059 | -2.5178 | -2.4442 |
96
+ | 0.09 | 1.78 | 3500 | 0.6231 | -1.8455 | -4.0234 | 0.75 | 2.1779 | -298.3694 | -296.7055 | -2.5261 | -2.4561 |
97
+ | 0.1238 | 1.83 | 3600 | 0.6047 | -1.6771 | -3.5997 | 0.7424 | 1.9226 | -294.1321 | -295.0213 | -2.6294 | -2.5512 |
98
+ | 0.0847 | 1.88 | 3700 | 0.5898 | -1.6725 | -3.5743 | 0.7347 | 1.9018 | -293.8779 | -294.9758 | -2.6224 | -2.5471 |
99
+ | 0.0908 | 1.93 | 3800 | 0.5817 | -1.6076 | -3.5381 | 0.7366 | 1.9304 | -293.5158 | -294.3269 | -2.5778 | -2.5047 |
100
+ | 0.0666 | 1.98 | 3900 | 0.6063 | -1.6950 | -3.7437 | 0.7309 | 2.0487 | -295.5718 | -295.2004 | -2.5784 | -2.5061 |
101
+ | 0.0173 | 2.03 | 4000 | 0.6213 | -2.1227 | -4.3451 | 0.7309 | 2.2224 | -301.5862 | -299.4778 | -2.6197 | -2.5495 |
102
+ | 0.0213 | 2.09 | 4100 | 0.6529 | -2.4461 | -4.9221 | 0.7366 | 2.4759 | -307.3557 | -302.7117 | -2.6029 | -2.5335 |
103
+ | 0.0149 | 2.14 | 4200 | 0.6934 | -3.0653 | -5.7847 | 0.7347 | 2.7194 | -315.9821 | -308.9039 | -2.5938 | -2.5272 |
104
+ | 0.0084 | 2.19 | 4300 | 0.7083 | -3.1845 | -6.0188 | 0.7405 | 2.8343 | -318.3230 | -310.0955 | -2.5088 | -2.4404 |
105
+ | 0.0059 | 2.24 | 4400 | 0.7193 | -3.3983 | -6.2807 | 0.7405 | 2.8824 | -320.9418 | -312.2334 | -2.5109 | -2.4479 |
106
+ | 0.0116 | 2.29 | 4500 | 0.7128 | -3.3425 | -6.1944 | 0.7462 | 2.8519 | -320.0795 | -311.6758 | -2.4787 | -2.4132 |
107
+ | 0.0077 | 2.34 | 4600 | 0.7219 | -3.2306 | -6.1475 | 0.7481 | 2.9169 | -319.6102 | -310.5562 | -2.4449 | -2.3779 |
108
+ | 0.0177 | 2.39 | 4700 | 0.7451 | -3.5469 | -6.5210 | 0.75 | 2.9742 | -323.3456 | -313.7194 | -2.3861 | -2.3174 |
109
+ | 0.0112 | 2.44 | 4800 | 0.7547 | -3.4801 | -6.4397 | 0.7424 | 2.9595 | -322.5316 | -313.0519 | -2.3939 | -2.3242 |
110
+ | 0.0071 | 2.49 | 4900 | 0.7691 | -3.8596 | -6.8490 | 0.7443 | 2.9895 | -326.6253 | -316.8460 | -2.3524 | -2.2834 |
111
+ | 0.0118 | 2.54 | 5000 | 0.7717 | -3.8862 | -6.8731 | 0.7462 | 2.9868 | -326.8659 | -317.1129 | -2.3347 | -2.2658 |
112
+ | 0.014 | 2.59 | 5100 | 0.7685 | -3.5970 | -6.5998 | 0.7481 | 3.0028 | -324.1335 | -314.2205 | -2.3512 | -2.2783 |
113
+ | 0.0208 | 2.64 | 5200 | 0.7741 | -3.9029 | -6.8895 | 0.7443 | 2.9866 | -327.0299 | -317.2794 | -2.3875 | -2.3143 |
114
+ | 0.0076 | 2.7 | 5300 | 0.7600 | -3.6159 | -6.5800 | 0.7424 | 2.9641 | -323.9353 | -314.4092 | -2.4331 | -2.3592 |
115
+ | 0.0146 | 2.75 | 5400 | 0.7768 | -3.7657 | -6.8555 | 0.7424 | 3.0898 | -326.6905 | -315.9074 | -2.4475 | -2.3751 |
116
+ | 0.0161 | 2.8 | 5500 | 0.7902 | -3.9170 | -7.0635 | 0.7481 | 3.1465 | -328.7701 | -317.4208 | -2.4332 | -2.3620 |
117
+ | 0.0056 | 2.85 | 5600 | 0.7827 | -3.9513 | -7.0687 | 0.7424 | 3.1174 | -328.8217 | -317.7632 | -2.4313 | -2.3599 |
118
+ | 0.0083 | 2.9 | 5700 | 0.7741 | -3.8805 | -6.9708 | 0.7443 | 3.0903 | -327.8432 | -317.0560 | -2.4324 | -2.3598 |
119
+ | 0.0243 | 2.95 | 5800 | 0.7657 | -3.8176 | -6.8913 | 0.7405 | 3.0737 | -327.0486 | -316.4268 | -2.4355 | -2.3620 |
 
 
 
120
 
121
 
122
  ### Framework versions
all_results.json CHANGED
@@ -1,21 +1,21 @@
1
  {
2
  "epoch": 3.0,
3
- "eval_logits/chosen": -2.785656690597534,
4
- "eval_logits/rejected": -2.7799313068389893,
5
- "eval_logps/chosen": -339.6573486328125,
6
- "eval_logps/rejected": -336.6309509277344,
7
- "eval_loss": 0.6578378677368164,
8
- "eval_rewards/accuracies": 0.7849264740943909,
9
- "eval_rewards/chosen": -2.4796314239501953,
10
- "eval_rewards/margins": 4.230152130126953,
11
- "eval_rewards/rejected": -6.709782600402832,
12
- "eval_runtime": 303.5228,
13
- "eval_samples": 2172,
14
- "eval_samples_per_second": 7.156,
15
- "eval_steps_per_second": 0.448,
16
- "train_loss": 0.23823042969992683,
17
- "train_runtime": 80075.9645,
18
- "train_samples": 65498,
19
- "train_samples_per_second": 2.454,
20
  "train_steps_per_second": 0.077
21
  }
 
1
  {
2
  "epoch": 3.0,
3
+ "eval_logits/chosen": -2.3639509677886963,
4
+ "eval_logits/rejected": -2.4372596740722656,
5
+ "eval_logps/chosen": -316.3568115234375,
6
+ "eval_logps/rejected": -327.0231628417969,
7
+ "eval_loss": 0.7655794024467468,
8
+ "eval_rewards/accuracies": 0.7404580116271973,
9
+ "eval_rewards/chosen": -3.8106329441070557,
10
+ "eval_rewards/margins": 3.0781688690185547,
11
+ "eval_rewards/rejected": -6.8888020515441895,
12
+ "eval_runtime": 297.1799,
13
+ "eval_samples": 2086,
14
+ "eval_samples_per_second": 7.019,
15
+ "eval_steps_per_second": 0.441,
16
+ "train_loss": 0.22745071912974513,
17
+ "train_runtime": 76961.4992,
18
+ "train_samples": 62901,
19
+ "train_samples_per_second": 2.452,
20
  "train_steps_per_second": 0.077
21
  }
config.json CHANGED
@@ -3,6 +3,7 @@
3
  "architectures": [
4
  "MistralForCausalLM"
5
  ],
 
6
  "bos_token_id": 1,
7
  "eos_token_id": 2,
8
  "hidden_act": "silu",
 
3
  "architectures": [
4
  "MistralForCausalLM"
5
  ],
6
+ "attention_dropout": 0.0,
7
  "bos_token_id": 1,
8
  "eos_token_id": 2,
9
  "hidden_act": "silu",
eval_results.json CHANGED
@@ -1,16 +1,16 @@
1
  {
2
  "epoch": 3.0,
3
- "eval_logits/chosen": -2.785656690597534,
4
- "eval_logits/rejected": -2.7799313068389893,
5
- "eval_logps/chosen": -339.6573486328125,
6
- "eval_logps/rejected": -336.6309509277344,
7
- "eval_loss": 0.6578378677368164,
8
- "eval_rewards/accuracies": 0.7849264740943909,
9
- "eval_rewards/chosen": -2.4796314239501953,
10
- "eval_rewards/margins": 4.230152130126953,
11
- "eval_rewards/rejected": -6.709782600402832,
12
- "eval_runtime": 303.5228,
13
- "eval_samples": 2172,
14
- "eval_samples_per_second": 7.156,
15
- "eval_steps_per_second": 0.448
16
  }
 
1
  {
2
  "epoch": 3.0,
3
+ "eval_logits/chosen": -2.3639509677886963,
4
+ "eval_logits/rejected": -2.4372596740722656,
5
+ "eval_logps/chosen": -316.3568115234375,
6
+ "eval_logps/rejected": -327.0231628417969,
7
+ "eval_loss": 0.7655794024467468,
8
+ "eval_rewards/accuracies": 0.7404580116271973,
9
+ "eval_rewards/chosen": -3.8106329441070557,
10
+ "eval_rewards/margins": 3.0781688690185547,
11
+ "eval_rewards/rejected": -6.8888020515441895,
12
+ "eval_runtime": 297.1799,
13
+ "eval_samples": 2086,
14
+ "eval_samples_per_second": 7.019,
15
+ "eval_steps_per_second": 0.441
16
  }
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:af8b495a095a9dee7ba10a4b15bedfa6a0718b2cd6c735da1409b22509fd83e7
3
  size 4943162336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4683c9835639ca88199a30fa18efd71dfbdf7f86c6cdc1e59e9af471ae3c203c
3
  size 4943162336
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:49b12ada264b41b5c09f42e7d971842fc36a8aedf26dd403a7c3e8036f221192
3
  size 4999819336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2c2940338a9799f0bce8c9893fe7223c746414eb11a0ad6e3b7743a5b46916b
3
  size 4999819336
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:353c13b7eae5ae59900e44c7137185450d73c137f9af5a9d964eb40e54a585f8
3
  size 4540516344
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb5f7ff92c2bedd5ebb4b61b6acae9dd26d9b527458c1e03afe05b5f5979a5dc
3
  size 4540516344
runs/Jan09_01-55-21_babel-5-7/events.out.tfevents.1704783595.babel-5-7.3168828.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5094ed6c1e421f3b7258b9c0b0862675f2d61d62ebd88061ae84bf5deef0e223
3
+ size 212695
runs/Jan09_14-15-49_babel-5-7/events.out.tfevents.1704827798.babel-5-7.11663.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc7083e4a098ba2d43c7ea6428f4b4d9acb811a04782092192642f5d6a0605ea
3
+ size 421539
runs/Jan09_14-15-49_babel-5-7/events.out.tfevents.1704905056.babel-5-7.11663.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e08622733104859505c29b8352b27a92a198da21e3d39ecca67027d4596a18c6
3
+ size 828
tokenizer_config.json CHANGED
@@ -37,5 +37,5 @@
37
  "spaces_between_special_tokens": false,
38
  "tokenizer_class": "LlamaTokenizer",
39
  "unk_token": "<unk>",
40
- "use_default_system_prompt": true
41
  }
 
37
  "spaces_between_special_tokens": false,
38
  "tokenizer_class": "LlamaTokenizer",
39
  "unk_token": "<unk>",
40
+ "use_default_system_prompt": false
41
  }
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 3.0,
3
- "train_loss": 0.23823042969992683,
4
- "train_runtime": 80075.9645,
5
- "train_samples": 65498,
6
- "train_samples_per_second": 2.454,
7
  "train_steps_per_second": 0.077
8
  }
 
1
  {
2
  "epoch": 3.0,
3
+ "train_loss": 0.22745071912974513,
4
+ "train_runtime": 76961.4992,
5
+ "train_samples": 62901,
6
+ "train_samples_per_second": 2.452,
7
  "train_steps_per_second": 0.077
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2f498982c816f77c4a028605f52db4687309be01541cc29774c6624b9063e658
3
  size 5688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50752cbc2a1d72d46415c2572ea96096ac9dfdc00339b3735a404339aa3ba7db
3
  size 5688