Hugging Face's logo

UW-Madison-Lee-Lab
/
VersaPRM-Math-Subset

PEFT
Safetensors
Generated from Trainer
Model card Files Community
UW-Madison-Lee-Lab commited on
Commit
6c9817b
·
verified ·
1 Parent(s): b8541c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -179
README.md CHANGED
@@ -14,182 +14,38 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  # VersaPRM-Math-Subset
16
 
17
- This model is a fine-tuned version of [UW-Madison-Lee-Lab/Llama-PRM800K](https://huggingface.co/UW-Madison-Lee-Lab/Llama-PRM800K) on an unknown dataset.
18
- It achieves the following results on the evaluation set:
19
- - Loss: 0.5381
20
- - Prm accuracy: 0.8651
21
- - Prm precision: 0.875
22
- - Prm recall: 0.9813
23
- - Prm specificty: 0.2105
24
- - Prm npv: 0.6667
25
- - Prm f1: 0.9251
26
- - Prm f1 neg: 0.32
27
- - Prm f1 auc: 0.5959
28
- - Prm f1 auc (fixed): 0.8903
29
-
30
- ## Model description
31
-
32
- More information needed
33
-
34
- ## Intended uses & limitations
35
-
36
- More information needed
37
-
38
- ## Training and evaluation data
39
-
40
- More information needed
41
-
42
- ## Training procedure
43
-
44
- ### Training hyperparameters
45
-
46
- The following hyperparameters were used during training:
47
- - learning_rate: 0.0001
48
- - train_batch_size: 2
49
- - eval_batch_size: 4
50
- - seed: 908932403
51
- - distributed_type: multi-GPU
52
- - num_devices: 8
53
- - gradient_accumulation_steps: 2
54
- - total_train_batch_size: 32
55
- - total_eval_batch_size: 32
56
- - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
57
- - lr_scheduler_type: cosine
58
- - lr_scheduler_warmup_ratio: 0.1
59
- - num_epochs: 3
60
-
61
- ### Training results
62
-
63
- | Training Loss | Epoch | Step | Validation Loss | Prm accuracy | Prm precision | Prm recall | Prm specificty | Prm npv | Prm f1 | Prm f1 neg | Prm f1 auc | Prm f1 auc (fixed) |
64
- |:-------------:|:------:|:----:|:---------------:|:------------:|:-------------:|:----------:|:--------------:|:-------:|:------:|:----------:|:----------:|:------------------:|
65
- | No log | 0 | 0 | 0.3535 | 0.8333 | 0.8772 | 0.9346 | 0.2632 | 0.4167 | 0.9050 | 0.3226 | 0.5989 | 0.8195 |
66
- | 0.2836 | 0.0246 | 5 | 0.3544 | 0.8333 | 0.8772 | 0.9346 | 0.2632 | 0.4167 | 0.9050 | 0.3226 | 0.5989 | 0.8182 |
67
- | 0.2806 | 0.0493 | 10 | 0.3487 | 0.8413 | 0.8718 | 0.9533 | 0.2105 | 0.4444 | 0.9107 | 0.2857 | 0.5819 | 0.8234 |
68
- | 0.269 | 0.0739 | 15 | 0.3733 | 0.8571 | 0.8678 | 0.9813 | 0.1579 | 0.6 | 0.9211 | 0.25 | 0.5696 | 0.8325 |
69
- | 0.2777 | 0.0985 | 20 | 0.4630 | 0.8571 | 0.856 | 1.0 | 0.0526 | 1.0 | 0.9224 | 0.1 | 0.5263 | 0.8524 |
70
- | 0.211 | 0.1232 | 25 | 0.3822 | 0.8571 | 0.856 | 1.0 | 0.0526 | 1.0 | 0.9224 | 0.1 | 0.5263 | 0.8574 |
71
- | 0.2731 | 0.1478 | 30 | 0.3164 | 0.8492 | 0.8729 | 0.9626 | 0.2105 | 0.5 | 0.9156 | 0.2963 | 0.5866 | 0.8534 |
72
- | 0.2432 | 0.1724 | 35 | 0.3146 | 0.8492 | 0.8729 | 0.9626 | 0.2105 | 0.5 | 0.9156 | 0.2963 | 0.5866 | 0.8620 |
73
- | 0.2814 | 0.1970 | 40 | 0.3221 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8706 |
74
- | 0.2079 | 0.2217 | 45 | 0.2970 | 0.8571 | 0.8803 | 0.9626 | 0.2632 | 0.5556 | 0.9196 | 0.3571 | 0.6129 | 0.8728 |
75
- | 0.1482 | 0.2463 | 50 | 0.2934 | 0.8651 | 0.8879 | 0.9626 | 0.3158 | 0.6 | 0.9238 | 0.4138 | 0.6392 | 0.8714 |
76
- | 0.1697 | 0.2709 | 55 | 0.3032 | 0.8571 | 0.8803 | 0.9626 | 0.2632 | 0.5556 | 0.9196 | 0.3571 | 0.6129 | 0.8669 |
77
- | 0.1885 | 0.2956 | 60 | 0.3136 | 0.8492 | 0.8667 | 0.9720 | 0.1579 | 0.5 | 0.9163 | 0.24 | 0.5649 | 0.8741 |
78
- | 0.2268 | 0.3202 | 65 | 0.2937 | 0.8571 | 0.8870 | 0.9533 | 0.3158 | 0.5455 | 0.9189 | 0.4 | 0.6345 | 0.8669 |
79
- | 0.1668 | 0.3448 | 70 | 0.3171 | 0.8492 | 0.8667 | 0.9720 | 0.1579 | 0.5 | 0.9163 | 0.24 | 0.5649 | 0.8758 |
80
- | 0.3353 | 0.3695 | 75 | 0.3119 | 0.8492 | 0.8667 | 0.9720 | 0.1579 | 0.5 | 0.9163 | 0.24 | 0.5649 | 0.8674 |
81
- | 0.1821 | 0.3941 | 80 | 0.3046 | 0.8413 | 0.8655 | 0.9626 | 0.1579 | 0.4286 | 0.9115 | 0.2308 | 0.5603 | 0.8699 |
82
- | 0.2495 | 0.4187 | 85 | 0.3200 | 0.8413 | 0.8595 | 0.9720 | 0.1053 | 0.4 | 0.9123 | 0.1667 | 0.5386 | 0.8788 |
83
- | 0.147 | 0.4433 | 90 | 0.2893 | 0.8571 | 0.8803 | 0.9626 | 0.2632 | 0.5556 | 0.9196 | 0.3571 | 0.6129 | 0.8763 |
84
- | 0.1799 | 0.4680 | 95 | 0.2999 | 0.8651 | 0.8814 | 0.9720 | 0.2632 | 0.625 | 0.9244 | 0.3704 | 0.6176 | 0.8824 |
85
- | 0.1852 | 0.4926 | 100 | 0.3146 | 0.8571 | 0.8739 | 0.9720 | 0.2105 | 0.5714 | 0.9204 | 0.3077 | 0.5912 | 0.8918 |
86
- | 0.1373 | 0.5172 | 105 | 0.2766 | 0.8571 | 0.8803 | 0.9626 | 0.2632 | 0.5556 | 0.9196 | 0.3571 | 0.6129 | 0.8879 |
87
- | 0.1716 | 0.5419 | 110 | 0.2689 | 0.8810 | 0.9035 | 0.9626 | 0.4211 | 0.6667 | 0.9321 | 0.5161 | 0.6918 | 0.8871 |
88
- | 0.1792 | 0.5665 | 115 | 0.3018 | 0.8571 | 0.8739 | 0.9720 | 0.2105 | 0.5714 | 0.9204 | 0.3077 | 0.5912 | 0.9001 |
89
- | 0.1899 | 0.5911 | 120 | 0.2957 | 0.8651 | 0.8814 | 0.9720 | 0.2632 | 0.625 | 0.9244 | 0.3704 | 0.6176 | 0.8930 |
90
- | 0.2089 | 0.6158 | 125 | 0.2795 | 0.8730 | 0.8889 | 0.9720 | 0.3158 | 0.6667 | 0.9286 | 0.4286 | 0.6439 | 0.8888 |
91
- | 0.2129 | 0.6404 | 130 | 0.2894 | 0.8651 | 0.8689 | 0.9907 | 0.1579 | 0.75 | 0.9258 | 0.2609 | 0.5743 | 0.8915 |
92
- | 0.2025 | 0.6650 | 135 | 0.2856 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8886 |
93
- | 0.2093 | 0.6897 | 140 | 0.2958 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8893 |
94
- | 0.1605 | 0.7143 | 145 | 0.2938 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8829 |
95
- | 0.1634 | 0.7389 | 150 | 0.2747 | 0.8651 | 0.8814 | 0.9720 | 0.2632 | 0.625 | 0.9244 | 0.3704 | 0.6176 | 0.8778 |
96
- | 0.143 | 0.7635 | 155 | 0.3065 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8844 |
97
- | 0.1322 | 0.7882 | 160 | 0.2988 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8842 |
98
- | 0.2057 | 0.8128 | 165 | 0.2965 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8800 |
99
- | 0.1796 | 0.8374 | 170 | 0.3372 | 0.8571 | 0.8618 | 0.9907 | 0.1053 | 0.6667 | 0.9217 | 0.1818 | 0.5480 | 0.8864 |
100
- | 0.1777 | 0.8621 | 175 | 0.3096 | 0.8651 | 0.8689 | 0.9907 | 0.1579 | 0.75 | 0.9258 | 0.2609 | 0.5743 | 0.8829 |
101
- | 0.1288 | 0.8867 | 180 | 0.2887 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8778 |
102
- | 0.1733 | 0.9113 | 185 | 0.3124 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8802 |
103
- | 0.1491 | 0.9360 | 190 | 0.3040 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8736 |
104
- | 0.2341 | 0.9606 | 195 | 0.3299 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8731 |
105
- | 0.159 | 0.9852 | 200 | 0.3395 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8751 |
106
- | 0.0717 | 1.0099 | 205 | 0.3115 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8763 |
107
- | 0.0784 | 1.0345 | 210 | 0.3288 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8824 |
108
- | 0.1389 | 1.0591 | 215 | 0.3810 | 0.8571 | 0.8678 | 0.9813 | 0.1579 | 0.6 | 0.9211 | 0.25 | 0.5696 | 0.8842 |
109
- | 0.1232 | 1.0837 | 220 | 0.3459 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8859 |
110
- | 0.0996 | 1.1084 | 225 | 0.3710 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8871 |
111
- | 0.0474 | 1.1330 | 230 | 0.3889 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8829 |
112
- | 0.0599 | 1.1576 | 235 | 0.3975 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8834 |
113
- | 0.1198 | 1.1823 | 240 | 0.3659 | 0.8730 | 0.8824 | 0.9813 | 0.2632 | 0.7143 | 0.9292 | 0.3846 | 0.6222 | 0.8842 |
114
- | 0.1293 | 1.2069 | 245 | 0.3608 | 0.8730 | 0.8824 | 0.9813 | 0.2632 | 0.7143 | 0.9292 | 0.3846 | 0.6222 | 0.8839 |
115
- | 0.1206 | 1.2315 | 250 | 0.3820 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8807 |
116
- | 0.0965 | 1.2562 | 255 | 0.3273 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8795 |
117
- | 0.1145 | 1.2808 | 260 | 0.3459 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8810 |
118
- | 0.1165 | 1.3054 | 265 | 0.3533 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8851 |
119
- | 0.0685 | 1.3300 | 270 | 0.3501 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8842 |
120
- | 0.1442 | 1.3547 | 275 | 0.3470 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8898 |
121
- | 0.1234 | 1.3793 | 280 | 0.3652 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8886 |
122
- | 0.0673 | 1.4039 | 285 | 0.3286 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8837 |
123
- | 0.0821 | 1.4286 | 290 | 0.3253 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8829 |
124
- | 0.1246 | 1.4532 | 295 | 0.3151 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8864 |
125
- | 0.0882 | 1.4778 | 300 | 0.3561 | 0.8571 | 0.8739 | 0.9720 | 0.2105 | 0.5714 | 0.9204 | 0.3077 | 0.5912 | 0.8829 |
126
- | 0.1419 | 1.5025 | 305 | 0.4027 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8819 |
127
- | 0.0634 | 1.5271 | 310 | 0.3743 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8871 |
128
- | 0.1573 | 1.5517 | 315 | 0.3510 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8851 |
129
- | 0.0799 | 1.5764 | 320 | 0.3380 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8910 |
130
- | 0.1266 | 1.6010 | 325 | 0.3646 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8851 |
131
- | 0.1259 | 1.6256 | 330 | 0.3471 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8874 |
132
- | 0.0772 | 1.6502 | 335 | 0.3350 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8920 |
133
- | 0.1339 | 1.6749 | 340 | 0.3411 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8935 |
134
- | 0.0761 | 1.6995 | 345 | 0.3401 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8989 |
135
- | 0.1278 | 1.7241 | 350 | 0.3467 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8982 |
136
- | 0.0595 | 1.7488 | 355 | 0.3457 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8955 |
137
- | 0.0868 | 1.7734 | 360 | 0.3476 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8955 |
138
- | 0.1122 | 1.7980 | 365 | 0.3520 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8918 |
139
- | 0.0634 | 1.8227 | 370 | 0.3592 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8930 |
140
- | 0.0884 | 1.8473 | 375 | 0.3708 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8925 |
141
- | 0.0621 | 1.8719 | 380 | 0.3713 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8940 |
142
- | 0.0729 | 1.8966 | 385 | 0.4072 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8925 |
143
- | 0.0641 | 1.9212 | 390 | 0.4109 | 0.8730 | 0.8760 | 0.9907 | 0.2105 | 0.8 | 0.9298 | 0.3333 | 0.6006 | 0.8876 |
144
- | 0.0699 | 1.9458 | 395 | 0.3871 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8886 |
145
- | 0.0888 | 1.9704 | 400 | 0.3757 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8881 |
146
- | 0.0968 | 1.9951 | 405 | 0.3812 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8881 |
147
- | 0.026 | 2.0197 | 410 | 0.4012 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8906 |
148
- | 0.0429 | 2.0443 | 415 | 0.4218 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8938 |
149
- | 0.0412 | 2.0690 | 420 | 0.4361 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8982 |
150
- | 0.01 | 2.0936 | 425 | 0.4486 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8977 |
151
- | 0.1012 | 2.1182 | 430 | 0.4561 | 0.8571 | 0.8739 | 0.9720 | 0.2105 | 0.5714 | 0.9204 | 0.3077 | 0.5912 | 0.8992 |
152
- | 0.0518 | 2.1429 | 435 | 0.4691 | 0.8571 | 0.8739 | 0.9720 | 0.2105 | 0.5714 | 0.9204 | 0.3077 | 0.5912 | 0.9021 |
153
- | 0.0527 | 2.1675 | 440 | 0.5193 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8989 |
154
- | 0.0763 | 2.1921 | 445 | 0.5364 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8984 |
155
- | 0.0222 | 2.2167 | 450 | 0.5032 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8967 |
156
- | 0.0892 | 2.2414 | 455 | 0.4710 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8955 |
157
- | 0.0236 | 2.2660 | 460 | 0.4699 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8947 |
158
- | 0.0547 | 2.2906 | 465 | 0.5024 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8923 |
159
- | 0.0177 | 2.3153 | 470 | 0.5419 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8915 |
160
- | 0.0503 | 2.3399 | 475 | 0.5531 | 0.8571 | 0.8678 | 0.9813 | 0.1579 | 0.6 | 0.9211 | 0.25 | 0.5696 | 0.8923 |
161
- | 0.0179 | 2.3645 | 480 | 0.5546 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8910 |
162
- | 0.0227 | 2.3892 | 485 | 0.5521 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8903 |
163
- | 0.0238 | 2.4138 | 490 | 0.5510 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8925 |
164
- | 0.0448 | 2.4384 | 495 | 0.5606 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8908 |
165
- | 0.0558 | 2.4631 | 500 | 0.5546 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8925 |
166
- | 0.0647 | 2.4877 | 505 | 0.5585 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8910 |
167
- | 0.0238 | 2.5123 | 510 | 0.5570 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8891 |
168
- | 0.0574 | 2.5369 | 515 | 0.5500 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8898 |
169
- | 0.0407 | 2.5616 | 520 | 0.5473 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8901 |
170
- | 0.021 | 2.5862 | 525 | 0.5454 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8901 |
171
- | 0.0203 | 2.6108 | 530 | 0.5482 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8881 |
172
- | 0.0049 | 2.6355 | 535 | 0.5464 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8901 |
173
- | 0.0483 | 2.6601 | 540 | 0.5463 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8893 |
174
- | 0.02 | 2.6847 | 545 | 0.5462 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8881 |
175
- | 0.0166 | 2.7094 | 550 | 0.5434 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8886 |
176
- | 0.0283 | 2.7340 | 555 | 0.5440 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8906 |
177
- | 0.0056 | 2.7586 | 560 | 0.5415 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8883 |
178
- | 0.0246 | 2.7833 | 565 | 0.5409 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8901 |
179
- | 0.054 | 2.8079 | 570 | 0.5360 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8879 |
180
- | 0.0309 | 2.8325 | 575 | 0.5436 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8891 |
181
- | 0.0276 | 2.8571 | 580 | 0.5374 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8893 |
182
- | 0.0344 | 2.8818 | 585 | 0.5395 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8886 |
183
- | 0.0116 | 2.9064 | 590 | 0.5385 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8896 |
184
- | 0.0169 | 2.9310 | 595 | 0.5359 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8908 |
185
- | 0.0115 | 2.9557 | 600 | 0.5375 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8869 |
186
- | 0.0563 | 2.9803 | 605 | 0.5381 | 0.8651 | 0.875 | 0.9813 | 0.2105 | 0.6667 | 0.9251 | 0.32 | 0.5959 | 0.8903 |
187
-
188
-
189
- ### Framework versions
190
-
191
- - PEFT 0.12.0
192
- - Transformers 4.46.0
193
- - Pytorch 2.4.0+cu118
194
- - Datasets 3.0.0
195
- - Tokenizers 0.20.1
 
14
 
15
  # VersaPRM-Math-Subset
16
 
17
+ This model is a fine-tuned version of [UW-Madison-Lee-Lab/Llama-PRM800K](https://huggingface.co/UW-Madison-Lee-Lab/Llama-PRM800K) on [UW-Madison-Lee-Lab/MMLU-Pro-CoT-Train-Labeled](https://huggingface.co/datasets/UW-Madison-Lee-Lab/MMLU-Pro-CoT-Train-Labeled).
18
+
19
+ ## Get rewards
20
+ ```python
21
+ import torch
22
+ from transformers import AutoModelForCausalLM, AutoTokenizer
23
+
24
+ def get_tokenizer(model_id):
25
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
26
+ tokenizer.pad_token = tokenizer.eos_token
27
+ tokenizer.padding_side = 'left'
28
+ tokenizer.truncation_side = 'left'
29
+ return tokenizer
30
+
31
+ device = 'cuda' if torch.cuda.is_available() else 'cpu'
32
+ tokenizer = get_tokenizer('UW-Madison-Lee-Lab/VersaPRM-Math-Subset')
33
+ model = AutoModelForCausalLM.from_pretrained('UW-Madison-Lee-Lab/VersaPRM-Math-Subset')
34
+ candidate_tokens = [12, 10]
35
+ model.to(device)
36
+
37
+ question = 'Question: In Python 3, which of the following function convert a string to an int in python?\nA. short(x)\nB. float(x)\nC. integer(x [,base])\nD. double(x)\nE. int(x [,base])\nF. long(x [,base] )\nG. num(x)\nH. str(x)\nI. char(x)\nJ. digit(x [,base])'
38
+ solution = ["To convert a string to an integer in Python 3, we use the built-in function int().",
39
+ "The int() function takes two arguments: the string to be converted and an optional base (default is 10, which is for decimal).",
40
+ "For example: int(\"123\", 10) converts the string \"123\" to the integer 123.",
41
+ "Looking at the options, we can see that the correct function is option E: int(x [,base]).",
42
+ "The answer is (E)."]
43
+ input_text = question + ' \n\n' + ' \n\n\n\n'.join(solution) + ' \n\n\n\n' # solution steps are separated by ' \n\n\n\n'
44
+ input_id = torch.tensor([tokenizer.encode(input_text)]).to(device)
45
+
46
+ with torch.no_grad():
47
+ logits = model(input_id).logits[:,:,candidate_tokens]
48
+ scores = logits.softmax(dim=-1)[:,:,1]
49
+ step_scores = scores[input_id == 23535]
50
+ step_probs = step_scores.tolist()
51
+ ```