perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	1815	18.3139	12.49	50.82	0.29
IQ1_M	1947	14.7056	13.40	63.29	0.24
IQ2_XXS	2167	12.3126	14.91	75.59	0.20
IQ2_XS	2354	11.0302	16.20	84.38	0.17
IQ2_S	2475	10.7236	17.03	86.79	0.17
IQ2_M	2651	10.3509	18.24	89.92	0.16
Q2_K_S	2702	10.6991	18.59	86.99	0.17
Q2_K	2876	10.3696	19.79	89.75	0.16
IQ3_XXS	2970	9.8351	20.44	94.63	0.15
IQ3_XS	3191	9.5737	21.96	97.21	0.15
Q3_K_S	3330	9.7610	22.92	95.35	0.15
IQ3_S	3337	9.5520	22.96	97.44	0.15
IQ3_M	3408	9.5248	23.45	97.71	0.15
Q3_K_M	3631	9.6151	24.99	96.80	0.15
Q3_K_L	3899	9.5675	26.83	97.28	0.15
IQ4_XS	4023	9.3942	27.68	99.07	0.15
IQ4_NL	4232	9.3991	29.12	99.02	0.15
Q4_0	4238	9.4483	29.17	98.50	0.15
Q4_K_S	4251	9.3869	29.25	99.15	0.15
Q4_K_M	4466	9.3699	30.73	99.33	0.15
Q4_1	4647	9.3797	31.98	99.22	0.15
Q5_K_S	5068	9.3144	34.88	99.92	0.14
Q5_0	5081	9.3419	34.97	99.63	0.14
Q5_K_M	5192	9.3152	35.73	99.91	0.14
Q5_1	5490	9.3111	37.78	99.96	0.14
Q6_K	5964	9.3168	41.04	99.89	0.14
Q8_0	7723	9.3061	53.15	100.01	0.14
F16	14531	9.3071	100	100	0.14