perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	416	65.5925	14.10	18.53	1.27
IQ1_M	442	37.5490	14.98	32.37	0.67
IQ2_XXS	487	28.2797	16.51	42.98	0.52
IQ2_XS	524	19.7773	17.76	61.45	0.34
IQ2_S	537	18.3488	18.20	66.24	0.31
IQ2_M	573	16.4023	19.42	74.10	0.28
Q2_K_S	610	16.9464	20.68	71.72	0.28
IQ3_XXS	637	14.0319	21.59	86.61	0.23
Q2_K	644	15.4455	21.83	78.69	0.25
IQ3_XS	697	13.2295	23.63	91.87	0.21
Q3_K_S	725	13.6908	24.58	88.77	0.22
IQ3_S	727	13.0527	24.64	93.11	0.21
IQ3_M	740	12.9347	25.08	93.96	0.21
Q3_K_M	785	13.1079	26.61	92.72	0.21
Q3_K_L	839	12.9958	28.44	93.52	0.21
IQ4_XS	854	12.5238	28.95	97.04	0.20
IQ4_NL	892	12.5165	30.24	97.10	0.20
Q4_0	894	12.6002	30.31	96.46	0.20
Q4_K_S	896	12.4550	30.37	97.58	0.20
Q4_K_M	940	12.4048	31.86	97.97	0.20
Q4_1	969	12.4660	32.85	97.49	0.20
Q5_K_S	1047	12.2279	35.49	99.39	0.20
Q5_0	1050	12.2580	35.59	99.15	0.20
Q5_K_M	1072	12.2216	36.34	99.44	0.20
Q5_1	1125	12.2391	38.13	99.30	0.20
Q6_K	1213	12.1951	41.12	99.66	0.20
Q8_0	1570	12.1583	53.22	99.96	0.20
F16	2950	12.1537	100	100	0.20