EXL2 quants of CodeLlama2-34B-instruct
2.70 bits per weight
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight (currently broken, new version is uploading...)
4.65 bits per weight
6.00 bits per weight (currently broken, new version is uploading...)