ExLlamaV2 quantizations of: qingy2019 - NaturalLM
Quantizations (6hb)
8.0bpw
7.5bpw
7.0bpw
6.5bpw
6.0bpw
5.5bpw
5.0bpw
4.5bpw
4.25bpw
4.0bpw
3.75bpw
3.5bpw
3.0bpw
2.75bpw
2.5bpw
2.25bpw
2.0bpw
If you need a specific model quantization or a particular bits per weight, please let me know. I’m happy to help quantize lesser known models.
If you have any suggestions for improvements or feedback, feel free to reach out. Your input is greatly appreciated and helps me make quantizations better for everyone.
Special thanks to turboderp for developing the tools that made these quantizations possible. Your contributions are greatly appreciated!
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.
Model tree for TheMelonGod/NaturalLM-exl2
Base model
qingy2024/NaturalLM