More Quantization options
#3
by
kkalyfo
- opened
Please provide Q8 quantization too!
Sure, we'll soon update the GGUF model with Q8 as well.
I have Q8 up here if you need it before they get to it: https://huggingface.co/bartowski/ilsp_Llama-Krikri-8B-Instruct-GGUF
Thanks! Is it with the updated weights?
should be yes, it was created after the most recent update to their relevant files
though that gives me an idea that i should probably include the latest commit hash that the quants are made from just for extra clarity 🤔
Thanks for your response!
We have updated the model with the last quants (q8, q6 and q3).
Great, thanks!