More Quantization options

#3
by kkalyfo - opened

Please provide Q8 quantization too!

Institute for Language and Speech Processing org

Sure, we'll soon update the GGUF model with Q8 as well.

I have Q8 up here if you need it before they get to it: https://huggingface.co/bartowski/ilsp_Llama-Krikri-8B-Instruct-GGUF

Thanks! Is it with the updated weights?

should be yes, it was created after the most recent update to their relevant files

though that gives me an idea that i should probably include the latest commit hash that the quants are made from just for extra clarity 🤔

Thanks for your response!

Institute for Language and Speech Processing org

We have updated the model with the last quants (q8, q6 and q3).

Great, thanks!

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment