Spaces:

ggml-org
/

gguf-my-repo

Running on A10G

Arm optimized quants

#113

by SaisExperiments - opened Aug 19, 2024

Aug 19, 2024

Is it possible to have the
Q4_0_4_4
Q4_0_4_8
Q4_0_8_8
formats added?

ggml.ai org Aug 28, 2024

•

Thanks for the feature request; let's wait a bit to see if we get more requests for this.

reach-vb changed discussion status to closed Aug 28, 2024

reach-vb changed discussion status to open Aug 28, 2024

noNyve

19 days ago

Afaik there is no support for those quants anymore... Instead there is something called online repacking.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment