5.0bpw quants
#1
by
bullerwins
- opened
Hi!
Are you going to upload a 5.0?, I think it could fit in a 4x3090/4090 system at low context.
Thanks!
Yes! It would be much appreciated!
I added 4.75bpw and 5.0bpw versions just now.
Hi!
Are you going to upload a 5.0?, I think it could fit in a 4x3090/4090 system at low context.
Thanks!
If I read your post correctly, you have 4x3090 and 1x4090? Why not use the 6.0bpw quant? That should fit in 120GB VRAM.