How did you quantize Phi-2?

#1
by Jenish-23 - opened

I am interested in learning quantization of microsoft's Phi models and would like to know how you did it. I tried, got errors and gave up half-way through.
Also, is it possible to quantize in AWQ?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment