missing quantize_config.json file

by aromo17 - opened Apr 11, 2024

Apr 11, 2024

Hello,

I am trying to evaluate the model in lm-harness. However when I tried to do it i get following error.
FileNotFoundError: [Errno 2] No such file or directory: './Qwen1.5-72B-Chat-GPTQ-Int4/quantize_config.json

I think I would need the quantize_config.json file

jklj077

Qwen org Apr 18, 2024

I think you have set the autogptq argument, which will load the model using AutoGPTQForCausalLM while this model should be loaded using AutoModelForCausalLM. Please first try removing the related arguments.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment