not able to use this model (tencent/Hunyuan-7B-Instruct) via vllm
#5
by
aditya-shinde
- opened
llm = LLM(model=MODEL_NAME, download_dir="./model_cache")
ValidationError: 1 validation error for ModelConfig
Value error, The checkpoint you are trying to load has model type hunyuan_v1_dense but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
using transformers==4.54.1 and vllm==0.10.0
It says clearly in the instructions that you need to 'pip install' their own modified version of transformers for now.
I did that, and it works fine on vLLM.
Good luck with it! And remember to close this issue.
I can confirm I missed that part of the README, and installing their version of transformers fixes the issue for me, thanks!