vllm reply garbled
#29
by
SongXiaoMao
- opened
vllm can be started correctly, but the answers are all repeated !!!!!!!!!!!!!!!!!!!!!
Model behavior. It can easily into a thinking loop...
Any question you enter will be output!!!!!!
I've redownloaded the HF model and updated vllm to the latest version, 0.6.5, and it's working perfectly now. Thank you!
SongXiaoMao
changed discussion status to
closed