vllm deploy cannot run

by pty819 - opened Feb 5

pty819

Feb 5

Hi
after pull this to my disk, I tried "uv run vllm serve /data/ollama/qwen32R1"
and then it reports error, said that

assert param_data.shape == loaded_weight.shape
Assertion Error

why this happen? how can I fix that?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment