Thankyou!! this version works with vllm

#1
by 0-hero - opened

Thankyou!

Awesome!

Great model Migel! You've done it again!

Thank you guys! Enjoy!

migtissera changed discussion status to closed

How do you use this with vllm? Can you share a script?

migtissera changed discussion status to open

You will need to load the pt files. safetensors and bin for mixtral don't work with vllm as of now

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment