mobiuslabsgmbh

/

Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bit-metaoffload-HQQ

like 15

Follow

Mobius Labs GmbH 93

https://github.com/huggingface/transformers/pull/29637

how to use device_map="auto"

it run in kagel
32vram
32 ram

not run in colab t4
16vram
12 ram

thank you
mobiuslabsgmbh