neuralmagic
/

Mixtral-8x7B-Instruct-v0.1-FP8

Mixture of Experts

compressed-tensors

Model card Files Files and versions Community

Mixtral-8x7B-Instruct-v0.1-FP8

File size: 158 Bytes

bb81080

DEFAULT_stage:
  DEFAULT_modifiers:
    QuantizationModifier:
      ignore: [lm_head, 're:.*block_sparse_moe.gate']
      targets: [Linear]
      scheme: FP8