license: mit
base_model:
- deepseek-ai/DeepSeek-R1
Model Overview
- Model Architecture: DeepSeek-R1
- Input: Text
- Output: Text
- Supported Hardware Microarchitecture: AMD MI350/MI355
- ROCm: 7.0
- Operating System(s): Linux
- Inference Engine: SGLang
- Model Optimizer: AMD-Quark
- Weight quantization: OCP MXFP4, Static
- Activation quantization: OCP MXFP4, Dynamic
- KV cache: OCP FP8, Static
- Calibration Dataset: Pile
This model was built with deepseek-ai DeepSeek-R1 model by applying AMD-Quark for MXFP4 quantization.
Model Quantization
The model was quantized from deepseek-ai/DeepSeek-R1 using AMD-Quark. Both weights and activations were quantized to MXFP4 format, and the AutoSmoothQuant algorithm was applied to enhance accuracy.
Preprocessing requirement:
Before executing the quantization script below, the original FP8 model must first be dequantized to BFloat16. You can either perform the dequantization manually using this conversion script, or use the pre-converted BFloat16 model available at unsloth/DeepSeek-R1-BF16.
Quantization scripts:
cd Quark/examples/torch/language_modeling/llm_ptq/
python3 quantize_quark.py --model_dir $MODEL_DIR \
--quant_scheme w_mxfp4_a_mxfp4 \
--group_size 32 \
--kv_cache_dtype fp8 \
--num_calib_data 128 \
--exclude_layers "lm_head" \
--multi_device \
--model_export hf_format \
--output_dir amd/DeepSeek-R1-MXFP4
Deployment
Use with SGLang
This model can be deployed efficiently using the SGLang backend.
License
Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.