| # Flux.dev quantized versions | |
| ## Quantized FLUX Transformer with Hyper-SD LoRA | |
| This repository contains quantized FLUX transformer model merged with Hyper-SD a,d Alimama LoRA weights, optimized for efficient inference. | |
| ### Model Details | |
| - **Base Model**: FLUX.1-dev transformer from Black Forest Labs | |
| - **LoRA**: Hyper-SD from ByteDance and Alimama | |
| - **Quantization**: FP8 (e5m2 format) | |
| - **LoRA Scale**: 0.125 | |
| ### Technical Specifications | |
| #### Quantization | |
| - The model uses 8-bit floating-point (FP8) quantization with e5m2 format | |
| - Implemented using the `optimum.quanto` library | |
| - Weights are frozen after quantization for inference | |
| #### Architecture | |
| - Based on FluxTransformer2DModel | |
| - Includes merged LoRA weights from Hyper-SD | |
| - Optimized for 8-step inference | |
| ### Model Creation Process | |
| 1. **Base Model Loading** | |
| - Loads FLUX.1-dev transformer in bfloat16 format | |
| - Source: `black-forest-labs/FLUX.1-dev` | |
| 2. **Quantization** | |
| - Applies FP8 quantization using `qfloat8_e5m2` | |
| - Reduces model size while maintaining performance | |
| 3. **LoRA Integration** | |
| - Loads Hyper-SD LoRA weights | |
| - Merges with base model using 0.125 scale factor | |
| - Source: `ByteDance/Hyper-SD` | |
| 4. **Model Freezing** | |
| - Freezes weights for efficient inference | |
| - Saves as PyTorch model file | |
| ### Usage | |
| ```python | |
| import torch | |
| # Load the model | |
| model = torch.load('flux-fp8-hyper8-transformers-lora.pt') | |
| # Model is ready for inference | |
| # Use with appropriate input formatting and processing | |
| ``` | |
| ### Requirements | |
| - PyTorch | |
| - optimum.quanto | |
| - diffusers | |
| - huggingface_hub | |
| - safetensors | |
| ### References | |
| - FLUX.1-dev: [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) | |
| - Hyper-SD: [ByteDance/Hyper-SD](https://huggingface.co/ByteDance/Hyper-SD) | |
| ### License | |
| Please refer to the original FLUX.1-dev and Hyper-SD licenses for usage terms and conditions. | |
| ## Acknowledgments | |
| - [Black Forest Labs](https://huggingface.co/black-forest-labs) for the base FluxTransformer2DModel. | |
| - [ByteDance](https://huggingface.co/ByteDance) for the LoRA weights. | |
| - The developers of the `optimum.quanto` and `safetensors` libraries for their tools. | |
| ``` | |
| --- | |
| license: other | |
| license_name: flux-dev | |
| license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md | |
| --- | |