trithemius/flux-fp8-transformers

Flux.dev quantized versions

Quantized FLUX Transformer with Hyper-SD LoRA

This repository contains quantized FLUX transformer model merged with Hyper-SD a,d Alimama LoRA weights, optimized for efficient inference.

Model Details

• Base Model: FLUX.1-dev transformer from Black Forest Labs
• LoRA: Hyper-SD from ByteDance and Alimama
• Quantization: FP8 (e5m2 format)
• LoRA Scale: 0.125

Technical Specifications

Quantization

• The model uses 8-bit floating-point (FP8) quantization with e5m2 format
• Implemented using the optimum.quanto library
• Weights are frozen after quantization for inference

Architecture

• Based on FluxTransformer2DModel
• Includes merged LoRA weights from Hyper-SD
• Optimized for 8-step inference

Model Creation Process

1. Base Model Loading

   • Loads FLUX.1-dev transformer in bfloat16 format
   • Source: black-forest-labs/FLUX.1-dev

2. Quantization

   • Applies FP8 quantization using qfloat8_e5m2
   • Reduces model size while maintaining performance

3. LoRA Integration

   • Loads Hyper-SD LoRA weights
   • Merges with base model using 0.125 scale factor
   • Source: ByteDance/Hyper-SD

4. Model Freezing

   • Freezes weights for efficient inference
   • Saves as PyTorch model file

Usage

import torch

# Load the model
model = torch.load('flux-fp8-hyper8-transformers-lora.pt')

# Model is ready for inference
# Use with appropriate input formatting and processing

Requirements

• PyTorch
• optimum.quanto
• diffusers
• huggingface_hub
• safetensors

References

• FLUX.1-dev: black-forest-labs/FLUX.1-dev
• Hyper-SD: ByteDance/Hyper-SD

License

Please refer to the original FLUX.1-dev and Hyper-SD licenses for usage terms and conditions.

Acknowledgments

• Black Forest Labs for the base FluxTransformer2DModel.
• ByteDance for the LoRA weights.
• The developers of the optimum.quanto and safetensors libraries for their tools.

---
license: other
license_name: flux-dev
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
---