Update README.md
Browse files
README.md
CHANGED
|
@@ -18,22 +18,26 @@ base_model: black-forest-labs/FLUX.1-dev
|
|
| 18 |
<img src="./images/image_union.png" width = "1000" />
|
| 19 |
|
| 20 |
|
| 21 |
-
##
|
| 22 |
-
|
| 23 |
-
Until the next Diffusers pypi release, please install Diffusers from source and use [this PR(update 2024.08.20)](https://github.com/huggingface/diffusers/pull/9175) to be able to use.
|
| 24 |
-
TODO: change when new version.
|
| 25 |
|
|
|
|
|
|
|
|
|
|
| 26 |
Before merging into the official main branch of diffusers, you can use this [diffusers_flux](https://github.com/instantX-research/diffusers_flux).
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
|
| 30 |
## Checkpoint
|
| 31 |
|
| 32 |
The training of union controlnet requires a significant amount of computational power.
|
| 33 |
-
The current release is
|
| 34 |
-
The
|
| 35 |
We have conducted ablation studies that have demonstrated the validity of the code.
|
| 36 |
-
The open-source release of the
|
| 37 |
it is common to encounter bad cases (please accept my apologies).
|
| 38 |
It is worth noting that we have found that even a fully trained Union model may not perform as well as specialized models, such as pose control.
|
| 39 |
However, as training progresses, the performance of the Union model will continue to approach that of specialized models.
|
|
@@ -63,7 +67,7 @@ from diffusers.models.controlnet_flux import FluxControlNetModel
|
|
| 63 |
|
| 64 |
# load
|
| 65 |
base_model = 'black-forest-labs/FLUX.1-dev'
|
| 66 |
-
controlnet_model = 'InstantX/FLUX.1-dev-Controlnet-Union
|
| 67 |
controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
|
| 68 |
pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=controlnet, torch_dtype=torch.bfloat16)
|
| 69 |
pipe.to("cuda")
|
|
|
|
| 18 |
<img src="./images/image_union.png" width = "1000" />
|
| 19 |
|
| 20 |
|
| 21 |
+
## Release
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
- [2024/08/20] 🔥 Release the first beta version.
|
| 24 |
+
Until the next Diffusers pypi release,
|
| 25 |
+
please install Diffusers from source and use [this PR](https://github.com/huggingface/diffusers/pull/9175) to be able to use.
|
| 26 |
Before merging into the official main branch of diffusers, you can use this [diffusers_flux](https://github.com/instantX-research/diffusers_flux).
|
| 27 |
|
| 28 |
+
- [2024/08/14] Release the alpha version.
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
|
| 32 |
|
| 33 |
|
| 34 |
## Checkpoint
|
| 35 |
|
| 36 |
The training of union controlnet requires a significant amount of computational power.
|
| 37 |
+
The current release is the first beta version checkpoint that maybe not been fully trained.
|
| 38 |
+
The fully trainedbeta version is in the training process.
|
| 39 |
We have conducted ablation studies that have demonstrated the validity of the code.
|
| 40 |
+
The open-source release of the first beta version is solely to facilitate the rapid growth of the open-source community and the Flux ecosystem;
|
| 41 |
it is common to encounter bad cases (please accept my apologies).
|
| 42 |
It is worth noting that we have found that even a fully trained Union model may not perform as well as specialized models, such as pose control.
|
| 43 |
However, as training progresses, the performance of the Union model will continue to approach that of specialized models.
|
|
|
|
| 67 |
|
| 68 |
# load
|
| 69 |
base_model = 'black-forest-labs/FLUX.1-dev'
|
| 70 |
+
controlnet_model = 'InstantX/FLUX.1-dev-Controlnet-Union'
|
| 71 |
controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
|
| 72 |
pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=controlnet, torch_dtype=torch.bfloat16)
|
| 73 |
pipe.to("cuda")
|