wanghaofan commited on
Commit
6897762
·
verified ·
1 Parent(s): ee7b075

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -86,6 +86,44 @@ image = pipe(
86
  ).images[0]
87
  ```
88
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
  # Recommended Parameters
90
  You can adjust controlnet_conditioning_scale and control_guidance_end for stronger control and better detail preservation. For better stability, we suggest to use multi-conditions.
91
  - Canny: use cv2.Canny, controlnet_conditioning_scale=0.7, control_guidance_end=0.8.
 
86
  ).images[0]
87
  ```
88
 
89
+ # Multi-Inference
90
+ ```python
91
+ import torch
92
+ from diffusers.utils import load_image
93
+
94
+ # https://github.com/huggingface/diffusers/pull/11350, after merging, you can directly import from diffusers
95
+ # from diffusers import FluxControlNetPipeline, FluxControlNetModel
96
+
97
+ # use local files for this moment
98
+ from pipeline_flux_controlnet import FluxControlNetPipeline
99
+ from controlnet_flux import FluxControlNetModel
100
+
101
+ base_model = 'black-forest-labs/FLUX.1-dev'
102
+ controlnet_model_union = 'Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro-2.0'
103
+
104
+ controlnet = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
105
+ pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=[controlnet], torch_dtype=torch.bfloat16) # use [] to enable multi-CNs
106
+ pipe.to("cuda")
107
+
108
+ # replace with other conds
109
+ control_image = load_image("./conds/canny.png")
110
+ width, height = control_image.size
111
+
112
+ prompt = "A young girl stands gracefully at the edge of a serene beach, her long, flowing hair gently tousled by the sea breeze. She wears a soft, pastel-colored dress that complements the tranquil blues and greens of the coastal scenery. The golden hues of the setting sun cast a warm glow on her face, highlighting her serene expression. The background features a vast, azure ocean with gentle waves lapping at the shore, surrounded by distant cliffs and a clear, cloudless sky. The composition emphasizes the girl's serene presence amidst the natural beauty, with a balanced blend of warm and cool tones."
113
+
114
+ image = pipe(
115
+ prompt,
116
+ control_image=[control_image, control_image], # try with different conds such as canny&depth, pose&depth
117
+ width=width,
118
+ height=height,
119
+ controlnet_conditioning_scale=[0.35, 0.35],
120
+ control_guidance_end=[0.8, 0.8],
121
+ num_inference_steps=30,
122
+ guidance_scale=3.5,
123
+ generator=torch.Generator(device="cuda").manual_seed(42),
124
+ ).images[0]
125
+ ```
126
+
127
  # Recommended Parameters
128
  You can adjust controlnet_conditioning_scale and control_guidance_end for stronger control and better detail preservation. For better stability, we suggest to use multi-conditions.
129
  - Canny: use cv2.Canny, controlnet_conditioning_scale=0.7, control_guidance_end=0.8.