File size: 2,288 Bytes
c38b68d e16de00 ce3ca15 e16de00 ce3ca15 e16de00 ce3ca15 e16de00 ce3ca15 e16de00 ce3ca15 018602f e16de00 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
license: mit
language:
- en
base_model:
- black-forest-labs/FLUX.1-Canny-dev
---
# Introduction
This model is fine-tuned based on black-forest-labs/FLUX.1-canny-dev. \
The model enhances the performance when used with black-forest-labs/FLUX.1-Redux-dev.
# Examples
## shoe/clothe generate demo
## texture generate demo
## finetuned vs original
# Inference
```
import os
import torch
from diffusers import FluxPriorReduxPipeline, FluxPipeline, FluxControlPipeline, FluxTransformer2DModel
from diffusers.utils import load_image
from PIL import Image
pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-Redux-dev",
torch_dtype=torch.bfloat16).to("cuda")
pipe = FluxControlPipeline.from_pretrained(
"black-forest-labs/FLUX.1-Canny-dev",
torch_dtype=torch.bfloat16
).to("cuda")
transformer = FluxTransformer2DModel.from_pretrained(
"woshin/FLUX.1-Canny-dev-redux",
subfolder="transformer",
torch_dtype=torch.bfloat16
).to("cuda")
pipe.transformer = transformer
def inference_control_redux(style_image, control_image, save_image, size=(1024, 1024)):
style_image = load_image(style_image).resize(size)
control_image = load_image(control_image).resize(size)
pipe_prior_redux_out = pipe_prior_redux(style_image, return_dict=False)
prompt_embeds, pooled_prompt_embeds = pipe_prior_redux_out[0], pipe_prior_redux_out[1]
image = pipe(
control_image=control_image,
height=size[1],
width=size[0],
num_inference_steps=50,
guidance_scale=30.0,
prompt_embeds=prompt_embeds,
pooled_prompt_embeds=pooled_prompt_embeds,
).images[0]
dst = Image.new('RGB', (size[0] * 3, size[1]))
dst.paste(style_image, (0, 0))
dst.paste(control_image, (size[0], 0))
dst.paste(image, (size[0] * 2, 0))
dst.save(save_image)
style_image = "style.png" # download from this repo
control_image = "control.png"
save_image = "save.png"
inference_control_redux(style_image, control_image, save_image)
```
|