pablo
add diffusers fork
a63d2a4

A newer version of the Gradio SDK is available: 5.23.1

Upgrade



Diffusers

๐Ÿค— Diffusers๋Š” ์ด๋ฏธ์ง€, ์˜ค๋””์˜ค, ์‹ฌ์ง€์–ด ๋ถ„์ž์˜ 3D ๊ตฌ์กฐ๋ฅผ ์ƒ์„ฑํ•˜๊ธฐ ์œ„ํ•œ ์ตœ์ฒจ๋‹จ ์‚ฌ์ „ ํ›ˆ๋ จ๋œ diffusion ๋ชจ๋ธ์„ ์œ„ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์ž…๋‹ˆ๋‹ค. ๊ฐ„๋‹จํ•œ ์ถ”๋ก  ์†”๋ฃจ์…˜์„ ์ฐพ๊ณ  ์žˆ๋“ , ์ž์ฒด diffusion ๋ชจ๋ธ์„ ํ›ˆ๋ จํ•˜๊ณ  ์‹ถ๋“ , ๐Ÿค— Diffusers๋Š” ๋‘ ๊ฐ€์ง€ ๋ชจ๋‘๋ฅผ ์ง€์›ํ•˜๋Š” ๋ชจ๋“ˆ์‹ ํˆด๋ฐ•์Šค์ž…๋‹ˆ๋‹ค. ์ €ํฌ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋Š” ์„ฑ๋Šฅ๋ณด๋‹ค ์‚ฌ์šฉ์„ฑ, ๊ฐ„ํŽธํ•จ๋ณด๋‹ค ๋‹จ์ˆœํ•จ, ๊ทธ๋ฆฌ๊ณ  ์ถ”์ƒํ™”๋ณด๋‹ค ์‚ฌ์šฉ์ž ์ง€์ • ๊ฐ€๋Šฅ์„ฑ์— ์ค‘์ ์„ ๋‘๊ณ  ์„ค๊ณ„๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

์ด ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์—๋Š” ์„ธ ๊ฐ€์ง€ ์ฃผ์š” ๊ตฌ์„ฑ ์š”์†Œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค:

  • ๋ช‡ ์ค„์˜ ์ฝ”๋“œ๋งŒ์œผ๋กœ ์ถ”๋ก ํ•  ์ˆ˜ ์žˆ๋Š” ์ตœ์ฒจ๋‹จ diffusion ํŒŒ์ดํ”„๋ผ์ธ.
  • ์ƒ์„ฑ ์†๋„์™€ ํ’ˆ์งˆ ๊ฐ„์˜ ๊ท ํ˜•์„ ๋งž์ถ”๊ธฐ ์œ„ํ•ด ์ƒํ˜ธ๊ตํ™˜์ ์œผ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋…ธ์ด์ฆˆ ์Šค์ผ€์ค„๋Ÿฌ.
  • ๋นŒ๋”ฉ ๋ธ”๋ก์œผ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ณ  ์Šค์ผ€์ค„๋Ÿฌ์™€ ๊ฒฐํ•ฉํ•˜์—ฌ ์ž์ฒด์ ์ธ end-to-end diffusion ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ๋Š” ์‚ฌ์ „ ํ•™์Šต๋œ ๋ชจ๋ธ.
Tutorials

๊ฒฐ๊ณผ๋ฌผ์„ ์ƒ์„ฑํ•˜๊ณ , ๋‚˜๋งŒ์˜ diffusion ์‹œ์Šคํ…œ์„ ๊ตฌ์ถ•ํ•˜๊ณ , ํ™•์‚ฐ ๋ชจ๋ธ์„ ํ›ˆ๋ จํ•˜๋Š” ๋ฐ ํ•„์š”ํ•œ ๊ธฐ๋ณธ ๊ธฐ์ˆ ์„ ๋ฐฐ์›Œ๋ณด์„ธ์š”. ๐Ÿค— Diffusers๋ฅผ ์ฒ˜์Œ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ ์—ฌ๊ธฐ์—์„œ ์‹œ์ž‘ํ•˜๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค!

How-to guides

ํŒŒ์ดํ”„๋ผ์ธ, ๋ชจ๋ธ, ์Šค์ผ€์ค„๋Ÿฌ๋ฅผ ๋กœ๋“œํ•˜๋Š” ๋ฐ ๋„์›€์ด ๋˜๋Š” ์‹ค์šฉ์ ์ธ ๊ฐ€์ด๋“œ์ž…๋‹ˆ๋‹ค. ๋˜ํ•œ ํŠน์ • ์ž‘์—…์— ํŒŒ์ดํ”„๋ผ์ธ์„ ์‚ฌ์šฉํ•˜๊ณ , ์ถœ๋ ฅ ์ƒ์„ฑ ๋ฐฉ์‹์„ ์ œ์–ดํ•˜๊ณ , ์ถ”๋ก  ์†๋„์— ๋งž๊ฒŒ ์ตœ์ ํ™”ํ•˜๊ณ , ๋‹ค์–‘ํ•œ ํ•™์Šต ๊ธฐ๋ฒ•์„ ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•๋„ ๋ฐฐ์šธ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Conceptual guides

๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๊ฐ€ ์™œ ์ด๋Ÿฐ ๋ฐฉ์‹์œผ๋กœ ์„ค๊ณ„๋˜์—ˆ๋Š”์ง€ ์ดํ•ดํ•˜๊ณ , ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์ด์šฉ์— ๋Œ€ํ•œ ์œค๋ฆฌ์  ๊ฐ€์ด๋“œ๋ผ์ธ๊ณผ ์•ˆ์ „ ๊ตฌํ˜„์— ๋Œ€ํ•ด ์ž์„ธํžˆ ์•Œ์•„๋ณด์„ธ์š”.

Reference

๐Ÿค— Diffusers ํด๋ž˜์Šค ๋ฐ ๋ฉ”์„œ๋“œ์˜ ์ž‘๋™ ๋ฐฉ์‹์— ๋Œ€ํ•œ ๊ธฐ์ˆ  ์„ค๋ช….

Supported pipelines

Pipeline Paper/Repository Tasks
alt_diffusion AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities Image-to-Image Text-Guided Generation
audio_diffusion Audio Diffusion Unconditional Audio Generation
controlnet Adding Conditional Control to Text-to-Image Diffusion Models Image-to-Image Text-Guided Generation
cycle_diffusion Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance Image-to-Image Text-Guided Generation
dance_diffusion Dance Diffusion Unconditional Audio Generation
ddpm Denoising Diffusion Probabilistic Models Unconditional Image Generation
ddim Denoising Diffusion Implicit Models Unconditional Image Generation
if IF Image Generation
if_img2img IF Image-to-Image Generation
if_inpainting IF Image-to-Image Generation
latent_diffusion High-Resolution Image Synthesis with Latent Diffusion Models Text-to-Image Generation
latent_diffusion High-Resolution Image Synthesis with Latent Diffusion Models Super Resolution Image-to-Image
latent_diffusion_uncond High-Resolution Image Synthesis with Latent Diffusion Models Unconditional Image Generation
paint_by_example Paint by Example: Exemplar-based Image Editing with Diffusion Models Image-Guided Image Inpainting
pndm Pseudo Numerical Methods for Diffusion Models on Manifolds Unconditional Image Generation
score_sde_ve Score-Based Generative Modeling through Stochastic Differential Equations Unconditional Image Generation
score_sde_vp Score-Based Generative Modeling through Stochastic Differential Equations Unconditional Image Generation
semantic_stable_diffusion Semantic Guidance Text-Guided Generation
stable_diffusion_text2img Stable Diffusion Text-to-Image Generation
stable_diffusion_img2img Stable Diffusion Image-to-Image Text-Guided Generation
stable_diffusion_inpaint Stable Diffusion Text-Guided Image Inpainting
stable_diffusion_panorama MultiDiffusion Text-to-Panorama Generation
stable_diffusion_pix2pix InstructPix2Pix: Learning to Follow Image Editing Instructions Text-Guided Image Editing
stable_diffusion_pix2pix_zero Zero-shot Image-to-Image Translation Text-Guided Image Editing
stable_diffusion_attend_and_excite Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models Text-to-Image Generation
stable_diffusion_self_attention_guidance Improving Sample Quality of Diffusion Models Using Self-Attention Guidance Text-to-Image Generation Unconditional Image Generation
stable_diffusion_image_variation Stable Diffusion Image Variations Image-to-Image Generation
stable_diffusion_latent_upscale Stable Diffusion Latent Upscaler Text-Guided Super Resolution Image-to-Image
stable_diffusion_model_editing Editing Implicit Assumptions in Text-to-Image Diffusion Models Text-to-Image Model Editing
stable_diffusion_2 Stable Diffusion 2 Text-to-Image Generation
stable_diffusion_2 Stable Diffusion 2 Text-Guided Image Inpainting
stable_diffusion_2 Depth-Conditional Stable Diffusion Depth-to-Image Generation
stable_diffusion_2 Stable Diffusion 2 Text-Guided Super Resolution Image-to-Image
stable_diffusion_safe Safe Stable Diffusion Text-Guided Generation
stable_unclip Stable unCLIP Text-to-Image Generation
stable_unclip Stable unCLIP Image-to-Image Text-Guided Generation
stochastic_karras_ve Elucidating the Design Space of Diffusion-Based Generative Models Unconditional Image Generation
text_to_video_sd Modelscope's Text-to-video-synthesis Model in Open Domain Text-to-Video Generation
unclip Hierarchical Text-Conditional Image Generation with CLIP Latents(implementation by kakaobrain) Text-to-Image Generation
versatile_diffusion Versatile Diffusion: Text, Images and Variations All in One Diffusion Model Text-to-Image Generation
versatile_diffusion Versatile Diffusion: Text, Images and Variations All in One Diffusion Model Image Variations Generation
versatile_diffusion Versatile Diffusion: Text, Images and Variations All in One Diffusion Model Dual Image and Text Guided Generation
vq_diffusion Vector Quantized Diffusion Model for Text-to-Image Synthesis Text-to-Image Generation