graph for an A100 with PyTorch 2.0 and torch.compile:  In the following tables, we report our findings in terms of the number of iterations/second.  A100 (batch size: 1) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 21.66 23.13 44.03 49.74 SD - img2img 21.81 22.40 43.92 46.32 SD - inpaint 22.24 23.23 43.76 49.25 SD - controlnet 15.02 15.82 32.13 36.08 IF 20.21 / 13.84 / 24.00 20.12 / 13.70 / 24.03 ❌ 97.34 / 27.23 / 111.66 SDXL - txt2img 8.64 9.9 - -  A100 (batch size: 4) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 11.6 13.12 14.62 17.27 SD - img2img 11.47 13.06 14.66 17.25 SD - inpaint 11.67 13.31 14.88 17.48 SD - controlnet 8.28 9.38 10.51 12.41 IF 25.02 18.04 ❌ 48.47 SDXL - txt2img 2.44 2.74 - -  A100 (batch size: 16) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 3.04 3.6 3.83 4.68 SD - img2img 2.98 3.58 3.83 4.67 SD - inpaint 3.04 3.66 3.9 4.76 SD - controlnet 2.15 2.58 2.74 3.35 IF 8.78 9.82 ❌ 16.77 SDXL - txt2img 0.64 0.72 - -  V100 (batch size: 1) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 18.99 19.14 20.95 22.17 SD - img2img 18.56 19.18 20.95 22.11 SD - inpaint 19.14 19.06 21.08 22.20 SD - controlnet 13.48 13.93 15.18 15.88 IF 20.01 / 9.08 / 23.34 19.79 / 8.98 / 24.10 ❌ 55.75 / 11.57 / 57.67  V100 (batch size: 4) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 5.96 5.89 6.83 6.86 SD - img2img 5.90 5.91 6.81 6.82 SD - inpaint 5.99 6.03 6.93 6.95 SD - controlnet 4.26 4.29 4.92 4.93 IF 15.41 14.76 ❌ 22.95  V100 (batch size: 16) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 1.66 1.66 1.92 1.90 SD - img2img 1.65 1.65 1.91 1.89 SD - inpaint 1.69 1.69 1.95 1.93 SD - controlnet 1.19 1.19 OOM after warmup 1.36 IF 5.43 5.29 ❌ 7.06  T4 (batch size: 1) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 6.9 6.95 7.3 7.56 SD - img2img 6.84 6.99 7.04 7.55 SD - inpaint 6.91 6.7 7.01 7.37 SD - controlnet 4.89 4.86 5.35 5.48 IF 17.42 / 2.47 / 18.52 16.96 / 2.45 / 18.69 ❌ 24.63 / 2.47 / 23.39 SDXL - txt2img 1.15 1.16 - -  T4 (batch size: 4) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 1.79 1.79 2.03 1.99 SD - img2img 1.77 1.77 2.05 2.04 SD - inpaint 1.81 1.82 2.09 2.09 SD - controlnet 1.34 1.27 1.47 1.46 IF 5.79 5.61 ❌ 7.39 SDXL - txt2img 0.288 0.289 - -  T4 (batch size: 16) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 2.34s 2.30s OOM after 2nd iteration 1.99s SD - img2img 2.35s 2.31s OOM after warmup 2.00s SD - inpaint 2.30s 2.26s OOM after 2nd iteration 1.95s SD - controlnet OOM after 2nd iteration OOM after 2nd iteration OOM after warmup OOM after warmup IF * 1.44 1.44 ❌ 1.94 SDXL - txt2img OOM OOM - -  RTX 3090 (batch size: 1) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 22.56 22.84 23.84 25.69 SD - img2img 22.25 22.61 24.1 25.83 SD - inpaint 22.22 22.54 24.26 26.02 SD - controlnet 16.03 16.33 17.38 18.56 IF 27.08 / 9.07 / 31.23 26.75 / 8.92 / 31.47 ❌ 68.08 / 11.16 / 65.29  RTX 3090 (batch size: 4) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 6.46 6.35 7.29 7.3 SD - img2img 6.33 6.27 7.31 7.26 SD - inpaint 6.47 6.4 7.44 7.39 SD - controlnet 4.59 4.54 5.27 5.26 IF 16.81 16.62 ❌ 21.57  RTX 3090 (batch size: 16) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 1.7 1.69 1.93 1.91 SD - img2img 1.68 1.67 1.93 1.9 SD - inpaint 1.72 1.71 1.97 1.94 SD - controlnet 1.23 1.22 1.4 1.38 IF 5.01 5.00 ❌ 6.33  RTX 4090 (batch size: 1) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 40.5 41.89 44.65 49.81 SD - img2img 40.39 41.95 44.46 49.8 SD - inpaint 40.51 41.88 44.58 49.72 SD - controlnet 29.27 30.29 32.26 36.03 IF 69.71 / 18.78 / 85.49 69.13 / 18.80 / 85.56 ❌ 124.60 / 26.37 / 138.79 SDXL - txt2img 6.8 8.18 - -  RTX 4090 (batch size: 4) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 12.62 12.84 15.32 15.59 SD - img2img 12.61 12,.79 15.35 15.66 SD - inpaint 12.65 12.81 15.3 15.58 SD - controlnet 9.1 9.25 11.03 11.22 IF 31.88 31.14 ❌ 43.92 SDXL - txt2img 2.19 2.35 - -  RTX 4090 (batch size: 16) Pipeline torch 2.0 - no compile torch nightly - no compile torch 2.0 - compile torch nightly - compile SD - txt2img 3.17 3.2 3.84 3.85 SD - img2img 3.16 3.2 3.84 3.85 SD - inpaint 3.17 3.2 3.85 3.85 SD - controlnet 2.23 2.3 2.7 2.75 IF 9.26 9.2 ❌ 13.31 SDXL - txt2img 0.52 0.53 - -  Notes Follow this PR for more details on the environment used for conducting the benchmarks. For the DeepFloyd IF pipeline where batch sizes > 1, we only used a batch size of > 1 in the first IF pipeline for text-to-image generation and NOT for upscaling. That means the two upscaling pipelines received a batch size of 1. Thanks to Horace He from the PyTorch team for their support in improving our support of torch.compile() in Diffusers.