image = pipe(prompt, num_inference_steps=30).images[0] torch.compile offers different backends and modes. As we’re aiming for maximum inference speed, we opt for the inductor backend using the “max-autotune”. “max-autotune” uses CUDA graphs and optimizes the compilation graph specifically for latency. Specifying fullgraph to be True ensures that there are no graph breaks in the underlying model, ensuring the fullest potential of torch.compile. Using SDPA attention and compiling both the UNet and VAE reduces the latency from 3.31 seconds to 2.54 seconds.   Combine the projection matrices of attention Both the UNet and the VAE used in SDXL make use of Transformer-like blocks. A Transformer block consists of attention blocks and feed-forward blocks. In an attention block, the input is projected into three sub-spaces using three different projection matrices – Q, K, and V. In the naive implementation, these projections are performed separately on the input. But we can horizontally combine the projection matrices into a single matrix and perform the projection in one shot. This increases the size of the matmuls of the input projections and improves the impact of quantization (to be discussed next). Enabling this kind of computation in Diffusers just takes a single line of code:   Copied pipe.fuse_qkv_projections() It provides a minor boost from 2.54 seconds to 2.52 seconds.  Support for fuse_qkv_projections() is limited and experimental. As such, it’s not available for many non-SD pipelines such as Kandinsky. You can refer to this PR to get an idea about how to support this kind of computation.  Dynamic quantization Aapply dynamic int8 quantization to both the UNet and the VAE. This is because quantization adds additional conversion overhead to the model that is hopefully made up for by faster matmuls (dynamic quantization). If the matmuls are too small, these techniques may degrade performance. Through experimentation, we found that certain linear layers in the UNet and the VAE don’t benefit from dynamic int8 quantization. You can check out the full code for filtering those layers here (referred to as dynamic_quant_filter_fn below). You will leverage the ultra-lightweight pure PyTorch library torchao to use its user-friendly APIs for quantization. First, configure all the compiler tags:   Copied from diffusers import StableDiffusionXLPipeline