+)[0]  Remove GPU sync after compilation During the iterative reverse diffusion process, the step() function is called on the scheduler each time after the denoiser predicts the less noisy latent embeddings. Inside step(), the sigmas variable is indexed which when placed on the GPU, causes a communication sync between the CPU and GPU. This introduces latency and it becomes more evident when the denoiser has already been compiled. But if the sigmas array always stays on the CPU, the CPU and GPU sync doesn’t occur and you don’t get any latency. In general, any CPU and GPU communication sync should be none or be kept to a bare minimum because it can impact inference latency.  Combine the attention block’s projection matrices The UNet and VAE in SDXL use Transformer-like blocks which consists of attention blocks and feed-forward blocks. In an attention block, the input is projected into three sub-spaces using three different projection matrices – Q, K, and V. These projections are performed separately on the input. But we can horizontally combine the projection matrices into a single matrix and perform the projection in one step. This increases the size of the matrix multiplications of the input projections and improves the impact of quantization. You can combine the projection matrices with just a single line of code:   Copied pipe.fuse_qkv_projections() This provides a minor improvement from 2.54 seconds to 2.52 seconds.  Support for fuse_qkv_projections() is limited and experimental. It’s not available for many non-Stable Diffusion pipelines such as Kandinsky. You can refer to this PR to get an idea about how to enable this for the other pipelines.  Dynamic quantization You can also use the ultra-lightweight PyTorch quantization library, torchao (commit SHA 54bcd5a10d0abbe7b0c045052029257099f83fd9), to apply dynamic int8 quantization to the UNet and VAE. Quantization adds additional conversion overhead to the model that is hopefully made up for by faster matmuls (dynamic quantization). If the matmuls are too small, these techniques may degrade performance. First, configure all the compiler tags:   Copied from diffusers import StableDiffusionXLPipeline