image = pipe(prompt, num_inference_steps=30).images[0] torch.compile offers different backends and modes. For maximum inference speed, use “max-autotune” for the inductor backend. “max-autotune” uses CUDA graphs and optimizes the compilation graph specifically for latency. CUDA graphs greatly reduces the overhead of launching GPU operations by using a mechanism to launch multiple GPU operations through a single CPU operation. Using SDPA attention and compiling both the UNet and VAE cuts the latency from 3.31 seconds to 2.54 seconds.   Prevent graph breaks Specifying fullgraph=True ensures there are no graph breaks in the underlying model to take full advantage of torch.compile without any performance degradation. For the UNet and VAE, this means changing how you access the return variables.   Copied - latents = unet(