 includes an optimized and memory-efficient attention implementation through the torch.nn.functional.scaled_dot_product_attention function, which automatically enables several optimizations depending on the inputs and the GPU type. This is similar to the memory_efficient_attention from xFormers, but built natively into PyTorch. 
These optimizations will be enabled by default in Diffusers if PyTorch 2.0 is installed and if torch.nn.functional.sc