toolbox for optimizing Transformers, and it includes a memory-efficient attention mechanism that is used in 🧨 Diffusers. You’ll need to install xFormers and then add the following argument to your training script:


	Copied
  --enable_xformers_memory_efficient_attention
xFormers is not available in Flax.

Set gradients to none
	
Another way you can lower your memory footprint is to set the gradients to None instead of zero. However, this may ch