Finally, you try and install xFormers to reduce your memory footprint with xFormers memory-efficient attention. Once you have xFormers installed, add the --enable_xformers_memory_efficient_attention argument to the training script. xFormers is not supported for Flax.