you can install and use xFormers’s implementation of memory-efficient attention. Both options reduce memory usage and accelerate inference. You can also offload the model to the CPU to save even more memory:   Copied + pipeline.enable_xformers_memory_efficient_attention()
+ pipeline.enable_model_cpu_offload() To speed-up your inference code even more, use torch_compile. You should wrap torch.compile around the most intensive component in the pi