lowing resources:
Enable xFormers memory efficient attention mechanism for faster speed and reduced memory consumption.
Learn how in PyTorch 2.0, torch.compile can yield 2-9% faster inference speed.
Many optimization techniques for inference are also included in this memory and speed guide, such as memory offloading.
