rsions of PyTorch 2.0. The tables below summarize the results we got.
Please refer to our featured blog post in the PyTorch site for more details.

FP16 benchmark
	
The table below shows the benchmark results for inference using fp16. As we can see, torch.nn.functional.scaled_dot_product_attention is as fast as xFormers (sometimes slightly faster/slower) on all the GPUs we tested.
And using torch.compile gives further speed-up of up of 10% over