91
7.24







4090
4
10.48
8.37
8.32
8.01
4.30
4090
8
14.33
10.22
10.42
9.78
4.31
4090
16

17.07
17.46
17.15
-0.47
4090
32 (1)

39.03
39.86
37.97
2.72
4090
64 (1)

77.29
79.44
77.67
-0.49

FP32 benchmark
	
The table below shows the benchmark results for inference using fp32. In this case, torch.nn.functional.scaled_dot_product_attention is faster than xFormers on all the GPUs we tested.
Using torch.compile in addition to the accelerated transf