02.78
100.89
9.94







4090
1
4.47
3.98
1.28
1.21
69.60
4090
4
10.48
8.37
3.76
3.56
57.47
4090
8
14.33
10.22
7.43
6.99
31.60
4090
16

17.07
14.98
14.58
14.59
4090
32 (1)

39.03
30.18
29.49
24.44
4090
64 (1)

77.29
61.34
59.96
22.42

FP32 benchmark
	
The table below shows the benchmark results for inference using fp32. In this case, torch.nn.functional.scaled_dot_product_attention is faster than xFormers on all the GPUs we tested.
Using torch.