ntion
xFormers
PyTorch2.0 SDPA
SDPA + torch.compile
Speed over xformers (%)
Speed over vanilla (%)
A100
4
16.56
12.42
12.2
11.84
4.67
28.50
A100
10
OOM
29.93
29.44
28.5
4.78

A100
16

47.08
46.27
44.8
4.84

A100
32

92.89
91.34
88.35
4.89

A100
64

185.3
182.71
176.48
4.76









A10
1
10.59
8.81
7.51
7.35
16.57
30.59
A10
4
34.77
27.63
22.77
22.07
20.12
36.53
A10
8

56.19
43.53
43.86
21.94

A10
16

116.49
88.56
86.64
25.62

A10
32

221.95
175