

5.3s (20.6%)
   



Optimized with mem. efficient attention and compilation

-
   
16.4s (2.1%)
   
7.2s (-2.3%)
   
6.6s (18.6%)
   
4.1s (38.5%)
   


Runtimes for batch size 2



Configuration

P100

T4

A10

V100

A100




Original without xFormers

58.0s (-21.6%)
   
57.6s (-84.0%)
   
24.4s (-95.2%)
   
18.6s (-63.0%)
   
12.0s (-50.6%)
   


Original with xFormers

47.7s (0.0%)
   
31.3s (0.0%)