The new X86 quantization backend, which utilizes FBGEMM and oneDNN kernel libraries, replaces FBGEMM as the default quantization backend for x86 CPU platforms and offers improved int8 inference performance compared to the original FBGEMM backend, leveraging the strengths of both libraries, with 1.3X – 2X inference performance speedup measured on 40+ deep learning models. The new backend is functionally compatible with the original FBGEMM backend.
Table: Geomean Speedup of X86 Quantization Backend vs. FBGEMM Backend




1 core/instance
   
2 cores/instance
   
4 cores/instance
   
1 socket (32 cores)/instance
   


Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
   
1.76X
   
1.80X
   
2.04X
   
1.34X
   

