 horizontally combine the projection matrices into a single matrix and perform the projection in one shot. This increases the size of the matmuls of the input projections and improves the impact of quantization (to be discussed next). Enabling this kind of computation in Diffusers just takes a single line of code:   Copied pipe.fuse_qkv_projections() It provides a minor boost from 2.54 seconds to 2.52 seconds.  Support for fuse_qkv_projections(