PR #9319: Avoid rewriting tiny matrices with cublas or triton
PR #9319: Avoid rewriting tiny matrices with cublas or triton

Imported from GitHub PR https://github.com/openxla/xla/pull/9319

This disables triton/cublas rewriting for matrix multiplication instructions that are too small to benefit from cublas/triton.

In particular, this will avoid triton/cublas if the sum of the sizes of the matrices is <100. This threshold is a conservative estimate and has been measured to be always beneficial (up to generally several times faster) on V100 and H100 GPUs.
Copybara import of the project:

--
b12ff5ade1de6b0243e06998168c712f78c1db5c by Jaroslav Sevcik <jsevcik@nvidia.com>:

Avoid rewriting tiny matrices with cublas or triton

--
024d7f9eef4fec9239a62f4bddad7bd5d0c69d29 by Jaroslav Sevcik <jsevcik@nvidia.com>:

Fix copy-pasta bug

--
f54f9043a77da60e239614c866aca9a21b5aa65f by Jaroslav Sevcik <jsevcik@nvidia.com>:

Add tests with testing flag

--
fb689cc164155fe36013a8f5739035b5f77f914c by Jaroslav Sevcik <jsevcik@nvidia.com>:

Address reviewer comments

--
a777fab3cf1ce811222e4c4f95f3ee1f717fee4e by Jaroslav Sevcik <jsevcik@nvidia.com>:

Test rename

--
bd64e289892b3115e01dffde9bcefee118421bb0 by Jaroslav Sevcik <jsevcik@nvidia.com>:

Only allow float dots to reach the backend

--
07bd1ad038ab517fd75673da01b24dbc8773ed3e by Jaroslav Sevcik <jsevcik@nvidia.com>:

Add background info into a comment

--
0d18f84f1c93dbc2de5614407b9b1092c3b36986 by Jaroslav Sevcik <jsevcik@nvidia.com>:

clang format

--
23ccff6698efafe1fc384b99d706158ee68b075f by Jaroslav Sevcik <jsevcik@nvidia.com>:

Also update gemm_broadcast_folding_rewrite test

--
690bb0e1d8b5894f64134d68a031d387399c638f by Jaroslav Sevcik <jsevcik@nvidia.com>:

Rename and test

Merging this change closes #9319

FUTURE_COPYBARA_INTEGRATE_REVIEW=https://github.com/openxla/xla/pull/9319 from jaro-sevcik:no-cublas-or-triton-for-small-matmuls 690bb0e1d8b5894f64134d68a031d387399c638f
