PR #13254: [NVIDIA] Support pipelining collectives in backward direction with loop invariant instructions in chain
PR #13254: [NVIDIA] Support pipelining collectives in backward direction with loop invariant instructions in chain

Imported from GitHub PR https://github.com/openxla/xla/pull/13254

In some quantization patterns like:
```
                constant
                      |
                broadcast
                  /       \
                /           \
           clamp         clamp
            /                  \
         dot            all-gather
```
The all-gathers cannot be pipelined due to the shared operand broadcast having multiple users and not all users are in all-gather's operand chain. The backward pipelining restricts this situation to avoiding exploding data dependencies across loop iterations.
An exception can be made if such operand chain only contains loop invariant instructions in this case: broadcast and constant.
This pr introduces a config to collective pipeliner backward direction to allow loop invariant instructions to be in the operand chain.
Copybara import of the project:

--
eeea881d084934214b6e2e34319cb3ff4fed9c1e by TJ <tjx@nvidia.com>:

Support pipelining collectives in backward direction with loop invariant instructions in chain

--
977e713643ae08337ba61ab15fe7812f95078900 by TJ <tjx@nvidia.com>:

Reduced unit test complexity

Merging this change closes #13254

FUTURE_COPYBARA_INTEGRATE_REVIEW=https://github.com/openxla/xla/pull/13254 from Tixxx:tixxx/pipeliner_improve 977e713643ae08337ba61ab15fe7812f95078900
