PR #10334: [ROCM] fixing collective ops tests
PR #10334: [ROCM] fixing collective ops tests

Imported from GitHub PR https://github.com/openxla/xla/pull/10334

Here I fix collectives ops tests which were not taken care of for quite a some time.
- xla/tests/collective_ops_test_e2e: failed because collectives were made async by default in https://github.com/openxla/xla/commit/79ff5de7601d27e78fc1760b4ca20baded1f9139 while this test checks for sync/async collectives
- xla/tests/collective_ops_test: failed with errors like:
```
INTERNAL: Unsupported instruction opcode: collective-permute
``` 
because HLO passes were not run for a number of subtests. The point is we need [AsyncCollectiveCreator](https://github.com/ROCm/xla/blob/7ab393fc384651f367fdff4c3b1d5b1c4cbd9342/xla/service/gpu/gpu_compiler.cc#L1103) call in order to rewrite sync collectives into "start" and "done" pairs: e.g., collective-permute gets rewritten into 'collective-permute-start' and 'collective-permute-done'.

@xla-rotation: could you have a look, please ?

Copybara import of the project:

--
4b8bcbc61b49855c2477ab23bba9c59ab81dd62e by Pavel Emeliyanenko <pavel.emeliyanenko@amd.com>:

fixing collective ops tests

Merging this change closes #10334

FUTURE_COPYBARA_INTEGRATE_REVIEW=https://github.com/openxla/xla/pull/10334 from ROCm:ci_collective_ops_tests_240307 4b8bcbc61b49855c2477ab23bba9c59ab81dd62e
