Allow merging streams in one stream group.
Hi @ezhulenev Can you please review this PR ? Thank you! Adding @jyingl3 here because I'm not super familiar with this part of TF. Discussed with @chuanhaozhuge. Chuanhao is working on changes related to GPU so will let Chuanhao review this PR.
This works as one part of the whole multi-stream feature, which is proposed in this [PR](https://github.com/tensorflow/tensorflow/pull/59843), but can also be used independently.

In TensorFlow, there are at least three streams in one stream group: 1 compute stream, 1 h2d stream, and 1 d2h stream. This is designed to overlap the computation and memory copy on GPU. However, for some copy-frequent and launch-bound models, there are significant stream-synchronization overhead caused by the frequent switching between the three kinds of streams. So, let the operations use the same stream may bring acceleration. This PR provides an option to do so, just by setting the environment variable `TF_GPU_STREAM_MERGE=true`.