PR #10782: Offloading 2/3: generate multi-stream async copies using events on GPUs
PR #10782: Offloading 2/3: generate multi-stream async copies using events on GPUs

Imported from GitHub PR https://github.com/openxla/xla/pull/10782

Emit asynchronous memory copies between hosts and devices using additional streams while the main stream does the computation. The async copies are guarded by RecordEvent() and WaitForEvent() created by the copy-start/copy-done thunks respectively. A hash table is utilized to map copy-start instructions to events. The corresponding events will be waited at copy-done and extracted from the hash table.
Copybara import of the project:

--
a9271ee9b99eb43a0a0806c8dbdfc7753aef4644 by Jane Liu <janeliu@nvidia.com>:

Add the multi-stream implementation for copy-start and copy-done.

Merging this change closes #10782

FUTURE_COPYBARA_INTEGRATE_REVIEW=https://github.com/openxla/xla/pull/10782 from zhenying-liu:async_copy a9271ee9b99eb43a0a0806c8dbdfc7753aef4644
