CUDA Stream Sanitizer
Note:
This is a prototype feature, which means it is at an early stage for
  feedback and testing, and its components are subject to change.
Overview
This module introduces CUDA Sanitizer, a tool for detecting
synchronization errors between kernels ran on different streams. It
stores information on accesses to tensors to determine if they are
synchronized or not. When enabled in a python program and a possible
data race is detected, a detailed warning will be printed and the
program will exit.
It can be enabled either by importing this module and calling
"enable_cuda_sanitizer()" or by exporting the "TORCH_CUDA_SANITIZER"
environment variable.
Usage
Here is an example of a simple synchronization error in PyTorch:
import torch
a = torch.rand(4, 2, device="cuda")
with torch.cuda.stream(torch.cuda.Stream()):
       torch.mul(a, 5, out=a)
The "a" tensor is initialized on the default stream and, without any