torch.utils.bottleneck
torch.utils.bottleneck is a tool that can be used as an initial step
for debugging bottlenecks in your program. It summarizes runs of your
script with the Python profiler and PyTorch's autograd profiler.
Run it on the command line with
python -m torch.utils.bottleneck /path/to/source/script.py [args]
where [args] are any number of arguments to script.py, or run
"python -m torch.utils.bottleneck -h" for more usage instructions.
Warning:
Because your script will be profiled, please ensure that it exits in
  a finite amount of time.
Warning:
Due to the asynchronous nature of CUDA kernels, when running against
  CUDA code, the cProfile output and CPU-mode autograd profilers may
  not show correct timings: the reported CPU time reports the amount
  of time used to launch the kernels but does not include the time the
  kernel spent executing on a GPU unless the operation does a
  synchronize. Ops that do synchronize appear to be extremely