In this blog, we present several features implemented in the open source version of HTA, which can be used as a Python script as well as interactively in a Jupyter notebook. HTA provides the following features:

Breakdown by Dimensions
Temporal: Breakdown of GPU time in terms of time spent in computation, communication, memory events, and idle time on a single node and across all ranks.
Idle Time: Breakdown of GPU idle time into waiting for the host, waiting for another kernel or attributed to an unknown cause.
Kernel: Find kernels with the longest duration on each rank.
Communication Computation Overlap: Calculate the percentage of time when communication overlaps computation.


Statistical Analysis
Kernel Duration Distribution: Distribution of average time taken by longest kernels across different ranks.
CUDA Kernel Launch: Distributions of GPU kernels with very small duration, large duration, and excessive launch time.


