
GPU Metric on Timeline:
This feature will help you debug performance issues when one or more GPU are underutilized. Ideally, your program should have high GPU utilization (aiming for 100% GPU utilization), minimal CPU to GPU communication, and no overhead. 
Overview:
The overview page highlights the results of three important GPU usage metrics at different levels (i.e. GPU Utilization, Est. SM Efficiency, and Est. Achieved Occupancy). Essentially, each GPU has a bunch of SM each with a bunch of warps that can execute a bunch of threads concurrently. Warps execute a bunch because the amount depends on the GPU. But at a high level, this GPU Metric on Timeline tool allows you can see the whole stack, which is useful. 
If the GPU utilization result is low, this suggests a potential bottleneck is present in your model. Common reasons: 
•Insufficient parallelism in kernels (i.e., low batch size) 
•Small kernels called in a loop. This is to say the launch overheads are not amortized