

Figure 4: GPU performance metrics in MAIProf.

We made the following observations from Figure 4:

The streaming multiprocessor (SM) runs the model’s CUDA kernels. Its utilization [1] is 9.1%, indicating that the parallel compute units on the GPU are not well utilized.
Tensor Core utilization is 0, meaning that Tensor Core (the mixed-precision compute unit on GPU) [2] is not used at all.
Max GPU memory utilization is 47.13%, indicating that half of the GPU memory is left unused.

Step 4:
Collect a GPU trace (aka Kineto trace) of the training loop as shown in Figure 5.




Figure 5: A GPU trace (aka Kineto trace) of the training loop.
