When record_shapes=True is specified, profiler will temporarily
     hold references to the tensors; that may further prevent certain
     optimizations that depend on the reference count and introduce
     extra tensor copies.
Examples:
  with torch.profiler.profile(
      activities=[
          torch.profiler.ProfilerActivity.CPU,
          torch.profiler.ProfilerActivity.CUDA,
      ]
  ) as p:
      code_to_profile()
  print(p.key_averages().table(
      sort_by="self_cuda_time_total", row_limit=-1))

Using the profiler's "schedule", "on_trace_ready" and "step"
   functions:
  # Non-default profiler schedule allows user to turn profiler on and off
  # on different iterations of the training loop;
  # trace_handler is called every time a new trace becomes available
  def trace_handler(prof):
      print(prof.key_averages().table(
          sort_by="self_cuda_time_total", row_limit=-1))
