class_docstr: Layer by layer profiling of PyTorch models, using the PyTorch profiler
  for memory profiling. Parts of the code are adapted from :obj:`torchprof`
  for layer-wise grouping.
  
  Args:
      model (torch.nn.Module): The underlying model to be profiled.
      enabled (bool, optional): If set to :obj:`True`, turn on the profiler.
          (default: :obj:`False`)
      use_cuda (bool, optional): Whether to profile CUDA execution.
          (default: :obj:`False`)
      profile_memory (bool, optional): If set to :obj:`True`, also profile
          memory usage. (default: :obj:`False`)
      paths ([str], optional): Pre-defined paths for fast loading.
          (default: :obj:`None`)
docstr: Add hooks to torch modules for profiling. The underlying model's
  forward pass is hooked/decorated here.
docstr_tok: ["Add","hooks","to","torch","modules","for","profiling",".","The","underlying","model's","forward","pass","is","hooked\/decorated","here","."]
code: def _hook_trace(self, trace):
      """Add hooks to torch modules for profiling. The underlying model's
      forward pass is hooked/decorated here.
      """
      [path, leaf, module] = trace
  
      # the id of the model  is guaranteed to be unique
      _id = id(module)
      if (self.paths is not None and path in self.paths) or (
          self.paths is None and leaf
      ):
          if _id in self._ids:
              # already wrapped
              return trace
          self._ids.add(_id)
          _forward = module.forward
          self._forwards[path] = _forward
  
          @functools.wraps(_forward)
          def wrap_forward(*args, **kwargs):
              """The forward pass is decorated and profiled here."""
              # only torch 1.8.1+ is supported
              torch_version = torch.__version__
              if torch_version <= "1.8.1":
                  raise NotImplementedError(
                      "Profiler requires at least torch 1.8.1"
                  )
  
              activities = [torch.profiler.ProfilerActivity.CPU]
              if self.use_cuda:
                  activities.append(
                      torch.profiler.ProfilerActivity.CUDA
                  )
              with torch_profiler.profile(
                  activities=activities,
                  profile_memory=self.profile_memory,
              ) as prof:
                  res = _forward(*args, **kwargs)
  
              event_list = prof.events()
  
              # each profile call should be contained in its own list
              self.trace_profile_events[path].append(event_list)
              return res
  
          # decorate the underlying model's forward pass
          module.forward = wrap_forward
      return trace
  
imports: import functools
  from collections import OrderedDict, defaultdict, namedtuple
  from typing import Any, List, NamedTuple, Optional, Tuple
  import torch
  import torch.profiler
label_desc: Use image node_id 7 for calling the Profiler obj's underlying member method code with example usage: obj._hook_trace(trace) and returns: trace, trace, res