class_docstr: Hook used to call torch.cuda.synchronize() between layer computations,
  slowing the performance down but decreasing the memory usage when using FSDP
code: def __init__(self):
      super().__init__()
      self._hooks = []
  
imports: import logging
  import os
  import torch
  import torch.nn
  from classy_vision import tasks
  from classy_vision.hooks.classy_hook import ClassyHook
  from fairscale.nn import FullyShardedDataParallel
  from fairscale.nn.misc import FlattenParamsWrapper
  from torch.nn import SyncBatchNorm
  from vissl.config.attr_dict import AttrDict
  from vissl.utils.env import get_machine_local_and_dist_rank
  from vissl.utils.layer_memory_tracking import LayerwiseMemoryTracker
  from vissl.utils.profiler import create_runtime_profiler
label_desc: Use image node_id 2 to create a new CudaSynchronizeHook object from inherited base classes: ClassyHook with example: obj = CudaSynchronizeHook()