class_docstr: Hook used to call torch.cuda.synchronize() between layer computations,
  slowing the performance down but decreasing the memory usage when using FSDP
code: def on_start(self, task: "tasks.ClassyTask") -> None:
      model = task.base_model
      for module_name, m in model.named_modules():
          if self._should_sync_module(module_name, m):
              self._hooks.append(
                  m.register_forward_pre_hook(self._pre_forward_hook)
              )
              self._hooks.append(
                  m.register_forward_hook(self._post_forward_hook)
              )
              self._hooks.append(
                  m.register_backward_hook(self._backward_hook)
              )
  
imports: import logging
  import os
  import torch
  import torch.nn
  from classy_vision import tasks
  from classy_vision.hooks.classy_hook import ClassyHook
  from fairscale.nn import FullyShardedDataParallel
  from fairscale.nn.misc import FlattenParamsWrapper
  from torch.nn import SyncBatchNorm
  from vissl.config.attr_dict import AttrDict
  from vissl.utils.env import get_machine_local_and_dist_rank
  from vissl.utils.layer_memory_tracking import LayerwiseMemoryTracker
  from vissl.utils.profiler import create_runtime_profiler
label_desc: Use image node_id 3 for calling the CudaSynchronizeHook obj's underlying member method code with example usage: obj.on_start(task) without return types