class_docstr: A way of adapting attributes of underlying class to distributed mode. Same as in WrappedDataParallel above.
  Even when using distributed on a single machine with multiple GPUs, apex can speed up training significantly.
  Distributed code must be launched with "python -m torch.distributed.launch --nproc_per_node=1 run_script.py"
class_docstr_tok: ["A","way","of","adapting","attributes","of","underlying","class","to","distributed","mode",".","Same","as","in","WrappedDataParallel","above",".","Even","when","using","distributed","on","a","single","machine","with","multiple","GPUs",",","apex","can","speed","up","training","significantly",".","Distributed","code","must","be","launched","with","``","python","-m","torch.distributed.launch","--","nproc_per_node=1","run_script.py","''"]
code: def __getattr__(self, name):
      try:
          return super().__getattr__(name)
      except AttributeError:
          return getattr(self.module, name)
  
file_path: FARM/farm/modeling/optimization.py
imports: from importlib import import_module
  import logging
  import sys
  import inspect
  import torch
  from torch.nn.parallel import DistributedDataParallel
  from torch.nn import DataParallel
  from transformers.optimization import AdamW
  from farm.utils import MLFlowLogger
label_desc: Use image node_id 1 for calling the WrappedDDP obj's underlying member method code with example usage: obj.__getattr__(name) and returns: super, getattr