docstr: Loss function and optimizer for multiple devices
  
  Parameters
  ----------
  criterion: torch.nn.Module
      criterion to compute loss
  ndev: int
      number of devices used
  grad_accum: int
      number of batches to accumulate gradients
  model: torch.nn.Module
      model to optimizer (needed to iterate and synchronize all parameters)
  opt: Optimizer
      Model optimizer to use. If None, then no backward and update will be
      performed
docstr_tok: ["Loss","function","and","optimizer","for","multiple","devices","Parameters","--","--","--","--","--","criterion",":","torch.nn.Module","criterion","to","compute","loss","ndev",":","int","number","of","devices","used","grad_accum",":","int","number","of","batches","to","accumulate","gradients","model",":","torch.nn.Module","model","to","optimizer","(","needed","to","iterate","and","synchronize","all","parameters",")","opt",":","Optimizer","Model","optimizer","to","use",".","If","None",",","then","no","backward","and","update","will","be","performed"]
code: def __init__(self, criterion, ndev, grad_accum, model, opt=None):
      """Loss function and optimizer for multiple devices
  
      Parameters
      ----------
      criterion: torch.nn.Module
          criterion to compute loss
      ndev: int
          number of devices used
      grad_accum: int
          number of batches to accumulate gradients
      model: torch.nn.Module
          model to optimizer (needed to iterate and synchronize all parameters)
      opt: Optimizer
          Model optimizer to use. If None, then no backward and update will be
          performed
      """
      super(MultiGPULossCompute, self).__init__(
          criterion, grad_accum, opt=opt
      )
      self.ndev = ndev
      self.model = model
  
imports: import torch
  import torch.distributed
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 1 to create a new MultiGPULossCompute object from inherited base classes: SimpleLossCompute with example: obj = MultiGPULossCompute(criterion, ndev, grad_accum, model, opt)