class_docstr: Adapted from: https://github.com/NVIDIA/apex/blob/master/apex/parallel/sync_batchnorm.py
  
  Normalizes a NxD input over the first dimension and across all processes.
code: def backward(ctx, grad_output):
      # mini batch mean & var are calculated by forward path.
      # mu = 1./N*np.sum(h, axis = 0)
      # var = 1./N*np.sum((h-mu)**2, axis = 0)
      last_input, mean, var = ctx.saved_tensors
  
      eps = ctx.eps
      grad_input = None
      num_features = mean.size()[0]
  
      # calculate grad_input
      if ctx.needs_input_grad[0]:
          # dh = gamma * (var + eps)**(-1. / 2.) * (dy - np.mean(dy, axis=0)
          #     - (h - mu) * (var + eps)**(-1.0) * np.mean(dy * (h - mu), axis=0))
          mean_dy = grad_output.mean(0)
          mean_dy_xmu = (
              (grad_output * (last_input - mean))
              .view(-1, num_features)
              .mean(0)
          )
          # If running on a distributed setting, perform mean reduction of tensors over
          # all processes.
          mean_dy = all_reduce_mean(mean_dy)
          mean_dy_xmu = all_reduce_mean(mean_dy_xmu)
  
          grad_input = (
              grad_output
              - mean_dy
              - (last_input - mean) / (var + eps) * mean_dy_xmu
          ) / torch.sqrt(var + eps)
  
      return grad_input, None
  
imports: import torch
  from classy_vision.generic.distributed_util import all_reduce_mean
  from torch.autograd import Function
label_desc: Use image node_id 2 for calling the SyncNormalizeFunction obj's underlying member method code with example usage: obj.backward(ctx, grad_output) and returns: grad_input,  None