class_docstr: Synchronous batch normalization. Stats are synchronized across all workers during training. 
docstr: Compute the mean and variance: it overrides the original _moments.
code: def _moments(self, inputs, reduction_axes, keep_dims, **kwargs):
      """Compute the mean and variance: it overrides the original _moments."""
      worker_mean, worker_variance = super(
          SyncBatchNormalization, self
      )._moments(inputs, reduction_axes, keep_dims=keep_dims, **kwargs)
  
      if size() > 1:
          # Compute variance using: Var[X] = E[X^2] - E[X]^2.
          worker_square_of_mean = tf.math.square(worker_mean)
          worker_mean_of_square = (
              worker_variance + worker_square_of_mean
          )
  
          # Average stats across all workers
          worker_stack = tf.stack([worker_mean, worker_mean_of_square])
          group_stack = _allreduce(worker_stack, op=Sum)
          group_stack /= size()
          group_mean, group_mean_of_square = tf.unstack(group_stack)
  
          group_variance = group_mean_of_square - tf.math.square(
              group_mean
          )
  
          return (group_mean, group_variance)
      else:
          return (worker_mean, worker_variance)
  
file_path: horovod/horovod/tensorflow/sync_batch_norm.py
imports: import tensorflow
  from horovod.tensorflow.mpi_ops import _allreduce
  from horovod.tensorflow.mpi_ops import size, rank
  from horovod.tensorflow.mpi_ops import Sum
label_desc: Use image node_id 2 for calling the SyncBatchNormalization obj's underlying member method code with example usage: obj._moments(inputs, reduction_axes, keep_dims) and returns: group_mean,  group_variance, worker_mean,  worker_variance