class_docstr: Reduction using Byteps Push Pull.
class_docstr_tok: ["Reduction","using","Byteps","Push","Pull","."]
docstr: NCCL all-reduce implementation of CrossDeviceOps.
  
  It uses Nvidia NCCL for all-reduce. Before performing all-reduce, tensors
  will be repacked or aggregated for more efficient cross-device
  transportation.
  
  Args:
    num_packs: values will be packed in this many splits.  `num_packs` should
      be greater than or equals 0. When it is zero, no packing will be done.
  
  Raises:
    ValueError if `num_packs` is negative.
code: def __init__(self, num_packs=1):
      """NCCL all-reduce implementation of CrossDeviceOps.
  
      It uses Nvidia NCCL for all-reduce. Before performing all-reduce, tensors
      will be repacked or aggregated for more efficient cross-device
      transportation.
  
      Args:
        num_packs: values will be packed in this many splits.  `num_packs` should
          be greater than or equals 0. When it is zero, no packing will be done.
  
      Raises:
        ValueError if `num_packs` is negative.
      """
      if num_packs < 0:
          raise ValueError(
              "NCCL all-reduce requires num_packs >= 0, but {} is specified".format(
                  num_packs
              )
          )
      super(BytepsAllReduce, self).__init__(
          all_reduce_alg="nccl", num_packs=num_packs
      )
      # self._simple_cross_replica_ops = ReductionToOneDevice()
      self._simple_cross_replica_ops = BytepsCrossDeviceOps()
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import enum
  import six
  from tensorflow.python.client import device_lib
  from tensorflow.python.distribute import cross_device_utils
  from tensorflow.python.distribute import device_util
  from tensorflow.python.distribute import reduce_util
  from tensorflow.python.distribute import values
  from tensorflow.python.eager import context
  from tensorflow.python.framework import kernels
  from tensorflow.python.framework import ops
  from tensorflow.python.framework import tensor_util
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import resource_variable_ops
  from tensorflow.python.platform import tf_logging
  from tensorflow.python.util.tf_export import tf_export
  from tensorflow.tools.docs import doc_controls
  import tensorflow
  from tensorflow.python.distribute.cross_device_ops import CollectiveAllReduce
  import tensorflow.python.distribute.cross_device_ops
  from byteps.tensorflow.ops import broadcast, _push_pull
  from byteps.tensorflow import local_rank, size
label_desc: Use image node_id 1 to create a new BytepsAllReduce object from inherited base classes: tf_cross_device_ops with example: obj = BytepsAllReduce(num_packs)