class_docstr: All-reduce cross device ops using collective ops.
  
  In the between-graph replicated training, it will still do all-reduces across
  all workers and then put results on the right destinations.
class_docstr_tok: ["All-reduce","cross","device","ops","using","collective","ops",".","In","the","between-graph","replicated","training",",","it","will","still","do","all-reduces","across","all","workers","and","then","put","results","on","the","right","destinations","."]
docstr: Initializes the object.
  
  Args:
    num_workers: number of workers in the between-graph replicated training.
    num_gpus_per_worker: number of GPUs per worker.
    num_packs: gradients will be packed into `num_packs` chunks.
    collective_keys: an optional CollectiveKey object.
    communication: indicates which collective communication to use.
code: def __init__(
      self,
      num_workers=1,
      num_gpus_per_worker=0,
      num_packs=1,
      collective_keys=None,
      communication=CollectiveCommunication.AUTO,
  ):
      """Initializes the object.
  
      Args:
        num_workers: number of workers in the between-graph replicated training.
        num_gpus_per_worker: number of GPUs per worker.
        num_packs: gradients will be packed into `num_packs` chunks.
        collective_keys: an optional CollectiveKey object.
        communication: indicates which collective communication to use.
      """
      self._num_workers = num_workers
      self._num_gpus_per_worker = num_gpus_per_worker
      self._num_packs = num_packs
      self._collective_keys = (
          collective_keys or cross_device_utils.CollectiveKeys()
      )
      self._communication = communication
      super(MyCollectiveAllReduce, self).__init__()
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import enum
  import six
  from tensorflow.python.client import device_lib
  from tensorflow.python.distribute import cross_device_utils
  from tensorflow.python.distribute import device_util
  from tensorflow.python.distribute import reduce_util
  from tensorflow.python.distribute import values
  from tensorflow.python.eager import context
  from tensorflow.python.framework import kernels
  from tensorflow.python.framework import ops
  from tensorflow.python.framework import tensor_util
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import resource_variable_ops
  from tensorflow.python.platform import tf_logging
  from tensorflow.python.util.tf_export import tf_export
  from tensorflow.tools.docs import doc_controls
  import tensorflow
  from tensorflow.python.distribute.cross_device_ops import CollectiveAllReduce
  import tensorflow.python.distribute.cross_device_ops
  from byteps.tensorflow.ops import broadcast, _push_pull
  from byteps.tensorflow import local_rank, size
label_desc: Use image node_id 1 to create a new MyCollectiveAllReduce object from inherited base classes: CollectiveAllReduce with example: obj = MyCollectiveAllReduce(num_workers, num_gpus_per_worker, num_packs, collective_keys, communication)