class_docstr: AllReduce Synchronizer.
  
  This AllReduce Synchronizer currently uses TensorFlow's `collective_device_ops`
  to insert their AllReduce ops into our graph.
  
  The class AllReduceSynchronizer class contains the following possible instantiations:
  
  1. spec=`auto`: single-node multiple devices, or cross-node AllReduce based on collective ops
  2. spec=`nccl`: single-node multiple devices, or cross-node AllReduce based on NCCL
  3. spec=`ring`/'tree', AllReduce with different reduction structures: ring, tree, etc.
  
  However note that it does not contain the following instantiations:
  
  1. shuffle reduce (reduce to CPU or GPU as in PS) + AllReduce across nodes
  2. any other types of hybrid reduction of PS and AllReduce.
class_docstr_tok: ["AllReduce","Synchronizer",".","This","AllReduce","Synchronizer","currently","uses","TensorFlow","'s","`","collective_device_ops","`","to","insert","their","AllReduce","ops","into","our","graph",".","The","class","AllReduceSynchronizer","class","contains","the","following","possible","instantiations",":","1.","spec=","`","auto","`",":","single-node","multiple","devices",",","or","cross-node","AllReduce","based","on","collective","ops","2.","spec=","`","nccl","`",":","single-node","multiple","devices",",","or","cross-node","AllReduce","based","on","NCCL","3.","spec=","`","ring","`","\/'tree","'",",","AllReduce","with","different","reduction","structures",":","ring",",","tree",",","etc",".","However","note","that","it","does","not","contain","the","following","instantiations",":","1.","shuffle","reduce","(","reduce","to","CPU","or","GPU","as","in","PS",")","+","AllReduce","across","nodes","2.","any","other","types","of","hybrid","reduction","of","PS","and","AllReduce","."]
docstr: Allreduce synchronizer will do nothing in between-graph synchronization.
docstr_tok: ["Allreduce","synchronizer","will","do","nothing","in","between-graph","synchronization","."]
code: def between_graph_apply(self, graph_item, var_name):
      """Allreduce synchronizer will do nothing in between-graph synchronization."""
      return graph_item
  
imports: from collections import defaultdict
  from tensorflow.python import ops
  from tensorflow.python.framework import device_spec
  from tensorflow.python.ops import collective_ops
  import autodist
  from autodist.const import ENV
  from autodist.kernel.common.utils import get_consumers, update_consumers, replica_prefix, get_control_consumers, update_control_consumers
  from autodist.kernel.common.utils import get_op_name
  from autodist.kernel.synchronization.collective_key import get_collective_keys
  from autodist.kernel.synchronization.compressor import Compressor, CollectiveOpsConfig
  from autodist.kernel.synchronization.synchronizer import Synchronizer
  from autodist.proto import synchronizers_pb2
  from autodist.utils import logging
label_desc: Use image node_id 6 for calling the AllReduceSynchronizer obj's underlying member method code with example usage: obj.between_graph_apply(graph_item, var_name) and returns: graph_item