class_docstr: Synchronizer.
  
  Given a variable, can modify the TF Graph to synchronize its
  gradients in either an in-graph or a between-graph fashion.
  
  - In-graph means the synchronization happens in one `tf.Graph`
  - Between-graph means the synchronization happens across
      multiple `tf.Graphs` (e.g., each worker has its own graph)
docstr: Make gradient's consumers consume the aggregated gradient instead of the original one of replica_0.
docstr_tok: ["Make","gradient","'s","consumers","consume","the","aggregated","gradient","instead","of","the","original","one","of","replica_0","."]
code: def _update_gradient_consumers(
      new_graph_item,
      consumer_ops,
      control_consumer_ops,
      old_tensor_name,
      new_tensor,
  ):
      """Make gradient's consumers consume the aggregated gradient instead of the original one of replica_0."""
      # Get the original tensor (the one from replica 0) to replace
      old_op_name = strip_replica_prefix(get_op_name(old_tensor_name))
      replica_0_op_name = ops.prepend_name_scope(
          old_op_name, replica_prefix(0)
      )
      replica_0_op = new_graph_item.graph.get_operation_by_name(
          replica_0_op_name
      )
      output_idx = get_index_from_tensor_name(old_tensor_name)
      replica_0_tensor = replica_0_op.outputs[output_idx]
  
      update_consumers(consumer_ops, replica_0_tensor, new_tensor)
      update_control_consumers(
          control_consumer_ops, replica_0_tensor.op, new_tensor.op
      )
  
file_path: autodist/autodist/kernel/synchronization/synchronizer.py
imports: from abc import ABC, abstractmethod
  from tensorflow.python import ops
  from autodist.kernel.common.utils import get_op_name, update_consumers, update_control_consumers, replica_prefix, strip_replica_prefix, get_index_from_tensor_name
label_desc: Use image node_id 6 for calling the Synchronizer obj's underlying member method code with example usage: obj._update_gradient_consumers(new_graph_item, consumer_ops, control_consumer_ops, old_tensor_name, new_tensor) without return types