class_docstr: Data-parallel training in 'ParameterServer' mode.
  It builds one tower on each GPU with
  shared variable scope. It synchronizes the gradients computed
  from each tower, averages them and applies to the shared variables.
  
  It is an equivalent of ``--variable_update=parameter_server`` in
  `tensorflow/benchmarks <https://github.com/tensorflow/benchmarks>`_.
class_docstr_tok: ["Data-parallel","training","in","'ParameterServer","'","mode",".","It","builds","one","tower","on","each","GPU","with","shared","variable","scope",".","It","synchronizes","the","gradients","computed","from","each","tower",",","averages","them","and","applies","to","the","shared","variables",".","It","is","an","equivalent","of","``","--","variable_update=parameter_server","``","in","`","tensorflow\/benchmarks","<","https",":","\/\/github.com\/tensorflow\/benchmarks",">","`","_","."]
docstr: Args:
      towers(list[int]): list of GPU id
      ps_device (str): either 'gpu' or 'cpu', where variables are stored.
docstr_tok: ["Args",":","towers","(","list","[","int","]",")",":","list","of","GPU","id","ps_device","(","str",")",":","either","'gpu","'","or","'cpu","'",",","where","variables","are","stored","."]
code: def __init__(self, towers, ps_device):
      """
      Args:
          towers(list[int]): list of GPU id
          ps_device (str): either 'gpu' or 'cpu', where variables are stored.
      """
      super(SyncMultiGPUParameterServerBuilder, self).__init__(towers)
      assert ps_device in ["cpu", "gpu"]
      self.ps_device = ps_device
  
imports: import copy
  import pprint
  import re
  from abc import ABCMeta, abstractmethod
  from contextlib import contextmanager
  import six
  import tensorflow
  from ..compat import tfv1
  from ..tfutils.common import get_tf_version_tuple
  from ..tfutils.gradproc import ScaleGradient
  from ..tfutils.tower import TrainTowerContext
  from ..utils import logger
  from ..utils.develop import HIDE_DOC
  from .utils import GradientPacker, LeastLoadedDeviceSetter, aggregate_grads_colocate, allreduce_grads_naive, allreduce_grads, allreduce_grads_hierarchical, merge_grad_list, override_to_local_variable, split_grad_list
label_desc: Use image node_id 1 to create a new SyncMultiGPUParameterServerBuilder object from inherited base classes: DataParallelBuilder with example: obj = SyncMultiGPUParameterServerBuilder(towers, ps_device)