class_docstr: We keep track of the last pull of each tensor from the server, and then when
  a push is requested, we accumulate the difference between the pulled tensor
  and the current version
class_docstr_tok: ["We","keep","track","of","the","last","pull","of","each","tensor","from","the","server",",","and","then","when","a","push","is","requested",",","we","accumulate","the","difference","between","the","pulled","tensor","and","the","current","version"]
code: def update(self, key: str, tensor: torch.Tensor) -> None:
      if key in self._cache:
          diff = tensor - self._cache[key]
          self._client.swap(key, diff, self._cache[key], accum=True)
          tensor.copy_(self._cache[key])
      else:
          self._cache[key] = tensor.clone()
          # all the clients race to set the initial value of the tensor, then
          # every clients just uses that one
          self._client.swap(
              key, self._cache[key], self._cache[key], overwrite=False
          )
          tensor.copy_(self._cache[key])
  
imports: import logging
  import multiprocessing
  import queue
  import time
  import traceback
  from functools import lru_cache
  from typing import Callable, Dict, List, Optional, Sequence
  import torch
  import torch.distributed
  import torch.multiprocessing
  from torchbiggraph.distributed import init_process_group, Startable
  from torchbiggraph.types import CharTensorType, Rank
  from torchbiggraph.util import allocate_shared_tensor, split_almost_equally, tag_logs_with_process_name
label_desc: Use image node_id 4 for calling the GradientParameterClient obj's underlying member method code with example usage: obj.update(key, tensor) without return types