class_docstr: Gradient Communication Quantization (GCQ) in multi-machine distributed training.
  
  It quantizes gradients to int8 locally before gradients communicate between machines.
  When the gradient communication is done, it dequantizes the int8 gradients to original dtype (default: fp16) locally.
  
  Note: When training on multiple machines, it can accelerate significantly, as multi-machine communication is the bottleneck.
  However, it is even slower when training on single machine.
  Please choose if GCQ should be on according to your training environment.
class_docstr_tok: ["Gradient","Communication","Quantization","(","GCQ",")","in","multi-machine","distributed","training",".","It","quantizes","gradients","to","int8","locally","before","gradients","communicate","between","machines",".","When","the","gradient","communication","is","done",",","it","dequantizes","the","int8","gradients","to","original","dtype","(","default",":","fp16",")","locally",".","Note",":","When","training","on","multiple","machines",",","it","can","accelerate","significantly",",","as","multi-machine","communication","is","the","bottleneck",".","However",",","it","is","even","slower","when","training","on","single","machine",".","Please","choose","if","GCQ","should","be","on","according","to","your","training","environment","."]
docstr: Quantize gradients to int8 locally before gradients communicate between machines.
docstr_tok: ["Quantize","gradients","to","int8","locally","before","gradients","communicate","between","machines","."]
code: def encode_bucket(self):
      """
      Quantize gradients to int8 locally before gradients communicate between machines.
      """
      # The original gradient need to divide the world_size to avoid all_reduce overflow.
      compressed_gradient = self.original_gradient.div(
          self.scale * self.world_size
      )
      self.encoded_gradient = (
          compressed_gradient.clamp_(
              min=self.new_min_bound, max=self.new_max_bound
          )
          .round_()
          .to(torch.int8)
      )
      return self.encoded_gradient
  
file_path: lightseq/lightseq/training/gcq/gcq.py
imports: import math
  import torch
  import torch.distributed
label_desc: Use image node_id 3 for calling the GCQ obj's underlying member method code with example usage: obj.encode_bucket() and returns: self