class_docstr: A variant of the Adam optimizer that handles sparse updates more efficiently.
  
  The original Adam algorithm maintains two moving-average accumulators for
  each trainable variable; the accumulators are updated at every step.
  In this variant, only moments that show up in the gradient get updated,
  and only those portions of the gradient get applied to the parameters.
  Compared with the original Adam optimizer, it can provide large improvements in
  model training throughput for some applications.
class_docstr_tok: ["A","variant","of","the","Adam","optimizer","that","handles","sparse","updates","more","efficiently",".","The","original","Adam","algorithm","maintains","two","moving-average","accumulators","for","each","trainable","variable",";","the","accumulators","are","updated","at","every","step",".","In","this","variant",",","only","moments","that","show","up","in","the","gradient","get","updated",",","and","only","those","portions","of","the","gradient","get","applied","to","the","parameters",".","Compared","with","the","original","Adam","optimizer",",","it","can","provide","large","improvements","in","model","training","throughput","for","some","applications","."]
docstr: Create a slightly modified version of tf.keras.optimizers.Adam.
  
  which only update moving-average accumulators for sparse variable
  indices that appear in the current batch.
  
  :param learning_rate: A `Tensor`, floating point value, or a schedule that is a
      `tf.keras.optimizers.schedules.LearningRateSchedule`, or a callable
      that takes no arguments and returns the actual value to use, The
      learning rate. Defaults to 0.001.
  :param beta_1: A float value or a constant float tensor, or a callable
      that takes no arguments and returns the actual value to use. The
      exponential decay rate for the 1st moment estimates. Defaults to 0.9.
  :param beta_2: A float value or a constant float tensor, or a callable
      that takes no arguments and returns the actual value to use, The
      exponential decay rate for the 2nd moment estimates. Defaults to 0.999.
  :param epsilon: A small constant for numerical stability. This epsilon is
      "epsilon hat" in the Kingma and Ba paper (in the formula just before
      Section 2.1), not the epsilon in Algorithm 1 of the paper. Defaults to
      1e-7.
  :param amsgrad: Boolean. Currently amsgrad is not supported and it can only
      set to False.
  :param name: Optional name for the operations created when applying gradients.
      Defaults to `"Adam"`.
  :param kwargs: Keyword arguments. Allowed to be one of
      `"clipnorm"` or `"clipvalue"`.
      `"clipnorm"` (float) clips gradients by norm; `"clipvalue"` (float) clips
      gradients by value.
docstr_tok: ["Create","a","slightly","modified","version","of","tf.keras.optimizers.Adam",".","which","only","update","moving-average","accumulators","for","sparse","variable","indices","that","appear","in","the","current","batch",".",":","param","learning_rate",":","A","`","Tensor","`",",","floating","point","value",",","or","a","schedule","that","is","a","`","tf.keras.optimizers.schedules.LearningRateSchedule","`",",","or","a","callable","that","takes","no","arguments","and","returns","the","actual","value","to","use",",","The","learning","rate",".","Defaults","to","0.001.",":","param","beta_1",":","A","float","value","or","a","constant","float","tensor",",","or","a","callable","that","takes","no","arguments","and","returns","the","actual","value","to","use",".","The","exponential","decay","rate","for","the","1st","moment","estimates",".","Defaults","to","0.9.",":","param","beta_2",":","A","float","value","or","a","constant","float","tensor",",","or","a","callable","that","takes","no","arguments","and","returns","the","actual","value","to","use",",","The","exponential","decay","rate","for","the","2nd","moment","estimates",".","Defaults","to","0.999.",":","param","epsilon",":","A","small","constant","for","numerical","stability",".","This","epsilon","is","``","epsilon","hat","''","in","the","Kingma","and","Ba","paper","(","in","the","formula","just","before","Section","2.1",")",",","not","the","epsilon","in","Algorithm","1","of","the","paper",".","Defaults","to","1e-7",".",":","param","amsgrad",":","Boolean",".","Currently","amsgrad","is","not","supported","and","it","can","only","set","to","False",".",":","param","name",":","Optional","name","for","the","operations","created","when","applying","gradients",".","Defaults","to","`","``","Adam","''","`",".",":","param","kwargs",":","Keyword","arguments",".","Allowed","to","be","one","of","`","``","clipnorm","''","`","or","`","``","clipvalue","''","`",".","`","``","clipnorm","''","`","(","float",")","clips","gradients","by","norm",";","`","``","clipvalue","''","`","(","float",")","clips","gradients","by","value","."]
code: def __init__(
      self,
      learning_rate=0.001,
      beta_1=0.9,
      beta_2=0.999,
      epsilon=1e-7,
      amsgrad=False,
      name="SparseAdam",
      **kwargs,
  ):
      """
      Create a slightly modified version of tf.keras.optimizers.Adam.
  
      which only update moving-average accumulators for sparse variable
      indices that appear in the current batch.
  
      :param learning_rate: A `Tensor`, floating point value, or a schedule that is a
          `tf.keras.optimizers.schedules.LearningRateSchedule`, or a callable
          that takes no arguments and returns the actual value to use, The
          learning rate. Defaults to 0.001.
      :param beta_1: A float value or a constant float tensor, or a callable
          that takes no arguments and returns the actual value to use. The
          exponential decay rate for the 1st moment estimates. Defaults to 0.9.
      :param beta_2: A float value or a constant float tensor, or a callable
          that takes no arguments and returns the actual value to use, The
          exponential decay rate for the 2nd moment estimates. Defaults to 0.999.
      :param epsilon: A small constant for numerical stability. This epsilon is
          "epsilon hat" in the Kingma and Ba paper (in the formula just before
          Section 2.1), not the epsilon in Algorithm 1 of the paper. Defaults to
          1e-7.
      :param amsgrad: Boolean. Currently amsgrad is not supported and it can only
          set to False.
      :param name: Optional name for the operations created when applying gradients.
          Defaults to `"Adam"`.
      :param kwargs: Keyword arguments. Allowed to be one of
          `"clipnorm"` or `"clipvalue"`.
          `"clipnorm"` (float) clips gradients by norm; `"clipvalue"` (float) clips
          gradients by value.
      """
      super().__init__(
          learning_rate=learning_rate,
          beta_1=beta_1,
          beta_2=beta_2,
          epsilon=epsilon,
          amsgrad=amsgrad,
          name=name,
          **kwargs,
      )
  
file_path: BigDL/python/nano/src/bigdl/nano/tf/optimizers/sparse_adam.py
imports: import tensorflow.compat.v2
  from keras import backend_config
  from tensorflow.python.util.tf_export import keras_export
  import tensorflow
  from bigdl.nano.utils.common import invalidInputError
  from bigdl.nano.utils.tf import KERAS_VERSION_LESS_2_9
  from tensorflow.keras.optimizers import Adam
label_desc: Use image node_id 1 to create a new SparseAdam object from inherited base classes: Adam with example: obj = SparseAdam(learning_rate, beta_1, beta_2, epsilon, amsgrad, name)