class_docstr: Optimizer that implements the FTRL algorithm.
  
  "Follow The Regularized Leader" (FTRL) is an optimization algorithm developed
  at Google for click-through rate prediction in the early 2010s. It is most
  suitable for shallow models with large and sparse feature spaces.
  The algorithm is described by
  [McMahan et al., 2013](https://research.google.com/pubs/archive/41159.pdf).
  The Keras version has support for both online L2 regularization
  (the L2 regularization described in the paper
  above) and shrinkage-type L2 regularization
  (which is the addition of an L2 penalty to the loss function).
  
  Initialization:
  
  ```python
  n = 0
  sigma = 0
  z = 0
  ```
  
  Update rule for one variable `w`:
  
  ```python
  prev_n = n
  n = n + g ** 2
  sigma = (sqrt(n) - sqrt(prev_n)) / lr
  z = z + g - sigma * w
  if abs(z) < lambda_1:
    w = 0
  else:
    w = (sgn(z) * lambda_1 - z) / ((beta + sqrt(n)) / alpha + lambda_2)
  ```
  
  Notation:
  
  - `lr` is the learning rate
  - `g` is the gradient for the variable
  - `lambda_1` is the L1 regularization strength
  - `lambda_2` is the L2 regularization strength
  
  Check the documentation for the `l2_shrinkage_regularization_strength`
  parameter for more details when shrinkage is enabled, in which case gradient
  is replaced with a gradient with shrinkage.
  
  Args:
    learning_rate: A `Tensor`, floating point value, or a schedule that is a
      `tf.keras.optimizers.schedules.LearningRateSchedule`. The learning rate.
    learning_rate_power: A float value, must be less or equal to zero.
      Controls how the learning rate decreases during training. Use zero for
      a fixed learning rate.
    initial_accumulator_value: The starting value for accumulators.
      Only zero or positive values are allowed.
    l1_regularization_strength: A float value, must be greater than or
      equal to zero. Defaults to 0.0.
    l2_regularization_strength: A float value, must be greater than or
      equal to zero. Defaults to 0.0.
    name: Optional name prefix for the operations created when applying
      gradients.  Defaults to `"Ftrl"`.
    l2_shrinkage_regularization_strength: A float value, must be greater than
      or equal to zero. This differs from L2 above in that the L2 above is a
      stabilization penalty, whereas this L2 shrinkage is a magnitude penalty.
      When input is sparse shrinkage will only happen on the active weights.
    beta: A float value, representing the beta value from the paper.
      Defaults to 0.0.
    **kwargs: Keyword arguments. Allowed to be one of
      `"clipnorm"` or `"clipvalue"`.
      `"clipnorm"` (float) clips gradients by norm; `"clipvalue"` (float) clips
      gradients by value.
  
  Reference:
    - [McMahan et al., 2013](
      https://research.google.com/pubs/archive/41159.pdf)
class_docstr_tok: ["Optimizer","that","implements","the","FTRL","algorithm",".","``","Follow","The","Regularized","Leader","''","(","FTRL",")","is","an","optimization","algorithm","developed","at","Google","for","click-through","rate","prediction","in","the","early","2010s",".","It","is","most","suitable","for","shallow","models","with","large","and","sparse","feature","spaces",".","The","algorithm","is","described","by","[","McMahan","et","al.",",","2013","]","(","https",":","\/\/research.google.com\/pubs\/archive\/41159.pdf",")",".","The","Keras","version","has","support","for","both","online","L2","regularization","(","the","L2","regularization","described","in","the","paper","above",")","and","shrinkage-type","L2","regularization","(","which","is","the","addition","of","an","L2","penalty","to","the","loss","function",")",".","Initialization",":","``","`","python","n","=","0","sigma","=","0","z","=","0","``","`","Update","rule","for","one","variable","`","w","`",":","``","`","python","prev_n","=","n","n","=","n","+","g","*","*","2","sigma","=","(","sqrt","(","n",")","-","sqrt","(","prev_n",")",")","\/","lr","z","=","z","+","g","-","sigma","*","w","if","abs","(","z",")","<","lambda_1",":","w","=","0","else",":","w","=","(","sgn","(","z",")","*","lambda_1","-","z",")","\/","(","(","beta","+","sqrt","(","n",")",")","\/","alpha","+","lambda_2",")","``","`","Notation",":","-","`","lr","`","is","the","learning","rate","-","`","g","`","is","the","gradient","for","the","variable","-","`","lambda_1","`","is","the","L1","regularization","strength","-","`","lambda_2","`","is","the","L2","regularization","strength","Check","the","documentation","for","the","`","l2_shrinkage_regularization_strength","`","parameter","for","more","details","when","shrinkage","is","enabled",",","in","which","case","gradient","is","replaced","with","a","gradient","with","shrinkage",".","Args",":","learning_rate",":","A","`","Tensor","`",",","floating","point","value",",","or","a","schedule","that","is","a","`","tf.keras.optimizers.schedules.LearningRateSchedule","`",".","The","learning","rate",".","learning_rate_power",":","A","float","value",",","must","be","less","or","equal","to","zero",".","Controls","how","the","learning","rate","decreases","during","training",".","Use","zero","for","a","fixed","learning","rate",".","initial_accumulator_value",":","The","starting","value","for","accumulators",".","Only","zero","or","positive","values","are","allowed",".","l1_regularization_strength",":","A","float","value",",","must","be","greater","than","or","equal","to","zero",".","Defaults","to","0.0.","l2_regularization_strength",":","A","float","value",",","must","be","greater","than","or","equal","to","zero",".","Defaults","to","0.0.","name",":","Optional","name","prefix","for","the","operations","created","when","applying","gradients",".","Defaults","to","`","``","Ftrl","''","`",".","l2_shrinkage_regularization_strength",":","A","float","value",",","must","be","greater","than","or","equal","to","zero",".","This","differs","from","L2","above","in","that","the","L2","above","is","a","stabilization","penalty",",","whereas","this","L2","shrinkage","is","a","magnitude","penalty",".","When","input","is","sparse","shrinkage","will","only","happen","on","the","active","weights",".","beta",":","A","float","value",",","representing","the","beta","value","from","the","paper",".","Defaults","to","0.0",".","*","*","kwargs",":","Keyword","arguments",".","Allowed","to","be","one","of","`","``","clipnorm","''","`","or","`","``","clipvalue","''","`",".","`","``","clipnorm","''","`","(","float",")","clips","gradients","by","norm",";","`","``","clipvalue","''","`","(","float",")","clips","gradients","by","value",".","Reference",":","-","[","McMahan","et","al.",",","2013","]","(","https",":","\/\/research.google.com\/pubs\/archive\/41159.pdf",")"]
code: def _resource_apply_sparse(
      self, grad, var, indices, apply_state=None
  ):
      var_device, var_dtype = var.device, var.dtype.base_dtype
      coefficients = (apply_state or {}).get(
          (var_device, var_dtype)
      ) or self._fallback_apply_state(var_device, var_dtype)
  
      # Adjust L2 regularization strength to include beta to avoid the underlying
      # TensorFlow ops needing to include it.
      adjusted_l2_regularization_strength = coefficients[
          "l2_regularization_strength"
      ] + coefficients["beta"] / (2.0 * coefficients["lr_t"])
  
      accum = self.get_slot(var, "accumulator")
      linear = self.get_slot(var, "linear")
  
      if self._l2_shrinkage_regularization_strength <= 0.0:
          return gen_training_ops.ResourceSparseApplyFtrl(
              var=var.handle,
              accum=accum.handle,
              linear=linear.handle,
              grad=grad,
              indices=indices,
              lr=coefficients["lr_t"],
              l1=coefficients["l1_regularization_strength"],
              l2=adjusted_l2_regularization_strength,
              lr_power=coefficients["learning_rate_power"],
              use_locking=self._use_locking,
          )
      else:
          return gen_training_ops.ResourceSparseApplyFtrlV2(
              var=var.handle,
              accum=accum.handle,
              linear=linear.handle,
              grad=grad,
              indices=indices,
              lr=coefficients["lr_t"],
              l1=coefficients["l1_regularization_strength"],
              l2=adjusted_l2_regularization_strength,
              l2_shrinkage=coefficients[
                  "l2_shrinkage_regularization_strength"
              ],
              lr_power=coefficients["learning_rate_power"],
              use_locking=self._use_locking,
          )
  
file_path: tensorflow/tensorflow/python/keras/optimizer_v2/ftrl.py
imports: from tensorflow.python.keras.optimizer_v2 import optimizer_v2
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import gen_training_ops
  from tensorflow.python.ops import init_ops
  from tensorflow.python.ops import math_ops
label_desc: Use image node_id 5 for calling the Ftrl obj's underlying member method code with example usage: obj._resource_apply_sparse(grad, var, indices, apply_state) and returns: gen_training_ops, gen_training_ops