class_docstr: Optimizer that implements the Conditional Gradient optimization.
  
  This optimizer helps handle constraints well.
  
  Currently only supports frobenius norm constraint or nuclear norm
  constraint.
  See https://arxiv.org/pdf/1803.06453.pdf
  
  ```
  variable -= (1-learning_rate) * (variable + lambda_ * gradient
      / (frobenius_norm(gradient) + epsilon))
  ```
  
  Note that `lambda_` here refers to the constraint "lambda" in
  the paper. `epsilon` is constant with tiny value as compared to
  the value of frobenius norm of gradient. The purpose of `epsilon`
  here is to avoid the case that the value of frobenius norm of
  gradient is 0.
  
  In this implementation, `epsilon` defaults to $10^{-7}$.
  
  For nucler norm constraint, the formula is as following:
  
  ```
  variable -= (1-learning_rate) * (variable
      + lambda_ * top_singular_vector(gradient))
  ```
class_docstr_tok: ["Optimizer","that","implements","the","Conditional","Gradient","optimization",".","This","optimizer","helps","handle","constraints","well",".","Currently","only","supports","frobenius","norm","constraint","or","nuclear","norm","constraint",".","See","https",":","\/\/arxiv.org\/pdf\/1803.06453.pdf","``","`","variable","-=","(","1-learning_rate",")","*","(","variable","+","lambda_","*","gradient","\/","(","frobenius_norm","(","gradient",")","+","epsilon",")",")","``","`","Note","that","`","lambda_","`","here","refers","to","the","constraint","``","lambda","''","in","the","paper",".","`","epsilon","`","is","constant","with","tiny","value","as","compared","to","the","value","of","frobenius","norm","of","gradient",".","The","purpose","of","`","epsilon","`","here","is","to","avoid","the","case","that","the","value","of","frobenius","norm","of","gradient","is","0",".","In","this","implementation",",","`","epsilon","`","defaults","to","$","10^","{","-7","}","$",".","For","nucler","norm","constraint",",","the","formula","is","as","following",":","``","`","variable","-=","(","1-learning_rate",")","*","(","variable","+","lambda_","*","top_singular_vector","(","gradient",")",")","``","`"]
docstr: Construct a new conditional gradient optimizer.
  
  Args:
      learning_rate: A `Tensor` or a floating point value. or a schedule
          that is a `tf.keras.optimizers.schedules.LearningRateSchedule`
          The learning rate.
      lambda_: A `Tensor` or a floating point value. The constraint.
      epsilon: A `Tensor` or a floating point value. A small constant
          for numerical stability when handling the case of norm of
          gradient to be zero.
      ord: Order of the norm. Supported values are `'fro'`
          and `'nuclear'`. Default is `'fro'`, which is frobenius norm.
      name: Optional name prefix for the operations created when
          applying gradients. Defaults to 'ConditionalGradient'.
      **kwargs: keyword arguments. Allowed to be {`clipnorm`,
          `clipvalue`, `lr`, `decay`}. `clipnorm` is clip gradients
          by norm; `clipvalue` is clip gradients by value, `decay` is
          included for backward compatibility to allow time inverse
          decay of learning rate. `lr` is included for backward
          compatibility, recommended to use `learning_rate` instead.
docstr_tok: ["Construct","a","new","conditional","gradient","optimizer",".","Args",":","learning_rate",":","A","`","Tensor","`","or","a","floating","point","value",".","or","a","schedule","that","is","a","`","tf.keras.optimizers.schedules.LearningRateSchedule","`","The","learning","rate",".","lambda_",":","A","`","Tensor","`","or","a","floating","point","value",".","The","constraint",".","epsilon",":","A","`","Tensor","`","or","a","floating","point","value",".","A","small","constant","for","numerical","stability","when","handling","the","case","of","norm","of","gradient","to","be","zero",".","ord",":","Order","of","the","norm",".","Supported","values","are","`","'fro","'","`","and","`","'nuclear","'","`",".","Default","is","`","'fro","'","`",",","which","is","frobenius","norm",".","name",":","Optional","name","prefix","for","the","operations","created","when","applying","gradients",".","Defaults","to","'ConditionalGradient","'",".","*","*","kwargs",":","keyword","arguments",".","Allowed","to","be","{","`","clipnorm","`",",","`","clipvalue","`",",","`","lr","`",",","`","decay","`","}",".","`","clipnorm","`","is","clip","gradients","by","norm",";","`","clipvalue","`","is","clip","gradients","by","value",",","`","decay","`","is","included","for","backward","compatibility","to","allow","time","inverse","decay","of","learning","rate",".","`","lr","`","is","included","for","backward","compatibility",",","recommended","to","use","`","learning_rate","`","instead","."]
code: def __init__(
      self,
      learning_rate: Union[FloatTensorLike, Callable],
      lambda_: Union[FloatTensorLike, Callable] = 0.01,
      epsilon: FloatTensorLike = 1e-7,
      ord: str = "fro",
      name: str = "ConditionalGradient",
      **kwargs,
  ):
      """Construct a new conditional gradient optimizer.
  
      Args:
          learning_rate: A `Tensor` or a floating point value. or a schedule
              that is a `tf.keras.optimizers.schedules.LearningRateSchedule`
              The learning rate.
          lambda_: A `Tensor` or a floating point value. The constraint.
          epsilon: A `Tensor` or a floating point value. A small constant
              for numerical stability when handling the case of norm of
              gradient to be zero.
          ord: Order of the norm. Supported values are `'fro'`
              and `'nuclear'`. Default is `'fro'`, which is frobenius norm.
          name: Optional name prefix for the operations created when
              applying gradients. Defaults to 'ConditionalGradient'.
          **kwargs: keyword arguments. Allowed to be {`clipnorm`,
              `clipvalue`, `lr`, `decay`}. `clipnorm` is clip gradients
              by norm; `clipvalue` is clip gradients by value, `decay` is
              included for backward compatibility to allow time inverse
              decay of learning rate. `lr` is included for backward
              compatibility, recommended to use `learning_rate` instead.
      """
      super().__init__(name=name, **kwargs)
      self._set_hyper("learning_rate", kwargs.get("lr", learning_rate))
      self._set_hyper("lambda_", lambda_)
      self.epsilon = epsilon or tf.keras.backend.epsilon()
      supported_norms = ["fro", "nuclear"]
      if ord not in supported_norms:
          raise ValueError(
              "'ord' must be a supported matrix norm in %s, got '%s' instead"
              % (supported_norms, ord)
          )
      self.ord = ord
  
file_path: addons/tensorflow_addons/optimizers/conditional_gradient.py
imports: import tensorflow
  from tensorflow_addons.optimizers import KerasLegacyOptimizer
  from tensorflow_addons.utils.types import FloatTensorLike
  from typeguard import typechecked
  from typing import Union, Callable
label_desc: Use image node_id 1 to create a new ConditionalGradient object from inherited base classes: KerasLegacyOptimizer with example: obj = ConditionalGradient(learning_rate, lambda_, epsilon, ord, name)