class_docstr: Multi Optimizer Wrapper for Discriminative Layer Training.
  
  Creates a wrapper around a set of instantiated optimizer layer pairs.
  Generally useful for transfer learning of deep networks.
  
  Each optimizer will optimize only the weights associated with its paired layer.
  This can be used to implement discriminative layer training by assigning
  different learning rates to each optimizer layer pair.
  `(tf.keras.optimizers.legacy.Optimizer, List[tf.keras.layers.Layer])` pairs are also supported.
  Please note that the layers must be instantiated before instantiating the optimizer.
  
  Args:
      optimizers_and_layers: a list of tuples of an optimizer and a layer or model.
          Each tuple should contain exactly 1 instantiated optimizer and 1 object that
          subclasses `tf.keras.Model`, `tf.keras.Sequential` or `tf.keras.layers.Layer`.
          Nested layers and models will be automatically discovered.
          Alternatively, in place of a single layer, you can pass a list of layers.
      optimizer_specs: specialized list for serialization.
          Should be left as None for almost all cases.
          If you are loading a serialized version of this optimizer,
          please use `tf.keras.models.load_model` after saving a model compiled with this optimizer.
  
  Usage:
  
  >>> model = tf.keras.Sequential([
  ...     tf.keras.Input(shape=(4,)),
  ...     tf.keras.layers.Dense(8),
  ...     tf.keras.layers.Dense(16),
  ...     tf.keras.layers.Dense(32),
  ... ])
  >>> optimizers = [
  ...     tf.keras.optimizers.Adam(learning_rate=1e-4),
  ...     tf.keras.optimizers.Adam(learning_rate=1e-2)
  ... ]
  >>> optimizers_and_layers = [(optimizers[0], model.layers[0]), (optimizers[1], model.layers[1:])]
  >>> optimizer = tfa.optimizers.MultiOptimizer(optimizers_and_layers)
  >>> model.compile(optimizer=optimizer, loss="mse")
  
  Reference:
      - [Universal Language Model Fine-tuning for Text Classification](https://arxiv.org/abs/1801.06146)
      - [Collaborative Layer-wise Discriminative Learning in Deep Neural Networks](https://arxiv.org/abs/1607.05440)
  
  Note: Currently, `tfa.optimizers.MultiOptimizer` does not support callbacks that modify optimizers.
      However, you can instantiate optimizer layer pairs with
      `tf.keras.optimizers.schedules.LearningRateSchedule`
      instead of a static learning rate.
  
  This code should function on CPU, GPU, and TPU. Apply with `tf.distribute.Strategy().scope()` context as you
  would with any other optimizer.
class_docstr_tok: ["Multi","Optimizer","Wrapper","for","Discriminative","Layer","Training",".","Creates","a","wrapper","around","a","set","of","instantiated","optimizer","layer","pairs",".","Generally","useful","for","transfer","learning","of","deep","networks",".","Each","optimizer","will","optimize","only","the","weights","associated","with","its","paired","layer",".","This","can","be","used","to","implement","discriminative","layer","training","by","assigning","different","learning","rates","to","each","optimizer","layer","pair",".","`","(","tf.keras.optimizers.legacy.Optimizer",",","List","[","tf.keras.layers.Layer","]",")","`","pairs","are","also","supported",".","Please","note","that","the","layers","must","be","instantiated","before","instantiating","the","optimizer",".","Args",":","optimizers_and_layers",":","a","list","of","tuples","of","an","optimizer","and","a","layer","or","model",".","Each","tuple","should","contain","exactly","1","instantiated","optimizer","and","1","object","that","subclasses","`","tf.keras.Model","`",",","`","tf.keras.Sequential","`","or","`","tf.keras.layers.Layer","`",".","Nested","layers","and","models","will","be","automatically","discovered",".","Alternatively",",","in","place","of","a","single","layer",",","you","can","pass","a","list","of","layers",".","optimizer_specs",":","specialized","list","for","serialization",".","Should","be","left","as","None","for","almost","all","cases",".","If","you","are","loading","a","serialized","version","of","this","optimizer",",","please","use","`","tf.keras.models.load_model","`","after","saving","a","model","compiled","with","this","optimizer",".","Usage",":",">",">",">","model","=","tf.keras.Sequential","(","[","...","tf.keras.Input","(","shape=","(","4",",",")",")",",","...","tf.keras.layers.Dense","(","8",")",",","...","tf.keras.layers.Dense","(","16",")",",","...","tf.keras.layers.Dense","(","32",")",",","...","]",")",">",">",">","optimizers","=","[","...","tf.keras.optimizers.Adam","(","learning_rate=1e-4",")",",","...","tf.keras.optimizers.Adam","(","learning_rate=1e-2",")","...","]",">",">",">","optimizers_and_layers","=","[","(","optimizers","[","0","]",",","model.layers","[","0","]",")",",","(","optimizers","[","1","]",",","model.layers","[","1",":","]",")","]",">",">",">","optimizer","=","tfa.optimizers.MultiOptimizer","(","optimizers_and_layers",")",">",">",">","model.compile","(","optimizer=optimizer",",","loss=","''","mse","''",")","Reference",":","-","[","Universal","Language","Model","Fine-tuning","for","Text","Classification","]","(","https",":","\/\/arxiv.org\/abs\/1801.06146",")","-","[","Collaborative","Layer-wise","Discriminative","Learning","in","Deep","Neural","Networks","]","(","https",":","\/\/arxiv.org\/abs\/1607.05440",")","Note",":","Currently",",","`","tfa.optimizers.MultiOptimizer","`","does","not","support","callbacks","that","modify","optimizers",".","However",",","you","can","instantiate","optimizer","layer","pairs","with","`","tf.keras.optimizers.schedules.LearningRateSchedule","`","instead","of","a","static","learning","rate",".","This","code","should","function","on","CPU",",","GPU",",","and","TPU",".","Apply","with","`","tf.distribute.Strategy","(",")",".scope","(",")","`","context","as","you","would","with","any","other","optimizer","."]
code: def __repr__(self):
      return "Multi Optimizer with %i optimizer layer pairs" % len(
          self.optimizer_specs
      )
  
file_path: addons/tensorflow_addons/optimizers/discriminative_layer_training.py
imports: from typing import List, Union
  import tensorflow
  from packaging.version import Version
  from tensorflow_addons.optimizers import KerasLegacyOptimizer
  from typeguard import typechecked
label_desc: Use image node_id 6 for calling the MultiOptimizer obj's underlying member method code with example usage: obj.__repr__() and returns: unknown