class_docstr: Base class for optimizers that work hand in hand with Trax layers.
  
  To define an optimizer subclass, specify its behavior with respect to a
  single node in the network (e.g., a single dense layer):
  
    - `init`: how to create/initialize optimizer-internal parameters ("slots"),
        as a function of the node's weights.
    - `update`: how to use gradient information to update node weights and
        optimizer slots.
  
  The Trax runtime combines these node-local computations into layer weight
  updates and optimizer slot updates for the whole tree of layers in the model.
class_docstr_tok: ["Base","class","for","optimizers","that","work","hand","in","hand","with","Trax","layers",".","To","define","an","optimizer","subclass",",","specify","its","behavior","with","respect","to","a","single","node","in","the","network","(","e.g.",",","a","single","dense","layer",")",":","-","`","init","`",":","how","to","create\/initialize","optimizer-internal","parameters","(","``","slots","''",")",",","as","a","function","of","the","node","'s","weights",".","-","`","update","`",":","how","to","use","gradient","information","to","update","node","weights","and","optimizer","slots",".","The","Trax","runtime","combines","these","node-local","computations","into","layer","weight","updates","and","optimizer","slot","updates","for","the","whole","tree","of","layers","in","the","model","."]
docstr: Returns an L2-like norm of all elements of all tensors in `flat_list`.
  
  Args:
    flat_list: Collection of tensors as a flat list (rather than, e.g., a
        tree).
  
  Returns:
    A scalar value computed as if all the tensors in `flat_list` were joined
    and flattened into a single vector, and then the L2 norm of that vector
    was calculated.
docstr_tok: ["Returns","an","L2-like","norm","of","all","elements","of","all","tensors","in","`","flat_list","`",".","Args",":","flat_list",":","Collection","of","tensors","as","a","flat","list","(","rather","than",",","e.g.",",","a","tree",")",".","Returns",":","A","scalar","value","computed","as","if","all","the","tensors","in","`","flat_list","`","were","joined","and","flattened","into","a","single","vector",",","and","then","the","L2","norm","of","that","vector","was","calculated","."]
code: def _l2_norm(self, flat_list):
      """Returns an L2-like norm of all elements of all tensors in `flat_list`.
  
      Args:
        flat_list: Collection of tensors as a flat list (rather than, e.g., a
            tree).
  
      Returns:
        A scalar value computed as if all the tensors in `flat_list` were joined
        and flattened into a single vector, and then the L2 norm of that vector
        was calculated.
      """
      if fastmath.is_backend(fastmath.Backend.JAX):
          norm = jnp.sqrt(sum(jnp.vdot(x, x) for x in flat_list))
      else:  # TODO(lukaszkaiser): add vdot to TF-numpy
          norm = jnp.sqrt(sum(jnp.sum(x * x) for x in flat_list))
      return norm
  
file_path: trax/trax/optimizers/base.py
imports: from trax import fastmath
  from trax.fastmath import numpy
label_desc: Use image node_id 10 for calling the Optimizer obj's underlying member method code with example usage: obj._l2_norm(flat_list) and returns: norm