class_docstr: The critic network described in [1], predicting values for policy learning.
  
  Contains a copy of itself (EMA net) for weight regularization.
  The EMA net is updated after each train step via EMA (using the `ema_decay`
  parameter and the actual critic's weights). The EMA net is NOT used for target
  computations (we use the actual critic for that), its only purpose is to compute a
  weights regularizer term for the critic's loss such that the actual critic does not
  move too quickly.
class_docstr_tok: ["The","critic","network","described","in","[","1","]",",","predicting","values","for","policy","learning",".","Contains","a","copy","of","itself","(","EMA","net",")","for","weight","regularization",".","The","EMA","net","is","updated","after","each","train","step","via","EMA","(","using","the","`","ema_decay","`","parameter","and","the","actual","critic","'s","weights",")",".","The","EMA","net","is","NOT","used","for","target","computations","(","we","use","the","actual","critic","for","that",")",",","its","only","purpose","is","to","compute","a","weights","regularizer","term","for","the","critic","'s","loss","such","that","the","actual","critic","does","not","move","too","quickly","."]
docstr: Updates the EMA-copy of the critic according to the update formula:
  
  ema_net=(`ema_decay`*ema_net) + (1.0-`ema_decay`)*critic_net
docstr_tok: ["Updates","the","EMA-copy","of","the","critic","according","to","the","update","formula",":","ema_net=","(","`","ema_decay","`","*","ema_net",")","+","(","1.0-","`","ema_decay","`",")","*","critic_net"]
code: def update_ema(self) -> None:
      """Updates the EMA-copy of the critic according to the update formula:
  
      ema_net=(`ema_decay`*ema_net) + (1.0-`ema_decay`)*critic_net
      """
      vars = (
          self.mlp.trainable_variables
          + self.return_layer.trainable_variables
      )
      vars_ema = (
          self.mlp_ema.variables + self.return_layer_ema.variables
      )
      assert len(vars) == len(vars_ema) and len(vars) > 0
      for var, var_ema in zip(vars, vars_ema):
          var_ema.assign(
              self.ema_decay * var_ema + (1.0 - self.ema_decay) * var
          )
  
file_path: ray/python/ray/rllib/algorithms/dreamerv3/tf/models/critic_network.py
imports: from typing import Optional
  from ray.rllib.algorithms.dreamerv3.tf.models.components.mlp import MLP
  from ray.rllib.algorithms.dreamerv3.tf.models.components.reward_predictor_layer import RewardPredictorLayer
  from ray.rllib.algorithms.dreamerv3.utils import get_gru_units, get_num_z_categoricals, get_num_z_classes
  from ray.rllib.utils.framework import try_import_tf
label_desc: Use image node_id 4 for calling the CriticNetwork obj's underlying member method code with example usage: obj.update_ema() without return types