class_docstr: MPO loss with decoupled KL constraints as in (Abdolmaleki et al., 2018).
  
  This implementation of the MPO loss includes the following features, as
  options:
  - Satisfying the KL-constraint on a per-dimension basis (on by default);
  - Penalizing actions that fall outside of [-1, 1] (on by default) as a
      special case of multi-objective MPO (MO-MPO; Abdolmaleki et al., 2020).
  For best results on the control suite, keep both of these on.
  
  (Abdolmaleki et al., 2018): https://arxiv.org/pdf/1812.02256.pdf
  (Abdolmaleki et al., 2020): https://arxiv.org/pdf/2005.07513.pdf
class_docstr_tok: ["MPO","loss","with","decoupled","KL","constraints","as","in","(","Abdolmaleki","et","al.",",","2018",")",".","This","implementation","of","the","MPO","loss","includes","the","following","features",",","as","options",":","-","Satisfying","the","KL-constraint","on","a","per-dimension","basis","(","on","by","default",")",";","-","Penalizing","actions","that","fall","outside","of","[","-1",",","1","]","(","on","by","default",")","as","a","special","case","of","multi-objective","MPO","(","MO-MPO",";","Abdolmaleki","et","al.",",","2020",")",".","For","best","results","on","the","control","suite",",","keep","both","of","these","on",".","(","Abdolmaleki","et","al.",",","2018",")",":","https",":","\/\/arxiv.org\/pdf\/1812.02256.pdf","(","Abdolmaleki","et","al.",",","2020",")",":","https",":","\/\/arxiv.org\/pdf\/2005.07513.pdf"]
docstr: Creates the dual variables the first time the loss module is called.
docstr_tok: ["Creates","the","dual","variables","the","first","time","the","loss","module","is","called","."]
code: def create_dual_variables_once(
      self, shape: tf.TensorShape, dtype: tf.DType
  ):
      """Creates the dual variables the first time the loss module is called."""
  
      # Create the dual variables.
      self._log_temperature = tf.Variable(
          initial_value=[self._init_log_temperature],
          dtype=dtype,
          name="log_temperature",
          shape=(1,),
      )
      self._log_alpha_mean = tf.Variable(
          initial_value=tf.fill(shape, self._init_log_alpha_mean),
          dtype=dtype,
          name="log_alpha_mean",
          shape=shape,
      )
      self._log_alpha_stddev = tf.Variable(
          initial_value=tf.fill(shape, self._init_log_alpha_stddev),
          dtype=dtype,
          name="log_alpha_stddev",
          shape=shape,
      )
  
      # Cast constraint thresholds to the expected dtype.
      self._epsilon = tf.cast(self._epsilon, dtype)
      self._epsilon_mean = tf.cast(self._epsilon_mean, dtype)
      self._epsilon_stddev = tf.cast(self._epsilon_stddev, dtype)
  
      # Maybe create the action penalization dual variable.
      if self._action_penalization:
          self._epsilon_penalty = tf.cast(self._epsilon_penalty, dtype)
          self._log_penalty_temperature = tf.Variable(
              initial_value=[self._init_log_temperature],
              dtype=dtype,
              name="log_penalty_temperature",
              shape=(1,),
          )
  
file_path: acme/acme/tf/losses/mpo.py
imports: from typing import Dict, Tuple, Union
  import sonnet
  import tensorflow
  import tensorflow_probability
label_desc: Use image node_id 2 for calling the MPO obj's underlying member method code with example usage: obj.create_dual_variables_once(shape, dtype) without return types