class_docstr: Extension of the standard TFModelV2 for SAC.
  
  To customize, do one of the following:
  - sub-class SACTFModel and override one or more of its methods.
  - Use SAC's `q_model_config` and `policy_model` keys to tweak the default model
    behaviors (e.g. fcnet_hiddens, conv_filters, etc..).
  - Use SAC's `q_model_config->custom_model` and `policy_model->custom_model` keys
    to specify your own custom Q-model(s) and policy-models, which will be
    created within this SACTFModel (see `build_policy_model` and
    `build_q_model`.
  
  Note: It is not recommended to override the `forward` method for SAC. This
  would lead to shared weights (between policy and Q-nets), which will then
  not be optimized by either of the critic- or actor-optimizers!
  
  Data flow:
      `obs` -> forward() (should stay a noop method!) -> `model_out`
      `model_out` -> get_policy_output() -> pi(actions|obs)
      `model_out`, `actions` -> get_q_values() -> Q(s, a)
      `model_out`, `actions` -> get_twin_q_values() -> Q_twin(s, a)
docstr: Initialize a SACTFModel instance.
  
  Args:
      policy_model_config: The config dict for the
          policy network.
      q_model_config: The config dict for the
          Q-network(s) (2 if twin_q=True).
      twin_q: Build twin Q networks (Q-net and target) for more
          stable Q-learning.
      initial_alpha: The initial value for the to-be-optimized
          alpha parameter (default: 1.0).
      target_entropy (Optional[float]): A target entropy value for
          the to-be-optimized alpha parameter. If None, will use the
          defaults described in the papers for SAC (and discrete SAC).
  
  Note that the core layers for forward() are not defined here, this
  only defines the layers for the output heads. Those layers for
  forward() should be defined in subclasses of SACModel.
docstr_tok: ["Initialize","a","SACTFModel","instance",".","Args",":","policy_model_config",":","The","config","dict","for","the","policy","network",".","q_model_config",":","The","config","dict","for","the","Q-network","(","s",")","(","2","if","twin_q=True",")",".","twin_q",":","Build","twin","Q","networks","(","Q-net","and","target",")","for","more","stable","Q-learning",".","initial_alpha",":","The","initial","value","for","the","to-be-optimized","alpha","parameter","(","default",":","1.0",")",".","target_entropy","(","Optional","[","float","]",")",":","A","target","entropy","value","for","the","to-be-optimized","alpha","parameter",".","If","None",",","will","use","the","defaults","described","in","the","papers","for","SAC","(","and","discrete","SAC",")",".","Note","that","the","core","layers","for","forward","(",")","are","not","defined","here",",","this","only","defines","the","layers","for","the","output","heads",".","Those","layers","for","forward","(",")","should","be","defined","in","subclasses","of","SACModel","."]
code: def __init__(
      self,
      obs_space: gym.spaces.Space,
      action_space: gym.spaces.Space,
      num_outputs: Optional[int],
      model_config: ModelConfigDict,
      name: str,
      policy_model_config: ModelConfigDict = None,
      q_model_config: ModelConfigDict = None,
      twin_q: bool = False,
      initial_alpha: float = 1.0,
      target_entropy: Optional[float] = None,
  ):
      """Initialize a SACTFModel instance.
  
      Args:
          policy_model_config: The config dict for the
              policy network.
          q_model_config: The config dict for the
              Q-network(s) (2 if twin_q=True).
          twin_q: Build twin Q networks (Q-net and target) for more
              stable Q-learning.
          initial_alpha: The initial value for the to-be-optimized
              alpha parameter (default: 1.0).
          target_entropy (Optional[float]): A target entropy value for
              the to-be-optimized alpha parameter. If None, will use the
              defaults described in the papers for SAC (and discrete SAC).
  
      Note that the core layers for forward() are not defined here, this
      only defines the layers for the output heads. Those layers for
      forward() should be defined in subclasses of SACModel.
      """
      super(SACTFModel, self).__init__(
          obs_space, action_space, num_outputs, model_config, name
      )
      if isinstance(action_space, Discrete):
          self.action_dim = action_space.n
          self.discrete = True
          action_outs = q_outs = self.action_dim
      elif isinstance(action_space, Box):
          self.action_dim = np.product(action_space.shape)
          self.discrete = False
          action_outs = 2 * self.action_dim
          q_outs = 1
      else:
          assert isinstance(action_space, Simplex)
          self.action_dim = np.product(action_space.shape)
          self.discrete = False
          action_outs = self.action_dim
          q_outs = 1
  
      self.action_model = self.build_policy_model(
          self.obs_space,
          action_outs,
          policy_model_config,
          "policy_model",
      )
  
      self.q_net = self.build_q_model(
          self.obs_space, self.action_space, q_outs, q_model_config, "q"
      )
      if twin_q:
          self.twin_q_net = self.build_q_model(
              self.obs_space,
              self.action_space,
              q_outs,
              q_model_config,
              "twin_q",
          )
      else:
          self.twin_q_net = None
  
      self.log_alpha = tf.Variable(
          np.log(initial_alpha), dtype=tf.float32, name="log_alpha"
      )
      self.alpha = tf.exp(self.log_alpha)
  
      # Auto-calculate the target entropy.
      if target_entropy is None or target_entropy == "auto":
          # See hyperparams in [2] (README.md).
          if self.discrete:
              target_entropy = 0.98 * np.array(
                  -np.log(1.0 / action_space.n), dtype=np.float32
              )
          # See [1] (README.md).
          else:
              target_entropy = -np.prod(action_space.shape)
      self.target_entropy = target_entropy
  
imports: import gymnasium
  from gymnasium.spaces import Box, Discrete
  import numpy
  import tree
  from typing import Dict, List, Optional
  from ray.rllib.models.catalog import ModelCatalog
  from ray.rllib.models.tf.tf_modelv2 import TFModelV2
  from ray.rllib.utils.annotations import override
  from ray.rllib.utils.framework import try_import_tf
  from ray.rllib.utils.spaces.simplex import Simplex
  from ray.rllib.utils.typing import ModelConfigDict, TensorType, TensorStructType
label_desc: Use image node_id 1 to create a new SACTFModel object from inherited base classes: TFModelV2 with example: obj = SACTFModel(obs_space, action_space, num_outputs, model_config, name, policy_model_config, q_model_config, twin_q, initial_alpha, target_entropy)