class_docstr: The Catalog class used to build models for PPO.
  
  PPOCatalog provides the following models:
      - ActorCriticEncoder: The encoder used to encode the observations.
      - Pi Head: The head used to compute the policy logits.
      - Value Function Head: The head used to compute the value function.
  
  The ActorCriticEncoder is a wrapper around Encoders to produce separate outputs
  for the policy and value function. See implementations of PPORLModule for
  more details.
  
  Any custom ActorCriticEncoder can be built by overriding the
  build_actor_critic_encoder() method. Alternatively, the ActorCriticEncoderConfig
  at PPOCatalog.actor_critic_encoder_config can be overridden to build a custom
  ActorCriticEncoder during RLModule runtime.
  
  Any custom head can be built by overriding the build_pi_head() and build_vf_head()
  methods. Alternatively, the PiHeadConfig and VfHeadConfig can be overridden to
  build custom heads during RLModule runtime.
class_docstr_tok: ["The","Catalog","class","used","to","build","models","for","PPO",".","PPOCatalog","provides","the","following","models",":","-","ActorCriticEncoder",":","The","encoder","used","to","encode","the","observations",".","-","Pi","Head",":","The","head","used","to","compute","the","policy","logits",".","-","Value","Function","Head",":","The","head","used","to","compute","the","value","function",".","The","ActorCriticEncoder","is","a","wrapper","around","Encoders","to","produce","separate","outputs","for","the","policy","and","value","function",".","See","implementations","of","PPORLModule","for","more","details",".","Any","custom","ActorCriticEncoder","can","be","built","by","overriding","the","build_actor_critic_encoder","(",")","method",".","Alternatively",",","the","ActorCriticEncoderConfig","at","PPOCatalog.actor_critic_encoder_config","can","be","overridden","to","build","a","custom","ActorCriticEncoder","during","RLModule","runtime",".","Any","custom","head","can","be","built","by","overriding","the","build_pi_head","(",")","and","build_vf_head","(",")","methods",".","Alternatively",",","the","PiHeadConfig","and","VfHeadConfig","can","be","overridden","to","build","custom","heads","during","RLModule","runtime","."]
docstr: Builds the policy head.
  
  The default behavior is to build the head from the pi_head_config.
  This can be overridden to build a custom policy head as a means of configuring
  the behavior of a PPORLModule implementation.
  
  Args:
      framework: The framework to use. Either "torch" or "tf2".
  
  Returns:
      The policy head.
docstr_tok: ["Builds","the","policy","head",".","The","default","behavior","is","to","build","the","head","from","the","pi_head_config",".","This","can","be","overridden","to","build","a","custom","policy","head","as","a","means","of","configuring","the","behavior","of","a","PPORLModule","implementation",".","Args",":","framework",":","The","framework","to","use",".","Either","``","torch","''","or","``","tf2","''",".","Returns",":","The","policy","head","."]
code: def build_pi_head(self, framework: str) -> Model:
      """Builds the policy head.
  
      The default behavior is to build the head from the pi_head_config.
      This can be overridden to build a custom policy head as a means of configuring
      the behavior of a PPORLModule implementation.
  
      Args:
          framework: The framework to use. Either "torch" or "tf2".
  
      Returns:
          The policy head.
      """
      # Get action_distribution_cls to find out about the output dimension for pi_head
      action_distribution_cls = self.get_action_dist_cls(
          framework=framework
      )
      if self._model_config_dict["free_log_std"]:
          _check_if_diag_gaussian(
              action_distribution_cls=action_distribution_cls,
              framework=framework,
          )
      required_output_dim = action_distribution_cls.required_input_dim(
          space=self.action_space, model_config=self._model_config_dict
      )
      # Now that we have the action dist class and number of outputs, we can define
      # our pi-config and build the pi head.
      pi_head_config_class = (
          FreeLogStdMLPHeadConfig
          if self._model_config_dict["free_log_std"]
          else MLPHeadConfig
      )
      self.pi_head_config = pi_head_config_class(
          input_dims=self.latent_dims,
          hidden_layer_dims=self.pi_and_vf_head_hiddens,
          hidden_layer_activation=self.pi_and_vf_head_activation,
          output_layer_dim=required_output_dim,
          output_layer_activation="linear",
      )
  
      return self.pi_head_config.build(framework=framework)
  
file_path: ray/rllib/algorithms/ppo/ppo_catalog.py
imports: import gymnasium
  from ray.rllib.core.models.catalog import Catalog
  from ray.rllib.core.models.configs import ActorCriticEncoderConfig, MLPHeadConfig, FreeLogStdMLPHeadConfig
  from ray.rllib.core.models.base import Encoder, ActorCriticEncoder, Model
  from ray.rllib.utils import override
  from ray.rllib.utils.annotations import OverrideToImplementCustomLogic
label_desc: Use image node_id 4 for calling the PPOCatalog obj's underlying member method code with example usage: obj.build_pi_head(framework) and returns: self