class_docstr: Policy Loss for PPO
code: def forward(
      self,
      log_probs: torch.Tensor,
      old_log_probs: torch.Tensor,
      advantages: torch.Tensor,
      action_mask: Optional[torch.Tensor] = None,
  ) -> torch.Tensor:
      ratio = (log_probs - old_log_probs).exp()
      surr1 = ratio * advantages
      surr2 = (
          ratio.clamp(1 - self.clip_eps, 1 + self.clip_eps) * advantages
      )
      loss = -torch.min(surr1, surr2)
      if action_mask is not None:
          loss = masked_mean(loss, action_mask)
      loss = loss.mean()
      return loss
  
imports: from typing import Optional
  import torch
  import torch.nn
  from .utils import masked_mean
label_desc: Use image node_id 2 for calling the PolicyLoss obj's underlying member method code with example usage: obj.forward(log_probs, old_log_probs, advantages, action_mask) and returns: loss