class_docstr: A wrapper of tianshou PPOPolicy.
  
  Differences:
  
  - Auto-create actor and critic network. Supports discrete action space only.
  - Dedup common parameters between actor network and critic network
    (not sure whether this is included in latest tianshou or not).
  - Support a ``weight_file`` that supports loading checkpoint.
  - Some parameters' default values are different from original.
code: def __init__(
      self,
      network: nn.Module,
      obs_space: gym.Space,
      action_space: gym.Space,
      lr: float,
      weight_decay: float = 0.0,
      discount_factor: float = 1.0,
      max_grad_norm: float = 100.0,
      reward_normalization: bool = True,
      eps_clip: float = 0.3,
      value_clip: bool = True,
      vf_coef: float = 1.0,
      gae_lambda: float = 1.0,
      max_batch_size: int = 256,
      deterministic_eval: bool = True,
      weight_file: Optional[Path] = None,
  ) -> None:
      assert isinstance(action_space, Discrete)
      actor = PPOActor(network, action_space.n)
      critic = PPOCritic(network)
      optimizer = torch.optim.Adam(
          chain_dedup(actor.parameters(), critic.parameters()),
          lr=lr,
          weight_decay=weight_decay,
      )
      super().__init__(
          actor,
          critic,
          optimizer,
          torch.distributions.Categorical,
          discount_factor=discount_factor,
          max_grad_norm=max_grad_norm,
          reward_normalization=reward_normalization,
          eps_clip=eps_clip,
          value_clip=value_clip,
          vf_coef=vf_coef,
          gae_lambda=gae_lambda,
          max_batchsize=max_batch_size,
          deterministic_eval=deterministic_eval,
          observation_space=obs_space,
          action_space=action_space,
      )
      if weight_file is not None:
          set_weight(self, Trainer.get_policy_state_dict(weight_file))
  
imports: from __future__ import annotations
  from pathlib import Path
  from typing import Any, Dict, Generator, Iterable, Optional, OrderedDict, Tuple, cast
  import gym
  import numpy
  import torch
  import torch.nn
  from gym.spaces import Discrete
  from tianshou.data import Batch, ReplayBuffer, to_torch
  from tianshou.policy import BasePolicy, PPOPolicy, DQNPolicy
  from qlib.rl.trainer.trainer import Trainer
label_desc: Use image node_id 1 to create a new PPO object from inherited base classes: PPOPolicy with example: obj = PPO(network, obs_space, action_space, lr, weight_decay, discount_factor, max_grad_norm, reward_normalization, eps_clip, value_clip, vf_coef, gae_lambda, max_batch_size, deterministic_eval, weight_file)