class_docstr: PyTorch Lightning implementation of PPO.
  
  Example:
      model = PPOLightning("CartPole-v0")
  Train:
      trainer = Trainer()
      trainer.fit(model)
docstr: Args:
      env: gym environment tag
      gamma: discount factor
      lam: advantage discount factor (lambda in the paper)
      lr_actor: learning rate of actor network
      lr_critic: learning rate of critic network
      max_episode_len: maximum number interactions (actions) in an episode
      batch_size:  batch_size when training network- can simulate number of policy updates performed per epoch
      steps_per_epoch: how many action-state pairs to rollout for trajectory collection per epoch
      nb_optim_iters: how many steps of gradient descent to perform on each batch
      clip_ratio: hyperparameter for clipping in the policy objective
code: def __init__(
      self,
      env: str,
      gamma: float = 0.99,
      lam: float = 0.95,
      lr_actor: float = 3e-4,
      lr_critic: float = 1e-3,
      max_episode_len: float = 200,
      batch_size: int = 512,
      steps_per_epoch: int = 2048,
      nb_optim_iters: int = 4,
      clip_ratio: float = 0.2,
      **kwargs,
  ) -> None:
      """
      Args:
          env: gym environment tag
          gamma: discount factor
          lam: advantage discount factor (lambda in the paper)
          lr_actor: learning rate of actor network
          lr_critic: learning rate of critic network
          max_episode_len: maximum number interactions (actions) in an episode
          batch_size:  batch_size when training network- can simulate number of policy updates performed per epoch
          steps_per_epoch: how many action-state pairs to rollout for trajectory collection per epoch
          nb_optim_iters: how many steps of gradient descent to perform on each batch
          clip_ratio: hyperparameter for clipping in the policy objective
      """
      super().__init__()
  
      # Hyperparameters
      self.lr_actor = lr_actor
      self.lr_critic = lr_critic
      self.steps_per_epoch = steps_per_epoch
      self.nb_optim_iters = nb_optim_iters
      self.batch_size = batch_size
      self.gamma = gamma
      self.lam = lam
      self.max_episode_len = max_episode_len
      self.clip_ratio = clip_ratio
      self.save_hyperparameters()
  
      self.automatic_optimization = False
  
      self.env = gym.make(env)
      # value network
      self.critic = create_mlp(self.env.observation_space.shape, 1)
      # policy network (agent)
      if isinstance(self.env.action_space, gym.spaces.box.Box):
          act_dim = self.env.action_space.shape[0]
          actor_mlp = create_mlp(
              self.env.observation_space.shape, act_dim
          )
          self.actor = ActorContinuous(actor_mlp, act_dim)
      elif isinstance(
          self.env.action_space, gym.spaces.discrete.Discrete
      ):
          actor_mlp = create_mlp(
              self.env.observation_space.shape, self.env.action_space.n
          )
          self.actor = ActorCategorical(actor_mlp)
      else:
          raise NotImplementedError(
              "Env action space should be of type Box (continuous) or Discrete (categorical)."
              f" Got type: {type(self.env.action_space)}"
          )
  
      self.batch_states = []
      self.batch_actions = []
      self.batch_adv = []
      self.batch_qvals = []
      self.batch_logp = []
  
      self.ep_rewards = []
      self.ep_values = []
      self.epoch_rewards = []
  
      self.episode_step = 0
      self.avg_ep_reward = 0
      self.avg_ep_len = 0
      self.avg_reward = 0
  
      self.state = torch.FloatTensor(self.env.reset())
  
imports: import argparse
  from typing import Callable, Iterator, List, Tuple
  import gym
  import torch
  from lightning.pytorch import LightningModule, Trainer, cli_lightning_logo, seed_everything
  from torch import nn
  from torch.distributions import Categorical, Normal
  from torch.optim.optimizer import Optimizer
  from torch.utils.data import DataLoader, IterableDataset
label_desc: Use image node_id 1 to create a new PPOLightning object from inherited base classes: LightningModule with example: obj = PPOLightning(env, gamma, lam, lr_actor, lr_critic, max_episode_len, batch_size, steps_per_epoch, nb_optim_iters, clip_ratio)