class_docstr: PyTorch Lightning implementation of PPO.
  
  Example:
      model = PPOLightning("CartPole-v0")
  Train:
      trainer = Trainer()
      trainer.fit(model)
docstr: Contains the logic for generating trajectory data to train policy and value network
  Yield:
     Tuple of Lists containing tensors for states, actions, log probs, qvals and advantage
code: def generate_trajectory_samples(
      self,
  ) -> Tuple[
      List[torch.Tensor], List[torch.Tensor], List[torch.Tensor]
  ]:
      """
      Contains the logic for generating trajectory data to train policy and value network
      Yield:
         Tuple of Lists containing tensors for states, actions, log probs, qvals and advantage
      """
      for step in range(self.steps_per_epoch):
          self.state = self.state.to(device=self.device)
  
          with torch.no_grad():
              pi, action, value = self(self.state)
              log_prob = self.actor.get_log_prob(pi, action)
  
          next_state, reward, done, _ = self.env.step(
              action.cpu().numpy()
          )
  
          self.episode_step += 1
  
          self.batch_states.append(self.state)
          self.batch_actions.append(action)
          self.batch_logp.append(log_prob)
  
          self.ep_rewards.append(reward)
          self.ep_values.append(value.item())
  
          self.state = torch.FloatTensor(next_state)
  
          epoch_end = step == (self.steps_per_epoch - 1)
          terminal = len(self.ep_rewards) == self.max_episode_len
  
          if epoch_end or done or terminal:
              # if trajectory ends abtruptly, bootstrap value of next state
              if (terminal or epoch_end) and not done:
                  self.state = self.state.to(device=self.device)
                  with torch.no_grad():
                      _, _, value = self(self.state)
                      last_value = value.item()
                      steps_before_cutoff = self.episode_step
              else:
                  last_value = 0
                  steps_before_cutoff = 0
  
              # discounted cumulative reward
              self.batch_qvals += self.discount_rewards(
                  self.ep_rewards + [last_value], self.gamma
              )[:-1]
              # advantage
              self.batch_adv += self.calc_advantage(
                  self.ep_rewards, self.ep_values, last_value
              )
              # logs
              self.epoch_rewards.append(sum(self.ep_rewards))
              # reset params
              self.ep_rewards = []
              self.ep_values = []
              self.episode_step = 0
              self.state = torch.FloatTensor(self.env.reset())
  
          if epoch_end:
              train_data = zip(
                  self.batch_states,
                  self.batch_actions,
                  self.batch_logp,
                  self.batch_qvals,
                  self.batch_adv,
              )
  
              for state, action, logp_old, qval, adv in train_data:
                  yield state, action, logp_old, qval, adv
  
              self.batch_states.clear()
              self.batch_actions.clear()
              self.batch_adv.clear()
              self.batch_logp.clear()
              self.batch_qvals.clear()
  
              # logging
              self.avg_reward = (
                  sum(self.epoch_rewards) / self.steps_per_epoch
              )
  
              # if epoch ended abruptly, exlude last cut-short episode to prevent stats skewness
              epoch_rewards = self.epoch_rewards
              if not done:
                  epoch_rewards = epoch_rewards[:-1]
  
              total_epoch_reward = sum(epoch_rewards)
              nb_episodes = len(epoch_rewards)
  
              self.avg_ep_reward = total_epoch_reward / nb_episodes
              self.avg_ep_len = (
                  self.steps_per_epoch - steps_before_cutoff
              ) / nb_episodes
  
              self.epoch_rewards.clear()
  
imports: import argparse
  from typing import Callable, Iterator, List, Tuple
  import gym
  import torch
  from lightning.pytorch import LightningModule, Trainer, cli_lightning_logo, seed_everything
  from torch import nn
  from torch.distributions import Categorical, Normal
  from torch.optim.optimizer import Optimizer
  from torch.utils.data import DataLoader, IterableDataset
label_desc: Use image node_id 5 for calling the PPOLightning obj's underlying member method code with example usage: obj.generate_trajectory_samples() without return types