docstr: ReplayBuffer for each agent
  
  Args:
      model (parl.Model): model that contains both value network and policy network
      episode_length (int): max length for any episode
      env_num (int): Number of parallel envs to train
      gamma (float): discount factor for rewards
      gae_lambda (float): gae lambda parameter
      obs_shape (int): obs dim for single agent
      share_obs_shape (int): concatenated obs dim for all agents
      act_space (MultiDiscrete/Discrete): action space for single agent
      use_popart (bool): whether to use PopArt to normalize rewards
code: def __init__(
      self,
      episode_length,
      env_num,
      gamma,
      gae_lambda,
      obs_shape,
      share_obs_shape,
      act_space,
      use_popart,
  ):
      """ReplayBuffer for each agent
  
      Args:
          model (parl.Model): model that contains both value network and policy network
          episode_length (int): max length for any episode
          env_num (int): Number of parallel envs to train
          gamma (float): discount factor for rewards
          gae_lambda (float): gae lambda parameter
          obs_shape (int): obs dim for single agent
          share_obs_shape (int): concatenated obs dim for all agents
          act_space (MultiDiscrete/Discrete): action space for single agent
          use_popart (bool): whether to use PopArt to normalize rewards
      """
      self.episode_length = episode_length
      self.env_num = env_num
      self.gamma = gamma
      self.gae_lambda = gae_lambda
      self._use_popart = use_popart
  
      self.share_obs = np.zeros(
          (self.episode_length + 1, self.env_num, share_obs_shape),
          dtype=np.float32,
      )
      self.obs = np.zeros(
          (self.episode_length + 1, self.env_num, obs_shape),
          dtype=np.float32,
      )
      self.value_preds = np.zeros(
          (self.episode_length + 1, self.env_num, 1), dtype=np.float32
      )
      self.returns = np.zeros(
          (self.episode_length + 1, self.env_num, 1), dtype=np.float32
      )
  
      if act_space.__class__.__name__ == "Discrete":
          act_shape = 1
      else:
          act_shape = act_space.shape
  
      self.actions = np.zeros(
          (self.episode_length, self.env_num, act_shape),
          dtype=np.float32,
      )
      self.action_log_probs = np.zeros(
          (self.episode_length, self.env_num, act_shape),
          dtype=np.float32,
      )
      self.rewards = np.zeros(
          (self.episode_length, self.env_num, 1), dtype=np.float32
      )
      self.masks = np.ones(
          (self.episode_length + 1, self.env_num, 1), dtype=np.float32
      )
      self.active_masks = np.ones_like(self.masks)
  
      self.step = 0
  
file_path: PARL/benchmark/torch/mappo/mappo_buffer.py
imports: import torch
  import numpy
label_desc: Use image node_id 1 to create a new SeparatedReplayBuffer object from inherited base classes: object with example: obj = SeparatedReplayBuffer(episode_length, env_num, gamma, gae_lambda, obs_shape, share_obs_shape, act_space, use_popart)