class_docstr: A PEARL model based on https://arxiv.org/abs/1903.08254.
  
  PEARL, which stands for Probablistic Embeddings for Actor-Critic
  Reinforcement Learning, is an off-policy meta-RL algorithm. It is built
  on top of SAC using two Q-functions and a value function with an addition
  of an inference network that estimates the posterior :math:`q(z \| c)`.
  The policy is conditioned on the latent variable Z in order to adpat its
  behavior to specific tasks.
  
  Args:
      env (list[Environment]): Batch of sampled environment updates(
          EnvUpdate), which, when invoked on environments, will configure
          them with new tasks.
      policy_class (type): Class implementing
          :pyclass:`~ContextConditionedPolicy`
      encoder_class (garage.torch.embeddings.ContextEncoder): Encoder class
          for the encoder in context-conditioned policy.
      inner_policy (garage.torch.policies.Policy): Policy.
      qf (torch.nn.Module): Q-function.
      vf (torch.nn.Module): Value function.
      sampler (garage.sampler.Sampler): Sampler.
      num_train_tasks (int): Number of tasks for training.
      num_test_tasks (int or None): Number of tasks for testing.
      latent_dim (int): Size of latent context vector.
      encoder_hidden_sizes (list[int]): Output dimension of dense layer(s) of
          the context encoder.
      test_env_sampler (garage.experiment.SetTaskSampler): Sampler for test
          tasks.
      policy_lr (float): Policy learning rate.
      qf_lr (float): Q-function learning rate.
      vf_lr (float): Value function learning rate.
      context_lr (float): Inference network learning rate.
      policy_mean_reg_coeff (float): Policy mean regulation weight.
      policy_std_reg_coeff (float): Policy std regulation weight.
      policy_pre_activation_coeff (float): Policy pre-activation weight.
      soft_target_tau (float): Interpolation parameter for doing the
          soft target update.
      kl_lambda (float): KL lambda value.
      optimizer_class (type): Type of optimizer for training networks.
      use_information_bottleneck (bool): False means latent context is
          deterministic.
      use_next_obs_in_context (bool): Whether or not to use next observation
          in distinguishing between tasks.
      meta_batch_size (int): Meta batch size.
      num_steps_per_epoch (int): Number of iterations per epoch.
      num_initial_steps (int): Number of transitions obtained per task before
          training.
      num_tasks_sample (int): Number of random tasks to obtain data for each
          iteration.
      num_steps_prior (int): Number of transitions to obtain per task with
          z ~ prior.
      num_steps_posterior (int): Number of transitions to obtain per task
          with z ~ posterior.
      num_extra_rl_steps_posterior (int): Number of additional transitions
          to obtain per task with z ~ posterior that are only used to train
          the policy and NOT the encoder.
      batch_size (int): Number of transitions in RL batch.
      embedding_batch_size (int): Number of transitions in context batch.
      embedding_mini_batch_size (int): Number of transitions in mini context
          batch; should be same as embedding_batch_size for non-recurrent
          encoder.
      discount (float): RL discount factor.
      replay_buffer_size (int): Maximum samples in replay buffer.
      reward_scale (int): Reward scale.
      update_post_train (int): How often to resample context when obtaining
          data during training (in episodes).
docstr: Obtain samples, train, and evaluate for each epoch.
  
  Args:
      trainer (Trainer): Gives the algorithm the access to
          :method:`Trainer..step_epochs()`, which provides services
          such as snapshotting and sampler control.
code: def train(self, trainer):
      """Obtain samples, train, and evaluate for each epoch.
  
      Args:
          trainer (Trainer): Gives the algorithm the access to
              :method:`Trainer..step_epochs()`, which provides services
              such as snapshotting and sampler control.
  
      """
      for _ in trainer.step_epochs():
          epoch = trainer.step_itr / self._num_steps_per_epoch
  
          # obtain initial set of samples from all train tasks
          if epoch == 0 or self._is_resuming:
              for idx in range(self._num_train_tasks):
                  self._task_idx = idx
                  self._obtain_samples(
                      trainer, epoch, self._num_initial_steps, np.inf
                  )
                  self._is_resuming = False
  
          # obtain samples from random tasks
          for _ in range(self._num_tasks_sample):
              idx = np.random.randint(self._num_train_tasks)
              self._task_idx = idx
              self._context_replay_buffers[idx].clear()
              # obtain samples with z ~ prior
              if self._num_steps_prior > 0:
                  self._obtain_samples(
                      trainer, epoch, self._num_steps_prior, np.inf
                  )
              # obtain samples with z ~ posterior
              if self._num_steps_posterior > 0:
                  self._obtain_samples(
                      trainer,
                      epoch,
                      self._num_steps_posterior,
                      self._update_post_train,
                  )
              # obtain extras samples for RL training but not encoder
              if self._num_extra_rl_steps_posterior > 0:
                  self._obtain_samples(
                      trainer,
                      epoch,
                      self._num_extra_rl_steps_posterior,
                      self._update_post_train,
                      add_to_enc_buffer=False,
                  )
  
          logger.log("Training...")
          # sample train tasks and optimize networks
          self._train_once()
          trainer.step_itr += 1
  
          logger.log("Evaluating...")
          # evaluate
          self._policy.reset_belief()
          self._evaluator.evaluate(self)
  
imports: import copy
  import akro
  from dowel import logger
  import numpy
  import torch
  from garage import EnvSpec, InOutSpec, StepType, TimeStep
  from garage.experiment import MetaEvaluator
  from garage.np.algos import MetaRLAlgorithm
  from garage.replay_buffer import PathBuffer
  from garage.sampler import DefaultWorker
  from garage.torch import global_device
  from garage.torch._functions import np_to_torch, zero_optim_grads
  from garage.torch.embeddings import MLPEncoder
  from garage.torch.policies import ContextConditionedPolicy
label_desc: Use image node_id 4 for calling the PEARL obj's underlying member method code with example usage: obj.train(trainer) without return types