class_docstr: Relative Entropy Policy Search.
  
  References
  ----------
  [1] J. Peters, K. Mulling, and Y. Altun, "Relative Entropy Policy Search,"
      Artif. Intell., pp. 1607-1612, 2008.
  
  Example:
      $ python garage/examples/tf/reps_gym_cartpole.py
  
  Args:
      env_spec (EnvSpec): Environment specification.
      policy (garage.tf.policies.StochasticPolicy): Policy.
      baseline (garage.tf.baselines.Baseline): The baseline.
      sampler (garage.sampler.Sampler): Sampler.
      scope (str): Scope for identifying the algorithm.
          Must be specified if running multiple algorithms
          simultaneously, each using different environments
          and policies.
      discount (float): Discount.
      gae_lambda (float): Lambda used for generalized advantage
          estimation.
      center_adv (bool): Whether to rescale the advantages
          so that they have mean 0 and standard deviation 1.
      positive_adv (bool): Whether to shift the advantages
          so that they are always positive. When used in
          conjunction with center_adv the advantages will be
          standardized before shifting.
      fixed_horizon (bool): Whether to fix horizon.
      epsilon (float): Dual func parameter.
      l2_reg_dual (float): Coefficient for dual func l2 regularization.
      l2_reg_loss (float): Coefficient for policy loss l2 regularization.
      optimizer (object): The optimizer of the algorithm. Should be the
          optimizers in garage.tf.optimizers.
      optimizer_args (dict): Arguments of the optimizer.
      dual_optimizer (object): Dual func optimizer.
      dual_optimizer_args (dict): Arguments of the dual optimizer.
      name (str): Name of the algorithm.
docstr: Optimize the policy using the samples.
  
  Args:
      episodes (EpisodeBatch): Batch of episodes.
docstr_tok: ["Optimize","the","policy","using","the","samples",".","Args",":","episodes","(","EpisodeBatch",")",":","Batch","of","episodes","."]
code: def _optimize_policy(self, episodes):
      """Optimize the policy using the samples.
  
      Args:
          episodes (EpisodeBatch): Batch of episodes.
  
      """
      # Initial BFGS parameter values.
      x0 = np.hstack([self._param_eta, self._param_v])
      # Set parameter boundaries: \eta>=1e-12, v unrestricted.
      bounds = [(-np.inf, np.inf) for _ in x0]
      bounds[0] = (1e-12, np.inf)
  
      # Optimize dual
      eta_before = self._param_eta
      logger.log("Computing dual before")
      self._feat_diff = self._features(episodes)
      dual_opt_input_values = self._dual_opt_input_values(episodes)
      dual_before = self._f_dual(*dual_opt_input_values)
      logger.log("Optimizing dual")
  
      def eval_dual(x):
          """Evaluate dual function loss.
  
          Args:
              x (numpy.ndarray): Input to dual function.
  
          Returns:
              numpy.float64: Dual function loss.
  
          """
          self._param_eta = x[0]
          self._param_v = x[1:]
          dual_opt_input_values = self._dual_opt_input_values(episodes)
          return self._f_dual(*dual_opt_input_values)
  
      def eval_dual_grad(x):
          """Evaluate gradient of dual function loss.
  
          Args:
              x (numpy.ndarray): Input to dual function.
  
          Returns:
              numpy.ndarray: Gradient of dual function loss.
  
          """
          self._param_eta = x[0]
          self._param_v = x[1:]
          dual_opt_input_values = self._dual_opt_input_values(episodes)
          grad = self._f_dual_grad(*dual_opt_input_values)
          eta_grad = np.float(grad[0])
          v_grad = grad[1]
          return np.hstack([eta_grad, v_grad])
  
      params_ast, _, _ = self._dual_optimizer(
          func=eval_dual,
          x0=x0,
          fprime=eval_dual_grad,
          bounds=bounds,
          **self._dual_optimizer_args,
      )
  
      logger.log("Computing dual after")
      self._param_eta, self._param_v = params_ast[0], params_ast[1:]
      dual_opt_input_values = self._dual_opt_input_values(episodes)
      dual_after = self._f_dual(*dual_opt_input_values)
  
      # Optimize policy
      policy_opt_input_values = self._policy_opt_input_values(episodes)
      logger.log("Computing policy loss before")
      loss_before = self._optimizer.loss(policy_opt_input_values)
      logger.log("Computing policy KL before")
      policy_kl_before = self._f_policy_kl(*policy_opt_input_values)
      logger.log("Optimizing policy")
      self._optimizer.optimize(policy_opt_input_values)
      logger.log("Computing policy KL")
      policy_kl = self._f_policy_kl(*policy_opt_input_values)
      logger.log("Computing policy loss after")
      loss_after = self._optimizer.loss(policy_opt_input_values)
      tabular.record("EtaBefore", eta_before)
      tabular.record("EtaAfter", self._param_eta)
      tabular.record("DualBefore", dual_before)
      tabular.record("DualAfter", dual_after)
      tabular.record(
          "{}/LossBefore".format(self.policy.name), loss_before
      )
      tabular.record(
          "{}/LossAfter".format(self.policy.name), loss_after
      )
      tabular.record(
          "{}/dLoss".format(self.policy.name), loss_before - loss_after
      )
      tabular.record(
          "{}/KLBefore".format(self.policy.name), policy_kl_before
      )
      tabular.record("{}/KL".format(self.policy.name), policy_kl)
  
      self._old_policy.parameters = self.policy.parameters
  
imports: import collections
  from dowel import logger, tabular
  import numpy
  import scipy.optimize
  import tensorflow
  from garage import _Default, log_performance, make_optimizer
  from garage.np import pad_batch_array
  from garage.np.algos import RLAlgorithm
  from garage.tf import compile_function, flatten_inputs, graph_inputs, new_tensor
  from garage.tf.optimizers import LBFGSOptimizer
label_desc: Use image node_id 7 for calling the REPS obj's underlying member method code with example usage: obj._optimize_policy(episodes) and returns: self, np