class_docstr: Natural Policy Optimization with Task Embeddings.
  
  See https://karolhausman.github.io/pdf/hausman17nips-ws2.pdf for algorithm
  reference.
  
  Args:
      env_spec (EnvSpec): Environment specification.
      policy (garage.tf.policies.TaskEmbeddingPolicy): Policy.
      baseline (garage.tf.baselines.Baseline): The baseline.
      sampler (garage.sampler.Sampler): Sampler.
      scope (str): Scope for identifying the algorithm.
          Must be specified if running multiple algorithms
          simultaneously, each using different environments
          and policies.
      discount (float): Discount.
      gae_lambda (float): Lambda used for generalized advantage
          estimation.
      center_adv (bool): Whether to rescale the advantages
          so that they have mean 0 and standard deviation 1.
      positive_adv (bool): Whether to shift the advantages
          so that they are always positive. When used in
          conjunction with center_adv the advantages will be
          standardized before shifting.
      fixed_horizon (bool): Whether to fix horizon.
      lr_clip_range (float): The limit on the likelihood ratio between
          policies, as in PPO.
      max_kl_step (float): The maximum KL divergence between old and new
          policies, as in TRPO.
      optimizer (object): The optimizer of the algorithm. Should be the
          optimizers in garage.tf.optimizers.
      optimizer_args (dict): The arguments of the optimizer.
      policy_ent_coeff (float): The coefficient of the policy entropy.
          Setting it to zero would mean no entropy regularization.
      encoder_ent_coeff (float): The coefficient of the policy encoder
          entropy. Setting it to zero would mean no entropy regularization.
      use_softplus_entropy (bool): Whether to estimate the softmax
          distribution of the entropy to prevent the entropy from being
          negative.
      stop_ce_gradient (bool): Whether to stop the cross entropy gradient.
      inference (garage.tf.embeddings.StochasticEncoder): A encoder
          that infers the task embedding from state trajectory.
      inference_optimizer (object): The optimizer of the inference. Should be
          an optimizer in garage.tf.optimizers.
      inference_optimizer_args (dict): The arguments of the inference
          optimizer.
      inference_ce_coeff (float): The coefficient of the cross entropy of
          task embeddings inferred from task one-hot and state trajectory.
          This is effectively the coefficient of log-prob of inference.
      name (str): The name of the algorithm.
code: def __init__(
      self,
      env_spec,
      policy,
      baseline,
      sampler,
      scope=None,
      discount=0.99,
      gae_lambda=1,
      center_adv=True,
      positive_adv=False,
      fixed_horizon=False,
      lr_clip_range=0.01,
      max_kl_step=0.01,
      optimizer=None,
      optimizer_args=None,
      policy_ent_coeff=0.0,
      encoder_ent_coeff=0.0,
      use_softplus_entropy=False,
      stop_ce_gradient=False,
      inference=None,
      inference_optimizer=None,
      inference_optimizer_args=None,
      inference_ce_coeff=0.0,
      name="NPOTaskEmbedding",
  ):
      # pylint: disable=too-many-statements
      assert isinstance(policy, TaskEmbeddingPolicy)
      assert isinstance(inference, StochasticEncoder)
  
      self.policy = policy
      self._scope = scope
      self.max_episode_length = env_spec.max_episode_length
  
      self._env_spec = env_spec
      self._baseline = baseline
      self._discount = discount
      self._gae_lambda = gae_lambda
      self._center_adv = center_adv
      self._positive_adv = positive_adv
      self._fixed_horizon = fixed_horizon
      self._name = name
      self._name_scope = tf.name_scope(self._name)
      self._old_policy = policy.clone("old_policy")
      self._use_softplus_entropy = use_softplus_entropy
      self._stop_ce_gradient = stop_ce_gradient
  
      optimizer = optimizer or LBFGSOptimizer
      optimizer_args = optimizer_args or dict()
  
      inference_opt = inference_optimizer or LBFGSOptimizer
      inference_opt_args = inference_optimizer_args or dict()
  
      with self._name_scope:
          self._optimizer = optimizer(**optimizer_args)
          self._lr_clip_range = float(lr_clip_range)
          self._max_kl_step = float(max_kl_step)
          self._policy_ent_coeff = float(policy_ent_coeff)
  
          self._inference = inference
          self._old_inference = inference.clone("old_inference")
          self.inference_ce_coeff = float(inference_ce_coeff)
          self.inference_optimizer = inference_opt(**inference_opt_args)
          self.encoder_ent_coeff = encoder_ent_coeff
  
      self._f_rewards = None
      self._f_returns = None
      self._f_policy_kl = None
      self._f_policy_entropy = None
      self._f_encoder_kl = None
      self._f_encoder_entropy = None
      self._f_task_entropies = None
      self._f_inference_ce = None
      self._policy_network = None
      self._old_policy_network = None
      self._encoder_network = None
      self._old_encoder_network = None
      self._infer_network = None
      self._old_infer_network = None
  
      self._sampler = sampler
  
      self._init_opt()
  
imports: import akro
  from dowel import Histogram, logger, tabular
  import numpy
  import scipy.stats
  import tensorflow
  from garage import InOutSpec, log_performance
  from garage.experiment import deterministic
  from garage.np import discount_cumsum, explained_variance_1d, pad_batch_array, rrse, sliding_window
  from garage.np.algos import RLAlgorithm
  from garage.tf import center_advs, compile_function, compute_advantages, concat_tensor_list, discounted_returns, flatten_inputs, graph_inputs, pad_tensor_dict, positive_advs, stack_tensor_dict_list
  from garage.tf.embeddings import StochasticEncoder
  from garage.tf.optimizers import LBFGSOptimizer
  from garage.tf.policies import TaskEmbeddingPolicy
label_desc: Use image node_id 1 to create a new TENPO object from inherited base classes: RLAlgorithm with example: obj = TENPO(env_spec, policy, baseline, sampler, scope, discount, gae_lambda, center_adv, positive_adv, fixed_horizon, lr_clip_range, max_kl_step, optimizer, optimizer_args, policy_ent_coeff, encoder_ent_coeff, use_softplus_entropy, stop_ce_gradient, inference, inference_optimizer, inference_optimizer_args, inference_ce_coeff, name)