class_docstr: PCL (Path Consistency Learning).
  
  Not only the batch PCL algorithm proposed in the paper but also its
  asynchronous variant is implemented.
  
  See https://arxiv.org/abs/1702.08892
  
  Args:
      model (chainer.Link): Model to train. It must be a callable that
          accepts a batch of observations as input and return two values:
  
          - action distributions (Distribution)
          - state values (chainer.Variable)
      optimizer (chainer.Optimizer): optimizer used to train the model
      t_max (int or None): The model is updated after every t_max local
          steps. If set None, the model is updated after every episode.
      gamma (float): Discount factor [0,1]
      tau (float): Weight coefficient for the entropy regularizaiton term.
      phi (callable): Feature extractor function
      pi_loss_coef (float): Weight coefficient for the loss of the policy
      v_loss_coef (float): Weight coefficient for the loss of the value
          function
      rollout_len (int): Number of rollout steps
      batchsize (int): Number of episodes or sub-trajectories used for an
          update. The total number of transitions used will be
          (batchsize x t_max).
      disable_online_update (bool): If set true, disable online on-policy
          update and rely only on experience replay.
      n_times_replay (int): Number of times experience replay is repeated per
          one time of online update.
      replay_start_size (int): Experience replay is disabled if the number of
          transitions in the replay buffer is lower than this value.
      normalize_loss_by_steps (bool): If set true, losses are normalized by
          the number of steps taken to accumulate the losses
      act_deterministically (bool): If set true, choose most probable actions
          in act method.
      average_loss_decay (float): Decay rate of average loss. Used only
          to record statistics.
      average_entropy_decay (float): Decay rate of average entropy. Used only
          to record statistics.
      average_value_decay (float): Decay rate of average value. Used only
          to record statistics.
      explorer (Explorer or None): If not None, this explorer is used for
          selecting actions.
      logger (None or Logger): Logger to be used
      batch_states (callable): Method which makes a batch of observations.
          default is `chainerrl.misc.batch_states.batch_states`
      backprop_future_values (bool): If set True, value gradients are
          computed not only wrt V(s_t) but also V(s_{t+d}).
      train_async (bool): If set True, use a process-local model to compute
          gradients and update the globally shared model.
code: def compute_loss(
      self, t_start, t_stop, rewards, values, next_values, log_probs
  ):
      seq_len = t_stop - t_start
      assert len(rewards) == seq_len
      assert len(values) == seq_len
      assert len(next_values) == seq_len
      assert len(log_probs) == seq_len
  
      pi_losses = []
      v_losses = []
      for t in range(t_start, t_stop):
          d = min(t_stop - t, self.rollout_len)
          # Discounted sum of immediate rewards
          R_seq = sum(
              self.gamma**i * rewards[t + i] for i in range(d)
          )
          # Discounted sum of log likelihoods
          G = chainerrl.functions.weighted_sum_arrays(
              xs=[log_probs[t + i] for i in range(d)],
              weights=[self.gamma**i for i in range(d)],
          )
          G = F.expand_dims(G, -1)
          last_v = next_values[t + d - 1]
          if not self.backprop_future_values:
              last_v = chainer.Variable(last_v.array)
  
          # C_pi only backprop through pi
          C_pi = (
              -values[t].array
              + self.gamma**d * last_v.array
              + R_seq
              - self.tau * G
          )
  
          # C_v only backprop through v
          C_v = (
              -values[t]
              + self.gamma**d * last_v
              + R_seq
              - self.tau * G.array
          )
  
          pi_losses.append(C_pi**2)
          v_losses.append(C_v**2)
  
      pi_loss = chainerrl.functions.sum_arrays(pi_losses) / 2
      v_loss = chainerrl.functions.sum_arrays(v_losses) / 2
  
      # Re-scale pi loss so that it is independent from tau
      pi_loss /= self.tau
  
      pi_loss *= self.pi_loss_coef
      v_loss *= self.v_loss_coef
  
      if self.normalize_loss_by_steps:
          pi_loss /= seq_len
          v_loss /= seq_len
  
      if self.process_idx == 0:
          self.logger.debug(
              "pi_loss:%s v_loss:%s", pi_loss.array, v_loss.array
          )
  
      return pi_loss + F.reshape(v_loss, pi_loss.array.shape)
  
imports: import copy
  from logging import getLogger
  import chainer
  from chainer import functions
  import chainerrl
  from chainerrl import agent
  from chainerrl.agents import a3c
  from chainerrl.misc import async_
  from chainerrl.misc.batch_states import batch_states
  from chainerrl.misc import copy_param
  from chainerrl.recurrent import Recurrent
  from chainerrl.recurrent import state_kept
  from chainerrl.recurrent import state_reset
  from chainerrl.replay_buffer import batch_experiences
label_desc: Use image node_id 4 for calling the PCL obj's underlying member method code with example usage: obj.compute_loss(t_start, t_stop, rewards, values, next_values, log_probs) and returns: unknown