class_docstr: William's episodic REINFORCE.
  
  Args:
      model (Policy): Model to train. It must be a callable that accepts
          observations as input and return action distributions
          (Distribution).
      optimizer (torch.optim.Optimizer): optimizer used to train the model
      gpu (int): GPU device id if not None nor negative
      beta (float): Weight coefficient for the entropy regularizaiton term.
      phi (callable): Feature extractor function
      act_deterministically (bool): If set true, choose most probable actions
          in act method.
      batchsize (int): Number of episodes used for each update
      backward_separately (bool): If set true, call backward separately for
          each episode and accumulate only gradients.
      average_entropy_decay (float): Decay rate of average entropy. Used only
          to record statistics.
      batch_states (callable): Method which makes a batch of observations.
          default is `pfrl.utils.batch_states`
      recurrent (bool): If set to True, `model` is assumed to implement
          `pfrl.nn.Recurrent` and update in a recurrent
          manner.
      max_grad_norm (float or None): Maximum L2 norm of the gradient used for
          gradient clipping. If set to None, the gradient is not clipped.
      logger (logging.Logger): Logger to be used.
code: def _act_eval(self, obs):
      with torch.no_grad():
          batch_obs = self.batch_states([obs], self.device, self.phi)
          if self.recurrent:
              (
                  action_distrib,
                  self.test_recurrent_states,
              ) = one_step_forward(
                  self.model, batch_obs, self.test_recurrent_states
              )
          else:
              action_distrib = self.model(batch_obs)
          if self.act_deterministically:
              return (
                  mode_of_distribution(action_distrib).cpu().numpy()[0]
              )
          else:
              return action_distrib.sample().cpu().numpy()[0]
  
imports: import warnings
  from logging import getLogger
  import numpy
  import torch
  import pfrl
  from pfrl import agent
  from pfrl.utils import clip_l2_grad_norm_
  from pfrl.utils.mode_of_distribution import mode_of_distribution
  from pfrl.utils.recurrent import one_step_forward
label_desc: Use image node_id 6 for calling the REINFORCE obj's underlying member method code with example usage: obj._act_eval(obs) and returns: mode_of_distribution, action_distrib