class_docstr: A policy for taking actions within an environment.
  
  A policy is defined by a tf.keras.Model that takes the current state as input
  and performs the necessary calculations.  There are many algorithms for
  reinforcement learning, and they differ in what values they require a policy to
  compute.  That makes it impossible to define a single interface allowing any
  policy to be optimized with any algorithm.  Instead, this interface just tries
  to be as flexible and generic as possible.  Each algorithm must document what
  values it expects the model to output.
  
  Special handling is needed for models that include recurrent layers.  In that
  case, the model has its own internal state which the learning algorithm must
  be able to specify and query.  To support this, the Policy must do three things:
  
  1. The Model must take additional inputs that specify the initial states of
      all its recurrent layers.  These will be appended to the list of arrays
      specifying the environment state.
  
  2. The Model must also return the final states of all its recurrent layers as
      outputs.
  
  3. The constructor argument rnn_initial_states must be specified to define
      the states to use for the Model's recurrent layers at the start of a new
      rollout.
  
  Policy objects should be written to support pickling.  Many algorithms involve
  creating multiple copies of the Policy, possibly running in different processes
  or even on different computers.
class_docstr_tok: ["A","policy","for","taking","actions","within","an","environment",".","A","policy","is","defined","by","a","tf.keras.Model","that","takes","the","current","state","as","input","and","performs","the","necessary","calculations",".","There","are","many","algorithms","for","reinforcement","learning",",","and","they","differ","in","what","values","they","require","a","policy","to","compute",".","That","makes","it","impossible","to","define","a","single","interface","allowing","any","policy","to","be","optimized","with","any","algorithm",".","Instead",",","this","interface","just","tries","to","be","as","flexible","and","generic","as","possible",".","Each","algorithm","must","document","what","values","it","expects","the","model","to","output",".","Special","handling","is","needed","for","models","that","include","recurrent","layers",".","In","that","case",",","the","model","has","its","own","internal","state","which","the","learning","algorithm","must","be","able","to","specify","and","query",".","To","support","this",",","the","Policy","must","do","three","things",":","1",".","The","Model","must","take","additional","inputs","that","specify","the","initial","states","of","all","its","recurrent","layers",".","These","will","be","appended","to","the","list","of","arrays","specifying","the","environment","state",".","2",".","The","Model","must","also","return","the","final","states","of","all","its","recurrent","layers","as","outputs",".","3",".","The","constructor","argument","rnn_initial_states","must","be","specified","to","define","the","states","to","use","for","the","Model","'s","recurrent","layers","at","the","start","of","a","new","rollout",".","Policy","objects","should","be","written","to","support","pickling",".","Many","algorithms","involve","creating","multiple","copies","of","the","Policy",",","possibly","running","in","different","processes","or","even","on","different","computers","."]
docstr: Subclasses should call the superclass constructor in addition to doing
  their own initialization.
  
  Parameters
  ----------
  output_names: list of strings
      the names of the Model's outputs, in order.  It is up to each reinforcement
      learning algorithm to document what outputs it expects policies to compute.
      Outputs that return the final states of recurrent layers should have the
      name 'rnn_state'.
  rnn_initial_states: list of NumPy arrays
      the initial states of the Model's recurrent layers at the start of a new
      rollout
docstr_tok: ["Subclasses","should","call","the","superclass","constructor","in","addition","to","doing","their","own","initialization",".","Parameters","--","--","--","--","--","output_names",":","list","of","strings","the","names","of","the","Model","'s","outputs",",","in","order",".","It","is","up","to","each","reinforcement","learning","algorithm","to","document","what","outputs","it","expects","policies","to","compute",".","Outputs","that","return","the","final","states","of","recurrent","layers","should","have","the","name","'rnn_state","'",".","rnn_initial_states",":","list","of","NumPy","arrays","the","initial","states","of","the","Model","'s","recurrent","layers","at","the","start","of","a","new","rollout"]
code: def __init__(self, output_names, rnn_initial_states=[]):
      """Subclasses should call the superclass constructor in addition to doing
      their own initialization.
  
      Parameters
      ----------
      output_names: list of strings
          the names of the Model's outputs, in order.  It is up to each reinforcement
          learning algorithm to document what outputs it expects policies to compute.
          Outputs that return the final states of recurrent layers should have the
          name 'rnn_state'.
      rnn_initial_states: list of NumPy arrays
          the initial states of the Model's recurrent layers at the start of a new
          rollout
      """
      self.output_names = output_names
      self.rnn_initial_states = rnn_initial_states
  
label_desc: Use image node_id 1 to create a new Policy object from inherited base classes: object with example: obj = Policy(output_names, rnn_initial_states)