class_docstr: A policy for taking actions within an environment.
  
  A policy is defined by a tf.keras.Model that takes the current state as input
  and performs the necessary calculations.  There are many algorithms for
  reinforcement learning, and they differ in what values they require a policy to
  compute.  That makes it impossible to define a single interface allowing any
  policy to be optimized with any algorithm.  Instead, this interface just tries
  to be as flexible and generic as possible.  Each algorithm must document what
  values it expects the model to output.
  
  Special handling is needed for models that include recurrent layers.  In that
  case, the model has its own internal state which the learning algorithm must
  be able to specify and query.  To support this, the Policy must do three things:
  
  1. The Model must take additional inputs that specify the initial states of
      all its recurrent layers.  These will be appended to the list of arrays
      specifying the environment state.
  
  2. The Model must also return the final states of all its recurrent layers as
      outputs.
  
  3. The constructor argument rnn_initial_states must be specified to define
      the states to use for the Model's recurrent layers at the start of a new
      rollout.
  
  Policy objects should be written to support pickling.  Many algorithms involve
  creating multiple copies of the Policy, possibly running in different processes
  or even on different computers.
class_docstr_tok: ["A","policy","for","taking","actions","within","an","environment",".","A","policy","is","defined","by","a","tf.keras.Model","that","takes","the","current","state","as","input","and","performs","the","necessary","calculations",".","There","are","many","algorithms","for","reinforcement","learning",",","and","they","differ","in","what","values","they","require","a","policy","to","compute",".","That","makes","it","impossible","to","define","a","single","interface","allowing","any","policy","to","be","optimized","with","any","algorithm",".","Instead",",","this","interface","just","tries","to","be","as","flexible","and","generic","as","possible",".","Each","algorithm","must","document","what","values","it","expects","the","model","to","output",".","Special","handling","is","needed","for","models","that","include","recurrent","layers",".","In","that","case",",","the","model","has","its","own","internal","state","which","the","learning","algorithm","must","be","able","to","specify","and","query",".","To","support","this",",","the","Policy","must","do","three","things",":","1",".","The","Model","must","take","additional","inputs","that","specify","the","initial","states","of","all","its","recurrent","layers",".","These","will","be","appended","to","the","list","of","arrays","specifying","the","environment","state",".","2",".","The","Model","must","also","return","the","final","states","of","all","its","recurrent","layers","as","outputs",".","3",".","The","constructor","argument","rnn_initial_states","must","be","specified","to","define","the","states","to","use","for","the","Model","'s","recurrent","layers","at","the","start","of","a","new","rollout",".","Policy","objects","should","be","written","to","support","pickling",".","Many","algorithms","involve","creating","multiple","copies","of","the","Policy",",","possibly","running","in","different","processes","or","even","on","different","computers","."]
docstr: Construct and return a tf.keras.Model that computes the policy.
  
  The inputs to the model consist of the arrays representing the current state
  of the environment, followed by the initial states for all recurrent layers.
  Depending on the algorithm being used, other inputs might get passed as
  well.  It is up to each algorithm to document that.
docstr_tok: ["Construct","and","return","a","tf.keras.Model","that","computes","the","policy",".","The","inputs","to","the","model","consist","of","the","arrays","representing","the","current","state","of","the","environment",",","followed","by","the","initial","states","for","all","recurrent","layers",".","Depending","on","the","algorithm","being","used",",","other","inputs","might","get","passed","as","well",".","It","is","up","to","each","algorithm","to","document","that","."]
code: def create_model(self, **kwargs):
      """Construct and return a tf.keras.Model that computes the policy.
  
      The inputs to the model consist of the arrays representing the current state
      of the environment, followed by the initial states for all recurrent layers.
      Depending on the algorithm being used, other inputs might get passed as
      well.  It is up to each algorithm to document that.
      """
      raise NotImplementedError("Subclasses must implement this")
  
label_desc: Use image node_id 2 for calling the Policy obj's underlying member method code with example usage: obj.create_model() without return types