class_docstr: A TFPolicy that auto-defines placeholders dynamically at runtime.
  
  Do not sub-class this class directly (neither should you sub-class
  TFPolicy), but rather use rllib.policy.tf_policy_template.build_tf_policy
  to generate your custom tf (graph-mode or eager) Policy classes.
docstr: Initializes a DynamicTFPolicy instance.
  
  Initialization of this class occurs in two phases and defines the
  static graph.
  
  Phase 1: The model is created and model variables are initialized.
  
  Phase 2: A fake batch of data is created, sent to the trajectory
  postprocessor, and then used to create placeholders for the loss
  function. The loss and stats functions are initialized with these
  placeholders.
  
  Args:
      observation_space: Observation space of the policy.
      action_space: Action space of the policy.
      config: Policy-specific configuration data.
      loss_fn: Function that returns a loss tensor for the policy graph.
      stats_fn: Optional callable that - given the policy and batch
          input tensors - returns a dict mapping str to TF ops.
          These ops are fetched from the graph after loss calculations
          and the resulting values can be found in the results dict
          returned by e.g. `Algorithm.train()` or in tensorboard (if TB
          logging is enabled).
      grad_stats_fn: Optional callable that - given the policy, batch
          input tensors, and calculated loss gradient tensors - returns
          a dict mapping str to TF ops. These ops are fetched from the
          graph after loss and gradient calculations and the resulting
          values can be found in the results dict returned by e.g.
          `Algorithm.train()` or in tensorboard (if TB logging is
          enabled).
      before_loss_init: Optional function to run prior to
          loss init that takes the same arguments as __init__.
      make_model: Optional function that returns a ModelV2 object
          given policy, obs_space, action_space, and policy config.
          All policy variables should be created in this function. If not
          specified, a default model will be created.
      action_sampler_fn: A callable returning either a sampled action and
          its log-likelihood or a sampled action, its log-likelihood,
          action distribution inputs and updated state given Policy,
          ModelV2, observation inputs, explore, and is_training.
          Provide `action_sampler_fn` if you would like to have full
          control over the action computation step, including the
          model forward pass, possible sampling from a distribution,
          and exploration logic.
          Note: If `action_sampler_fn` is given, `action_distribution_fn`
          must be None. If both `action_sampler_fn` and
          `action_distribution_fn` are None, RLlib will simply pass
          inputs through `self.model` to get distribution inputs, create
          the distribution object, sample from it, and apply some
          exploration logic to the results.
          The callable takes as inputs: Policy, ModelV2, obs_batch,
          state_batches (optional), seq_lens (optional),
          prev_actions_batch (optional), prev_rewards_batch (optional),
          explore, and is_training.
      action_distribution_fn: A callable returning distribution inputs
          (parameters), a dist-class to generate an action distribution
          object from, and internal-state outputs (or an empty list if
          not applicable).
          Provide `action_distribution_fn` if you would like to only
          customize the model forward pass call. The resulting
          distribution parameters are then used by RLlib to create a
          distribution object, sample from it, and execute any
          exploration logic.
          Note: If `action_distribution_fn` is given, `action_sampler_fn`
          must be None. If both `action_sampler_fn` and
          `action_distribution_fn` are None, RLlib will simply pass
          inputs through `self.model` to get distribution inputs, create
          the distribution object, sample from it, and apply some
          exploration logic to the results.
          The callable takes as inputs: Policy, ModelV2, input_dict,
          explore, timestep, is_training.
      existing_inputs: When copying a policy, this specifies an existing
          dict of placeholders to use instead of defining new ones.
      existing_model: When copying a policy, this specifies an existing
          model to clone and share weights with.
      get_batch_divisibility_req: Optional callable that returns the
          divisibility requirement for sample batches. If None, will
          assume a value of 1.
docstr_tok: ["Initializes","a","DynamicTFPolicy","instance",".","Initialization","of","this","class","occurs","in","two","phases","and","defines","the","static","graph",".","Phase","1",":","The","model","is","created","and","model","variables","are","initialized",".","Phase","2",":","A","fake","batch","of","data","is","created",",","sent","to","the","trajectory","postprocessor",",","and","then","used","to","create","placeholders","for","the","loss","function",".","The","loss","and","stats","functions","are","initialized","with","these","placeholders",".","Args",":","observation_space",":","Observation","space","of","the","policy",".","action_space",":","Action","space","of","the","policy",".","config",":","Policy-specific","configuration","data",".","loss_fn",":","Function","that","returns","a","loss","tensor","for","the","policy","graph",".","stats_fn",":","Optional","callable","that","-","given","the","policy","and","batch","input","tensors","-","returns","a","dict","mapping","str","to","TF","ops",".","These","ops","are","fetched","from","the","graph","after","loss","calculations","and","the","resulting","values","can","be","found","in","the","results","dict","returned","by","e.g",".","`","Algorithm.train","(",")","`","or","in","tensorboard","(","if","TB","logging","is","enabled",")",".","grad_stats_fn",":","Optional","callable","that","-","given","the","policy",",","batch","input","tensors",",","and","calculated","loss","gradient","tensors","-","returns","a","dict","mapping","str","to","TF","ops",".","These","ops","are","fetched","from","the","graph","after","loss","and","gradient","calculations","and","the","resulting","values","can","be","found","in","the","results","dict","returned","by","e.g",".","`","Algorithm.train","(",")","`","or","in","tensorboard","(","if","TB","logging","is","enabled",")",".","before_loss_init",":","Optional","function","to","run","prior","to","loss","init","that","takes","the","same","arguments","as","__init__",".","make_model",":","Optional","function","that","returns","a","ModelV2","object","given","policy",",","obs_space",",","action_space",",","and","policy","config",".","All","policy","variables","should","be","created","in","this","function",".","If","not","specified",",","a","default","model","will","be","created",".","action_sampler_fn",":","A","callable","returning","either","a","sampled","action","and","its","log-likelihood","or","a","sampled","action",",","its","log-likelihood",",","action","distribution","inputs","and","updated","state","given","Policy",",","ModelV2",",","observation","inputs",",","explore",",","and","is_training",".","Provide","`","action_sampler_fn","`","if","you","would","like","to","have","full","control","over","the","action","computation","step",",","including","the","model","forward","pass",",","possible","sampling","from","a","distribution",",","and","exploration","logic",".","Note",":","If","`","action_sampler_fn","`","is","given",",","`","action_distribution_fn","`","must","be","None",".","If","both","`","action_sampler_fn","`","and","`","action_distribution_fn","`","are","None",",","RLlib","will","simply","pass","inputs","through","`","self.model","`","to","get","distribution","inputs",",","create","the","distribution","object",",","sample","from","it",",","and","apply","some","exploration","logic","to","the","results",".","The","callable","takes","as","inputs",":","Policy",",","ModelV2",",","obs_batch",",","state_batches","(","optional",")",",","seq_lens","(","optional",")",",","prev_actions_batch","(","optional",")",",","prev_rewards_batch","(","optional",")",",","explore",",","and","is_training",".","action_distribution_fn",":","A","callable","returning","distribution","inputs","(","parameters",")",",","a","dist-class","to","generate","an","action","distribution","object","from",",","and","internal-state","outputs","(","or","an","empty","list","if","not","applicable",")",".","Provide","`","action_distribution_fn","`","if","you","would","like","to","only","customize","the","model","forward","pass","call",".","The","resulting","distribution","parameters","are","then","used","by","RLlib","to","create","a","distribution","object",",","sample","from","it",",","and","execute","any","exploration","logic",".","Note",":","If","`","action_distribution_fn","`","is","given",",","`","action_sampler_fn","`","must","be","None",".","If","both","`","action_sampler_fn","`","and","`","action_distribution_fn","`","are","None",",","RLlib","will","simply","pass","inputs","through","`","self.model","`","to","get","distribution","inputs",",","create","the","distribution","object",",","sample","from","it",",","and","apply","some","exploration","logic","to","the","results",".","The","callable","takes","as","inputs",":","Policy",",","ModelV2",",","input_dict",",","explore",",","timestep",",","is_training",".","existing_inputs",":","When","copying","a","policy",",","this","specifies","an","existing","dict","of","placeholders","to","use","instead","of","defining","new","ones",".","existing_model",":","When","copying","a","policy",",","this","specifies","an","existing","model","to","clone","and","share","weights","with",".","get_batch_divisibility_req",":","Optional","callable","that","returns","the","divisibility","requirement","for","sample","batches",".","If","None",",","will","assume","a","value","of","1","."]
code: def __init__(
      self,
      obs_space: gym.spaces.Space,
      action_space: gym.spaces.Space,
      config: AlgorithmConfigDict,
      loss_fn: Callable[
          [Policy, ModelV2, Type[TFActionDistribution], SampleBatch],
          TensorType,
      ],
      *,
      stats_fn: Optional[
          Callable[[Policy, SampleBatch], Dict[str, TensorType]]
      ] = None,
      grad_stats_fn: Optional[
          Callable[
              [Policy, SampleBatch, ModelGradients],
              Dict[str, TensorType],
          ]
      ] = None,
      before_loss_init: Optional[
          Callable[
              [
                  Policy,
                  gym.spaces.Space,
                  gym.spaces.Space,
                  AlgorithmConfigDict,
              ],
              None,
          ]
      ] = None,
      make_model: Optional[
          Callable[
              [
                  Policy,
                  gym.spaces.Space,
                  gym.spaces.Space,
                  AlgorithmConfigDict,
              ],
              ModelV2,
          ]
      ] = None,
      action_sampler_fn: Optional[
          Callable[
              [TensorType, List[TensorType]],
              Union[
                  Tuple[TensorType, TensorType],
                  Tuple[
                      TensorType,
                      TensorType,
                      TensorType,
                      List[TensorType],
                  ],
              ],
          ]
      ] = None,
      action_distribution_fn: Optional[
          Callable[
              [Policy, ModelV2, TensorType, TensorType, TensorType],
              Tuple[TensorType, type, List[TensorType]],
          ]
      ] = None,
      existing_inputs: Optional[Dict[str, "tf1.placeholder"]] = None,
      existing_model: Optional[ModelV2] = None,
      get_batch_divisibility_req: Optional[
          Callable[[Policy], int]
      ] = None,
      obs_include_prev_action_reward=DEPRECATED_VALUE,
  ):
      """Initializes a DynamicTFPolicy instance.
  
      Initialization of this class occurs in two phases and defines the
      static graph.
  
      Phase 1: The model is created and model variables are initialized.
  
      Phase 2: A fake batch of data is created, sent to the trajectory
      postprocessor, and then used to create placeholders for the loss
      function. The loss and stats functions are initialized with these
      placeholders.
  
      Args:
          observation_space: Observation space of the policy.
          action_space: Action space of the policy.
          config: Policy-specific configuration data.
          loss_fn: Function that returns a loss tensor for the policy graph.
          stats_fn: Optional callable that - given the policy and batch
              input tensors - returns a dict mapping str to TF ops.
              These ops are fetched from the graph after loss calculations
              and the resulting values can be found in the results dict
              returned by e.g. `Algorithm.train()` or in tensorboard (if TB
              logging is enabled).
          grad_stats_fn: Optional callable that - given the policy, batch
              input tensors, and calculated loss gradient tensors - returns
              a dict mapping str to TF ops. These ops are fetched from the
              graph after loss and gradient calculations and the resulting
              values can be found in the results dict returned by e.g.
              `Algorithm.train()` or in tensorboard (if TB logging is
              enabled).
          before_loss_init: Optional function to run prior to
              loss init that takes the same arguments as __init__.
          make_model: Optional function that returns a ModelV2 object
              given policy, obs_space, action_space, and policy config.
              All policy variables should be created in this function. If not
              specified, a default model will be created.
          action_sampler_fn: A callable returning either a sampled action and
              its log-likelihood or a sampled action, its log-likelihood,
              action distribution inputs and updated state given Policy,
              ModelV2, observation inputs, explore, and is_training.
              Provide `action_sampler_fn` if you would like to have full
              control over the action computation step, including the
              model forward pass, possible sampling from a distribution,
              and exploration logic.
              Note: If `action_sampler_fn` is given, `action_distribution_fn`
              must be None. If both `action_sampler_fn` and
              `action_distribution_fn` are None, RLlib will simply pass
              inputs through `self.model` to get distribution inputs, create
              the distribution object, sample from it, and apply some
              exploration logic to the results.
              The callable takes as inputs: Policy, ModelV2, obs_batch,
              state_batches (optional), seq_lens (optional),
              prev_actions_batch (optional), prev_rewards_batch (optional),
              explore, and is_training.
          action_distribution_fn: A callable returning distribution inputs
              (parameters), a dist-class to generate an action distribution
              object from, and internal-state outputs (or an empty list if
              not applicable).
              Provide `action_distribution_fn` if you would like to only
              customize the model forward pass call. The resulting
              distribution parameters are then used by RLlib to create a
              distribution object, sample from it, and execute any
              exploration logic.
              Note: If `action_distribution_fn` is given, `action_sampler_fn`
              must be None. If both `action_sampler_fn` and
              `action_distribution_fn` are None, RLlib will simply pass
              inputs through `self.model` to get distribution inputs, create
              the distribution object, sample from it, and apply some
              exploration logic to the results.
              The callable takes as inputs: Policy, ModelV2, input_dict,
              explore, timestep, is_training.
          existing_inputs: When copying a policy, this specifies an existing
              dict of placeholders to use instead of defining new ones.
          existing_model: When copying a policy, this specifies an existing
              model to clone and share weights with.
          get_batch_divisibility_req: Optional callable that returns the
              divisibility requirement for sample batches. If None, will
              assume a value of 1.
      """
      if obs_include_prev_action_reward != DEPRECATED_VALUE:
          deprecation_warning(
              old="obs_include_prev_action_reward", error=True
          )
      self.observation_space = obs_space
      self.action_space = action_space
      self.config = config
      self.framework = "tf"
      self._loss_fn = loss_fn
      self._stats_fn = stats_fn
      self._grad_stats_fn = grad_stats_fn
      self._seq_lens = None
      self._is_tower = existing_inputs is not None
  
      dist_class = None
      if action_sampler_fn or action_distribution_fn:
          if not make_model:
              raise ValueError(
                  "`make_model` is required if `action_sampler_fn` OR "
                  "`action_distribution_fn` is given"
              )
      else:
          dist_class, logit_dim = ModelCatalog.get_action_dist(
              action_space, self.config["model"]
          )
  
      # Setup self.model.
      if existing_model:
          if isinstance(existing_model, list):
              self.model = existing_model[0]
              # TODO: (sven) hack, but works for `target_[q_]?model`.
              for i in range(1, len(existing_model)):
                  setattr(
                      self, existing_model[i][0], existing_model[i][1]
                  )
      elif make_model:
          self.model = make_model(self, obs_space, action_space, config)
      else:
          self.model = ModelCatalog.get_model_v2(
              obs_space=obs_space,
              action_space=action_space,
              num_outputs=logit_dim,
              model_config=self.config["model"],
              framework="tf",
          )
      # Auto-update model's inference view requirements, if recurrent.
      self._update_model_view_requirements_from_init_state()
  
      # Input placeholders already given -> Use these.
      if existing_inputs:
          self._state_inputs = [
              v
              for k, v in existing_inputs.items()
              if k.startswith("state_in_")
          ]
          # Placeholder for RNN time-chunk valid lengths.
          if self._state_inputs:
              self._seq_lens = existing_inputs[SampleBatch.SEQ_LENS]
      # Create new input placeholders.
      else:
          self._state_inputs = [
              get_placeholder(
                  space=vr.space,
                  time_axis=not isinstance(vr.shift, int),
                  name=k,
              )
              for k, vr in self.model.view_requirements.items()
              if k.startswith("state_in_")
          ]
          # Placeholder for RNN time-chunk valid lengths.
          if self._state_inputs:
              self._seq_lens = tf1.placeholder(
                  dtype=tf.int32, shape=[None], name="seq_lens"
              )
  
      # Use default settings.
      # Add NEXT_OBS, STATE_IN_0.., and others.
      self.view_requirements = self._get_default_view_requirements()
      # Combine view_requirements for Model and Policy.
      self.view_requirements.update(self.model.view_requirements)
      # Disable env-info placeholder.
      if SampleBatch.INFOS in self.view_requirements:
          self.view_requirements[
              SampleBatch.INFOS
          ].used_for_training = False
  
      # Setup standard placeholders.
      if self._is_tower:
          timestep = existing_inputs["timestep"]
          explore = False
          (
              self._input_dict,
              self._dummy_batch,
          ) = self._get_input_dict_and_dummy_batch(
              self.view_requirements, existing_inputs
          )
      else:
          if not self.config.get("_disable_action_flattening"):
              action_ph = ModelCatalog.get_action_placeholder(
                  action_space
              )
              prev_action_ph = {}
              if SampleBatch.PREV_ACTIONS not in self.view_requirements:
                  prev_action_ph = {
                      SampleBatch.PREV_ACTIONS: ModelCatalog.get_action_placeholder(
                          action_space, "prev_action"
                      )
                  }
              (
                  self._input_dict,
                  self._dummy_batch,
              ) = self._get_input_dict_and_dummy_batch(
                  self.view_requirements,
                  dict(
                      {SampleBatch.ACTIONS: action_ph}, **prev_action_ph
                  ),
              )
          else:
              (
                  self._input_dict,
                  self._dummy_batch,
              ) = self._get_input_dict_and_dummy_batch(
                  self.view_requirements, {}
              )
          # Placeholder for (sampling steps) timestep (int).
          timestep = tf1.placeholder_with_default(
              tf.zeros((), dtype=tf.int64), (), name="timestep"
          )
          # Placeholder for `is_exploring` flag.
          explore = tf1.placeholder_with_default(
              True, (), name="is_exploring"
          )
  
      # Placeholder for `is_training` flag.
      self._input_dict.set_training(self._get_is_training_placeholder())
  
      # Multi-GPU towers do not need any action computing/exploration
      # graphs.
      sampled_action = None
      sampled_action_logp = None
      dist_inputs = None
      extra_action_fetches = {}
      self._state_out = None
      if not self._is_tower:
          # Create the Exploration object to use for this Policy.
          self.exploration = self._create_exploration()
  
          # Fully customized action generation (e.g., custom policy).
          if action_sampler_fn:
              action_sampler_outputs = action_sampler_fn(
                  self,
                  self.model,
                  obs_batch=self._input_dict[SampleBatch.CUR_OBS],
                  state_batches=self._state_inputs,
                  seq_lens=self._seq_lens,
                  prev_action_batch=self._input_dict.get(
                      SampleBatch.PREV_ACTIONS
                  ),
                  prev_reward_batch=self._input_dict.get(
                      SampleBatch.PREV_REWARDS
                  ),
                  explore=explore,
                  is_training=self._input_dict.is_training,
              )
              if len(action_sampler_outputs) == 4:
                  (
                      sampled_action,
                      sampled_action_logp,
                      dist_inputs,
                      self._state_out,
                  ) = action_sampler_outputs
              else:
                  dist_inputs = None
                  self._state_out = []
                  (
                      sampled_action,
                      sampled_action_logp,
                  ) = action_sampler_outputs
          # Distribution generation is customized, e.g., DQN, DDPG.
          else:
              if action_distribution_fn:
                  # Try new action_distribution_fn signature, supporting
                  # state_batches and seq_lens.
                  in_dict = self._input_dict
                  try:
                      (
                          dist_inputs,
                          dist_class,
                          self._state_out,
                      ) = action_distribution_fn(
                          self,
                          self.model,
                          input_dict=in_dict,
                          state_batches=self._state_inputs,
                          seq_lens=self._seq_lens,
                          explore=explore,
                          timestep=timestep,
                          is_training=in_dict.is_training,
                      )
                  # Trying the old way (to stay backward compatible).
                  # TODO: Remove in future.
                  except TypeError as e:
                      if (
                          "positional argument" in e.args[0]
                          or "unexpected keyword argument" in e.args[0]
                      ):
                          (
                              dist_inputs,
                              dist_class,
                              self._state_out,
                          ) = action_distribution_fn(
                              self,
                              self.model,
                              obs_batch=in_dict[SampleBatch.CUR_OBS],
                              state_batches=self._state_inputs,
                              seq_lens=self._seq_lens,
                              prev_action_batch=in_dict.get(
                                  SampleBatch.PREV_ACTIONS
                              ),
                              prev_reward_batch=in_dict.get(
                                  SampleBatch.PREV_REWARDS
                              ),
                              explore=explore,
                              is_training=in_dict.is_training,
                          )
                      else:
                          raise e
  
              # Default distribution generation behavior:
              # Pass through model. E.g., PG, PPO.
              else:
                  if isinstance(self.model, tf.keras.Model):
                      (
                          dist_inputs,
                          self._state_out,
                          extra_action_fetches,
                      ) = self.model(self._input_dict)
                  else:
                      dist_inputs, self._state_out = self.model(
                          self._input_dict
                      )
  
              action_dist = dist_class(dist_inputs, self.model)
  
              # Using exploration to get final action (e.g. via sampling).
              (
                  sampled_action,
                  sampled_action_logp,
              ) = self.exploration.get_exploration_action(
                  action_distribution=action_dist,
                  timestep=timestep,
                  explore=explore,
              )
  
      if dist_inputs is not None:
          extra_action_fetches[
              SampleBatch.ACTION_DIST_INPUTS
          ] = dist_inputs
  
      if sampled_action_logp is not None:
          extra_action_fetches[
              SampleBatch.ACTION_LOGP
          ] = sampled_action_logp
          extra_action_fetches[SampleBatch.ACTION_PROB] = tf.exp(
              tf.cast(sampled_action_logp, tf.float32)
          )
  
      # Phase 1 init.
      sess = tf1.get_default_session() or tf1.Session(
          config=tf1.ConfigProto(**self.config["tf_session_args"])
      )
  
      batch_divisibility_req = (
          get_batch_divisibility_req(self)
          if callable(get_batch_divisibility_req)
          else (get_batch_divisibility_req or 1)
      )
  
      prev_action_input = (
          self._input_dict[SampleBatch.PREV_ACTIONS]
          if SampleBatch.PREV_ACTIONS in self._input_dict.accessed_keys
          else None
      )
      prev_reward_input = (
          self._input_dict[SampleBatch.PREV_REWARDS]
          if SampleBatch.PREV_REWARDS in self._input_dict.accessed_keys
          else None
      )
  
      super().__init__(
          observation_space=obs_space,
          action_space=action_space,
          config=config,
          sess=sess,
          obs_input=self._input_dict[SampleBatch.OBS],
          action_input=self._input_dict[SampleBatch.ACTIONS],
          sampled_action=sampled_action,
          sampled_action_logp=sampled_action_logp,
          dist_inputs=dist_inputs,
          dist_class=dist_class,
          loss=None,  # dynamically initialized on run
          loss_inputs=[],
          model=self.model,
          state_inputs=self._state_inputs,
          state_outputs=self._state_out,
          prev_action_input=prev_action_input,
          prev_reward_input=prev_reward_input,
          seq_lens=self._seq_lens,
          max_seq_len=config["model"]["max_seq_len"],
          batch_divisibility_req=batch_divisibility_req,
          explore=explore,
          timestep=timestep,
      )
  
      # Phase 2 init.
      if before_loss_init is not None:
          before_loss_init(self, obs_space, action_space, config)
      if hasattr(self, "_extra_action_fetches"):
          self._extra_action_fetches.update(extra_action_fetches)
      else:
          self._extra_action_fetches = extra_action_fetches
  
      # Loss initialization and model/postprocessing test calls.
      if not self._is_tower:
          self._initialize_loss_from_dummy_batch(
              auto_remove_unneeded_view_reqs=True
          )
  
          # Create MultiGPUTowerStacks, if we have at least one actual
          # GPU or >1 CPUs (fake GPUs).
          if len(self.devices) > 1 or any(
              "gpu" in d for d in self.devices
          ):
              # Per-GPU graph copies created here must share vars with the
              # policy. Therefore, `reuse` is set to tf1.AUTO_REUSE because
              # Adam nodes are created after all of the device copies are
              # created.
              with tf1.variable_scope("", reuse=tf1.AUTO_REUSE):
                  self.multi_gpu_tower_stacks = [
                      TFMultiGPUTowerStack(policy=self)
                      for i in range(
                          self.config.get(
                              "num_multi_gpu_tower_stacks", 1
                          )
                      )
                  ]
  
          # Initialize again after loss and tower init.
          self.get_session().run(tf1.global_variables_initializer())
  
imports: from collections import namedtuple, OrderedDict
  import gymnasium
  import logging
  import re
  import tree
  from typing import Callable, Dict, List, Optional, Tuple, Type, Union
  from ray.util.debug import log_once
  from ray.rllib.models.tf.tf_action_dist import TFActionDistribution
  from ray.rllib.models.modelv2 import ModelV2
  from ray.rllib.policy.policy import Policy
  from ray.rllib.policy.sample_batch import SampleBatch
  from ray.rllib.policy.tf_policy import TFPolicy
  from ray.rllib.policy.view_requirement import ViewRequirement
  from ray.rllib.models.catalog import ModelCatalog
  from ray.rllib.utils import force_list
  from ray.rllib.utils.annotations import override, DeveloperAPI
  from ray.rllib.utils.debug import summarize
  from ray.rllib.utils.deprecation import deprecation_warning, DEPRECATED_VALUE
  from ray.rllib.utils.framework import try_import_tf
  from ray.rllib.utils.metrics import DIFF_NUM_GRAD_UPDATES_VS_SAMPLER_POLICY, NUM_GRAD_UPDATES_LIFETIME
  from ray.rllib.utils.spaces.space_utils import get_dummy_batch_for_space
  from ray.rllib.utils.tf_utils import get_placeholder
  from ray.rllib.utils.typing import LocalOptimizer, ModelGradients, TensorType, AlgorithmConfigDict
label_desc: Use image node_id 1 to create a new DynamicTFPolicy object from inherited base classes: TFPolicy with example: obj = DynamicTFPolicy(obs_space, action_space, config, loss_fn)