class_docstr: Ranking agent class.
docstr: Initializes an instance of RankingAgent.
  
  Args:
    time_step_spec: A `TimeStep` spec of the expected time_steps.
    action_spec: A nest of `BoundedTensorSpec` representing the actions.
    scoring_network: The network that outputs scores for items.
    optimizer: The optimizer for the agent.
    policy_type: The type of policy used. The only available type at this
      moment is COSINE_DISTANCE, that invokes the
      `PenalizeCosineDistanceRankingPolicy`. This policy uses the cosine
      similarity to penalize the scores of not yet selected items. If set to
      `UNKNOWN`, falls back to `COSINE_DISTANCE`.
    error_loss_fn: The loss function used.
    feedback_model: The type of feedback model. Implemented models are:
      --CASCADING : the feedback is a tuple `(k, v)`, where `k` is the index
      of the chosen item, and `v` is the value of the choice. If no item was
      chosen, then `k=num_slots` is used and `v` is ignored. --SCORE_VECTOR :
      the feedback is a vector of length `num_slots`, containing scores for
      every item in the recommendation. If set to `UNKNOWN`, falls back to
      CASCADING`.
    non_click_score: (float) For the cascading feedback model, this is the
      score value for items lying "before" the clicked item. If not set, -1 is
      used. It is recommended (but not enforced) to use a negative value.
    positional_bias_type: (string) If not set (or set to `None`), the agent
      does not apply bias adjustment. If set to either `base` or `exponent`,
      it parameter determines what way the positional bias is accounted for.
      `base`: The bias weight for each slot position is `k^s`, where `s` is
      the bias severity (set in the next parameter), and `k` is the position.
      `exponent`: The weights are `s^k`. These bias adjustment types are
      inspired by Ovaisi et al. `Correcting for Selection Bias in
      Learning-to-rank Systems` (WWW 2020).
    positional_bias_severity: (float) The severity `s`, used as explained
      above. If `positional_bias_type` is unset, this parameter has no effect.
    positional_bias_positive_only: Whether to use the above defined bias
      weights only for positives (that is, clicked items). If
      `positional_bias_type` is unset, this parameter has no effect.
    logits_temperature: temperature parameter for non-deterministic policies.
      This value must be positive.
    summarize_grads_and_vars: A Python bool, default False. When True,
      gradients and network variable summaries are written during training.
    enable_summaries: A Python bool, default True. When False, all summaries
      (debug or otherwise) should not be written.
    train_step_counter: An optional `tf.Variable` to increment every time the
      train op is run.  Defaults to the `global_step`.
    penalty_mixture_coefficient: A parameter responsible for the balance
      between selecting high scoring items and enforcing diverisity. Used Only
      by diversity-based policies.
    name: The name of this agent instance.
docstr_tok: ["Initializes","an","instance","of","RankingAgent",".","Args",":","time_step_spec",":","A","`","TimeStep","`","spec","of","the","expected","time_steps",".","action_spec",":","A","nest","of","`","BoundedTensorSpec","`","representing","the","actions",".","scoring_network",":","The","network","that","outputs","scores","for","items",".","optimizer",":","The","optimizer","for","the","agent",".","policy_type",":","The","type","of","policy","used",".","The","only","available","type","at","this","moment","is","COSINE_DISTANCE",",","that","invokes","the","`","PenalizeCosineDistanceRankingPolicy","`",".","This","policy","uses","the","cosine","similarity","to","penalize","the","scores","of","not","yet","selected","items",".","If","set","to","`","UNKNOWN","`",",","falls","back","to","`","COSINE_DISTANCE","`",".","error_loss_fn",":","The","loss","function","used",".","feedback_model",":","The","type","of","feedback","model",".","Implemented","models","are",":","--","CASCADING",":","the","feedback","is","a","tuple","`","(","k",",","v",")","`",",","where","`","k","`","is","the","index","of","the","chosen","item",",","and","`","v","`","is","the","value","of","the","choice",".","If","no","item","was","chosen",",","then","`","k=num_slots","`","is","used","and","`","v","`","is","ignored",".","--","SCORE_VECTOR",":","the","feedback","is","a","vector","of","length","`","num_slots","`",",","containing","scores","for","every","item","in","the","recommendation",".","If","set","to","`","UNKNOWN","`",",","falls","back","to","CASCADING","`",".","non_click_score",":","(","float",")","For","the","cascading","feedback","model",",","this","is","the","score","value","for","items","lying","``","before","''","the","clicked","item",".","If","not","set",",","-1","is","used",".","It","is","recommended","(","but","not","enforced",")","to","use","a","negative","value",".","positional_bias_type",":","(","string",")","If","not","set","(","or","set","to","`","None","`",")",",","the","agent","does","not","apply","bias","adjustment",".","If","set","to","either","`","base","`","or","`","exponent","`",",","it","parameter","determines","what","way","the","positional","bias","is","accounted","for",".","`","base","`",":","The","bias","weight","for","each","slot","position","is","`","k^s","`",",","where","`","s","`","is","the","bias","severity","(","set","in","the","next","parameter",")",",","and","`","k","`","is","the","position",".","`","exponent","`",":","The","weights","are","`","s^k","`",".","These","bias","adjustment","types","are","inspired","by","Ovaisi","et","al",".","`","Correcting","for","Selection","Bias","in","Learning-to-rank","Systems","`","(","WWW","2020",")",".","positional_bias_severity",":","(","float",")","The","severity","`","s","`",",","used","as","explained","above",".","If","`","positional_bias_type","`","is","unset",",","this","parameter","has","no","effect",".","positional_bias_positive_only",":","Whether","to","use","the","above","defined","bias","weights","only","for","positives","(","that","is",",","clicked","items",")",".","If","`","positional_bias_type","`","is","unset",",","this","parameter","has","no","effect",".","logits_temperature",":","temperature","parameter","for","non-deterministic","policies",".","This","value","must","be","positive",".","summarize_grads_and_vars",":","A","Python","bool",",","default","False",".","When","True",",","gradients","and","network","variable","summaries","are","written","during","training",".","enable_summaries",":","A","Python","bool",",","default","True",".","When","False",",","all","summaries","(","debug","or","otherwise",")","should","not","be","written",".","train_step_counter",":","An","optional","`","tf.Variable","`","to","increment","every","time","the","train","op","is","run",".","Defaults","to","the","`","global_step","`",".","penalty_mixture_coefficient",":","A","parameter","responsible","for","the","balance","between","selecting","high","scoring","items","and","enforcing","diverisity",".","Used","Only","by","diversity-based","policies",".","name",":","The","name","of","this","agent","instance","."]
code: def __init__(
      self,
      time_step_spec: types.TimeStep,
      action_spec: types.BoundedTensorSpec,
      scoring_network: types.Network,
      optimizer: types.Optimizer,
      policy_type: RankingPolicyType = RankingPolicyType.COSINE_DISTANCE,
      error_loss_fn: types.LossFn = tf.compat.v1.losses.mean_squared_error,
      feedback_model: FeedbackModel = FeedbackModel.CASCADING,
      non_click_score: Optional[float] = None,
      positional_bias_type: Optional[Text] = None,
      positional_bias_severity: Optional[float] = None,
      positional_bias_positive_only: bool = False,
      logits_temperature: float = 1.0,
      summarize_grads_and_vars: bool = False,
      enable_summaries: bool = True,
      train_step_counter: Optional[tf.Variable] = None,
      penalty_mixture_coefficient: float = 1.0,
      name: Optional[Text] = None,
  ):
      """Initializes an instance of RankingAgent.
  
      Args:
        time_step_spec: A `TimeStep` spec of the expected time_steps.
        action_spec: A nest of `BoundedTensorSpec` representing the actions.
        scoring_network: The network that outputs scores for items.
        optimizer: The optimizer for the agent.
        policy_type: The type of policy used. The only available type at this
          moment is COSINE_DISTANCE, that invokes the
          `PenalizeCosineDistanceRankingPolicy`. This policy uses the cosine
          similarity to penalize the scores of not yet selected items. If set to
          `UNKNOWN`, falls back to `COSINE_DISTANCE`.
        error_loss_fn: The loss function used.
        feedback_model: The type of feedback model. Implemented models are:
          --CASCADING : the feedback is a tuple `(k, v)`, where `k` is the index
          of the chosen item, and `v` is the value of the choice. If no item was
          chosen, then `k=num_slots` is used and `v` is ignored. --SCORE_VECTOR :
          the feedback is a vector of length `num_slots`, containing scores for
          every item in the recommendation. If set to `UNKNOWN`, falls back to
          CASCADING`.
        non_click_score: (float) For the cascading feedback model, this is the
          score value for items lying "before" the clicked item. If not set, -1 is
          used. It is recommended (but not enforced) to use a negative value.
        positional_bias_type: (string) If not set (or set to `None`), the agent
          does not apply bias adjustment. If set to either `base` or `exponent`,
          it parameter determines what way the positional bias is accounted for.
          `base`: The bias weight for each slot position is `k^s`, where `s` is
          the bias severity (set in the next parameter), and `k` is the position.
          `exponent`: The weights are `s^k`. These bias adjustment types are
          inspired by Ovaisi et al. `Correcting for Selection Bias in
          Learning-to-rank Systems` (WWW 2020).
        positional_bias_severity: (float) The severity `s`, used as explained
          above. If `positional_bias_type` is unset, this parameter has no effect.
        positional_bias_positive_only: Whether to use the above defined bias
          weights only for positives (that is, clicked items). If
          `positional_bias_type` is unset, this parameter has no effect.
        logits_temperature: temperature parameter for non-deterministic policies.
          This value must be positive.
        summarize_grads_and_vars: A Python bool, default False. When True,
          gradients and network variable summaries are written during training.
        enable_summaries: A Python bool, default True. When False, all summaries
          (debug or otherwise) should not be written.
        train_step_counter: An optional `tf.Variable` to increment every time the
          train op is run.  Defaults to the `global_step`.
        penalty_mixture_coefficient: A parameter responsible for the balance
          between selecting high scoring items and enforcing diverisity. Used Only
          by diversity-based policies.
        name: The name of this agent instance.
      """
      tf.Module.__init__(self, name=name)
      common.tf_agents_gauge.get_cell("TFABandit").set(True)
      self._num_items = time_step_spec.observation[
          bandit_spec_utils.PER_ARM_FEATURE_KEY
      ].shape[0]
      self._num_slots = action_spec.shape[0]
      self._item_feature_dim = time_step_spec.observation[
          bandit_spec_utils.PER_ARM_FEATURE_KEY
      ].shape[-1]
      self._global_feature_dim = time_step_spec.observation[
          bandit_spec_utils.GLOBAL_FEATURE_KEY
      ].shape[-1]
      self._use_num_actions = (
          bandit_spec_utils.NUM_ACTIONS_FEATURE_KEY
          in time_step_spec.observation
      )
  
      scoring_network.create_variables()
      self._scoring_network = scoring_network
      self._optimizer = optimizer
      self._error_loss_fn = error_loss_fn
      if feedback_model == FeedbackModel.UNKNOWN:
          feedback_model = FeedbackModel.CASCADING
      self._feedback_model = feedback_model
      if feedback_model == FeedbackModel.CASCADING:
          if non_click_score is None:
              self._non_click_score = -1.0
          else:
              self._non_click_score = non_click_score
      else:
          if non_click_score is not None:
              raise ValueError(
                  "Parameter `non_click_score` should only be used "
                  "together with CASCADING feedback model."
              )
      self._positional_bias_type = positional_bias_type
      self._positional_bias_severity = positional_bias_severity
      self._positional_bias_positive_only = (
          positional_bias_positive_only
      )
      if policy_type == RankingPolicyType.UNKNOWN:
          policy_type = RankingPolicyType.COSINE_DISTANCE
      if policy_type == RankingPolicyType.COSINE_DISTANCE:
          policy = ranking_policy.PenalizeCosineDistanceRankingPolicy(
              self._num_items,
              self._num_slots,
              time_step_spec,
              scoring_network,
              penalty_mixture_coefficient=penalty_mixture_coefficient,
              logits_temperature=logits_temperature,
          )
      elif policy_type == RankingPolicyType.NO_PENALTY:
          policy = ranking_policy.NoPenaltyRankingPolicy(
              self._num_items,
              self._num_slots,
              time_step_spec,
              scoring_network,
              logits_temperature=logits_temperature,
          )
      elif policy_type == RankingPolicyType.DESCENDING_SCORES:
          policy = ranking_policy.DescendingScoreRankingPolicy(
              self._num_items,
              self._num_slots,
              time_step_spec,
              scoring_network,
          )
      else:
          raise NotImplementedError(
              "Policy type {} is not implemented".format(policy_type)
          )
      use_num_actions = (
          isinstance(time_step_spec.observation, dict)
          and "num_actions" in time_step_spec.observation
      )
      training_obs_spec = (
          bandit_spec_utils.create_per_arm_observation_spec(
              self._global_feature_dim,
              self._item_feature_dim,
              self._num_slots,
              use_num_actions,
          )
      )
      training_time_step_spec = ts.time_step_spec(
          training_obs_spec, reward_spec=time_step_spec.reward
      )
      training_data_spec = trajectory.from_transition(
          training_time_step_spec,
          policy_step.PolicyStep(),
          training_time_step_spec,
      )
      super(RankingAgent, self).__init__(
          time_step_spec=time_step_spec,
          action_spec=action_spec,
          policy=policy,
          collect_policy=policy,
          train_sequence_length=None,
          training_data_spec=training_data_spec,
          summarize_grads_and_vars=summarize_grads_and_vars,
          enable_summaries=enable_summaries,
          train_step_counter=train_step_counter,
      )
  
imports: import enum
  from typing import Optional, Text
  import tensorflow
  from tf_agents.agents import tf_agent
  from tf_agents.bandits.policies import ranking_policy
  from tf_agents.specs import bandit_spec_utils
  from tf_agents.trajectories import policy_step
  from tf_agents.trajectories import time_step
  from tf_agents.trajectories import trajectory
  from tf_agents.typing import types
  from tf_agents.utils import common
  from tf_agents.utils import eager_utils
  from tf_agents.utils import nest_utils
label_desc: Use image node_id 1 to create a new RankingAgent object from inherited base classes: tf_agent with example: obj = RankingAgent(time_step_spec, action_spec, scoring_network, optimizer, policy_type, error_loss_fn, feedback_model, non_click_score, positional_bias_type, positional_bias_severity, positional_bias_positive_only, logits_temperature, summarize_grads_and_vars, enable_summaries, train_step_counter, penalty_mixture_coefficient, name)