class_docstr: A recommendation environment which generates items with visible features
  randomly (parametric actions).
  The environment can be configured to be multi-user, i.e. different models
  will be learned independently for each user, by setting num_users_in_db
  parameter.
  To enable slate recommendation, the `slate_size` config parameter can be
  set as > 1.
class_docstr_tok: ["A","recommendation","environment","which","generates","items","with","visible","features","randomly","(","parametric","actions",")",".","The","environment","can","be","configured","to","be","multi-user",",","i.e",".","different","models","will","be","learned","independently","for","each","user",",","by","setting","num_users_in_db","parameter",".","To","enable","slate","recommendation",",","the","`","slate_size","`","config","parameter","can","be","set","as",">","1","."]
code: def step(self, action):
      # Action is the suggested slate (indices of the docs in the
      # suggested ones).
  
      # We calculate scores as the dot product between document features and user
      # features. The softmax ensures regret<1 further down.
      scores = softmax(
          [
              np.dot(self.current_user, doc)
              for doc in self.currently_suggested_docs
          ]
      )
      best_reward = np.max(scores)
  
      # User choice model: User picks a doc stochastically,
      # where probs are dot products between user- and doc feature
      # (categories) vectors (rewards).
      # There is also a no-click doc whose weight is 0.0.
      user_doc_overlaps = np.array([scores[a] for a in action] + [0.0])
      # We have to softmax again so that probabilities add up to 1
      probabilities = softmax(user_doc_overlaps)
      which_clicked = np.random.choice(
          np.arange(self.slate_size + 1), p=probabilities
      )
  
      reward = 0.0
      if which_clicked < self.slate_size:
          # Reward is 1.0 - regret if clicked. 0.0 if not clicked.
          regret = best_reward - user_doc_overlaps[which_clicked]
          # The reward also represents the user engagement that we define to be
          # withing the range [0...100].
          reward = (1 - regret) * 100
          # If anything clicked, deduct from the current user's time budget.
          self.current_user_budget -= 1.0
      done = truncated = self.current_user_budget <= 0.0
  
      # Compile response.
      response = tuple(
          {
              "click": int(idx == which_clicked),
              "engagement": reward if idx == which_clicked else 0.0,
          }
          for idx in range(len(user_doc_overlaps) - 1)
      )
  
      return (
          self._get_obs(response=response),
          reward,
          done,
          truncated,
          {},
      )
  
file_path: ray/rllib/examples/env/bandit_envs_recommender_system.py
imports: import gymnasium
  import numpy
  from typing import Optional
  from ray.rllib.utils.numpy import softmax
label_desc: Use image node_id 4 for calling the ParametricRecSys obj's underlying member method code with example usage: obj.step(action) and returns: self,  reward,  done,  truncated,  dict