class_docstr: Design Document https://fburl.com/gdoc/0rh98o3e
  DynamicBanditEnv will interact with the DynamicBanditAgent.
  DynamicBanditAgent chooses an action. Env gives reward of this action.
  
  DynamicBanditEnv creates episodes where the number of bandit arms (CBInput.arms)
  is not constant across episodes, and it is much smaller than total available number of arms.
  In the application of Ads Container project, there can be 30 Millions of ads but each campaign contains only 3~30 ads.
  Here ads=arms, campaign = episode.
  
  Each batch contains batch_size episodes. Each episode has num_arms_per_episode arms.
  Example:
  batch_size = 2, num_arms_per_episode = 3, num_unique_batches = 4, num_arms_all = 2*3*4
  There are 24 unique arms (namely a~w):
  - [[a,b,c] [d,e,f]],  [[g,h,i] [j,k,l]],  [[m,n,o] [p,q,r]],  [[s,t,u] [v,w,x]]
  Some batch may re-appear (a group of ad campaigns reappear):
  - [[a,b,c] [d,e,f]],  [[g,h,i] [j,k,l]],  [[m,n,o] [p,q,r]],  [[s,t,u] [v,w,x]],  [[a,b,c] [d,e,f]]
  The re-appeared ads (a~f) will have similar but differetn features vs their prev appearance.
class_docstr_tok: ["Design","Document","https",":","\/\/fburl.com\/gdoc\/0rh98o3e","DynamicBanditEnv","will","interact","with","the","DynamicBanditAgent",".","DynamicBanditAgent","chooses","an","action",".","Env","gives","reward","of","this","action",".","DynamicBanditEnv","creates","episodes","where","the","number","of","bandit","arms","(","CBInput.arms",")","is","not","constant","across","episodes",",","and","it","is","much","smaller","than","total","available","number","of","arms",".","In","the","application","of","Ads","Container","project",",","there","can","be","30","Millions","of","ads","but","each","campaign","contains","only","3~30","ads",".","Here","ads=arms",",","campaign","=","episode",".","Each","batch","contains","batch_size","episodes",".","Each","episode","has","num_arms_per_episode","arms",".","Example",":","batch_size","=","2",",","num_arms_per_episode","=","3",",","num_unique_batches","=","4",",","num_arms_all","=","2","*","3","*","4","There","are","24","unique","arms","(","namely","a~w",")",":","-","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]",",","[","[","g",",","h",",","i","]","[","j",",","k",",","l","]","]",",","[","[","m",",","n",",","o","]","[","p",",","q",",","r","]","]",",","[","[","s",",","t",",","u","]","[","v",",","w",",","x","]","]","Some","batch","may","re-appear","(","a","group","of","ad","campaigns","reappear",")",":","-","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]",",","[","[","g",",","h",",","i","]","[","j",",","k",",","l","]","]",",","[","[","m",",","n",",","o","]","[","p",",","q",",","r","]","]",",","[","[","s",",","t",",","u","]","[","v",",","w",",","x","]","]",",","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]","The","re-appeared","ads","(","a~f",")","will","have","similar","but","differetn","features","vs","their","prev","appearance","."]
docstr: We assume there is linear relationship between feature and reward : r_k = g(f_k) + shift_k
  The g() is a linear function mapping feature to reward
  The shift_k is a shift of reward and it follows Gaussian distribution with mean 0 and variance sigma_shift
  The output is reward, a float number
docstr_tok: ["We","assume","there","is","linear","relationship","between","feature","and","reward",":","r_k","=","g","(","f_k",")","+","shift_k","The","g","(",")","is","a","linear","function","mapping","feature","to","reward","The","shift_k","is","a","shift","of","reward","and","it","follows","Gaussian","distribution","with","mean","0","and","variance","sigma_shift","The","output","is","reward",",","a","float","number"]
code: def features_to_rewards(
      self,
      inp_feature: torch.Tensor,
      sample_batch_idx: torch.Tensor,
  ) -> torch.Tensor:
      """
      We assume there is linear relationship between feature and reward : r_k = g(f_k) + shift_k
      The g() is a linear function mapping feature to reward
      The shift_k is a shift of reward and it follows Gaussian distribution with mean 0 and variance sigma_shift
      The output is reward, a float number
      """
      # map features to rewards through a linear function W
      reward = torch.nn.functional.linear(
          input=inp_feature, weight=self.weight, bias=None
      )
  
      # add some Gaussian noise to each reward
      shift_k = self.reward_shifts[sample_batch_idx]
      reward += shift_k
      noise_k = torch.randn_like(reward) * self.reward_noise_sigma
      reward += noise_k
      return reward
  
imports: import random
  from dataclasses import replace
  from typing import Any, Optional, Tuple
  import reagent.core.types
  import torch
  from reagent.core.types import CBInput
  from reagent.gym.policies.policy import Policy
  from reagent.gym.policies.samplers.discrete_sampler import GreedyActionSampler
  from reagent.gym.types import Sampler, Scorer
  from reagent.models.linear_regression import LinearRegressionUCB
  from reagent.training.cb.base_trainer import BaseCBTrainerWithEval
  from reagent.training.cb.linucb_trainer import LinUCBTrainer
  from reagent.training.parameters import LinUCBTrainerParameters
label_desc: Use image node_id 13 for calling the DynamicBanditEnv obj's underlying member method code with example usage: obj.features_to_rewards(inp_feature, sample_batch_idx) and returns: reward