class_docstr: Design Document https://fburl.com/gdoc/0rh98o3e
  DynamicBanditEnv will interact with the DynamicBanditAgent.
  DynamicBanditAgent chooses an action. Env gives reward of this action.
  
  DynamicBanditEnv creates episodes where the number of bandit arms (CBInput.arms)
  is not constant across episodes, and it is much smaller than total available number of arms.
  In the application of Ads Container project, there can be 30 Millions of ads but each campaign contains only 3~30 ads.
  Here ads=arms, campaign = episode.
  
  Each batch contains batch_size episodes. Each episode has num_arms_per_episode arms.
  Example:
  batch_size = 2, num_arms_per_episode = 3, num_unique_batches = 4, num_arms_all = 2*3*4
  There are 24 unique arms (namely a~w):
  - [[a,b,c] [d,e,f]],  [[g,h,i] [j,k,l]],  [[m,n,o] [p,q,r]],  [[s,t,u] [v,w,x]]
  Some batch may re-appear (a group of ad campaigns reappear):
  - [[a,b,c] [d,e,f]],  [[g,h,i] [j,k,l]],  [[m,n,o] [p,q,r]],  [[s,t,u] [v,w,x]],  [[a,b,c] [d,e,f]]
  The re-appeared ads (a~f) will have similar but differetn features vs their prev appearance.
class_docstr_tok: ["Design","Document","https",":","\/\/fburl.com\/gdoc\/0rh98o3e","DynamicBanditEnv","will","interact","with","the","DynamicBanditAgent",".","DynamicBanditAgent","chooses","an","action",".","Env","gives","reward","of","this","action",".","DynamicBanditEnv","creates","episodes","where","the","number","of","bandit","arms","(","CBInput.arms",")","is","not","constant","across","episodes",",","and","it","is","much","smaller","than","total","available","number","of","arms",".","In","the","application","of","Ads","Container","project",",","there","can","be","30","Millions","of","ads","but","each","campaign","contains","only","3~30","ads",".","Here","ads=arms",",","campaign","=","episode",".","Each","batch","contains","batch_size","episodes",".","Each","episode","has","num_arms_per_episode","arms",".","Example",":","batch_size","=","2",",","num_arms_per_episode","=","3",",","num_unique_batches","=","4",",","num_arms_all","=","2","*","3","*","4","There","are","24","unique","arms","(","namely","a~w",")",":","-","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]",",","[","[","g",",","h",",","i","]","[","j",",","k",",","l","]","]",",","[","[","m",",","n",",","o","]","[","p",",","q",",","r","]","]",",","[","[","s",",","t",",","u","]","[","v",",","w",",","x","]","]","Some","batch","may","re-appear","(","a","group","of","ad","campaigns","reappear",")",":","-","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]",",","[","[","g",",","h",",","i","]","[","j",",","k",",","l","]","]",",","[","[","m",",","n",",","o","]","[","p",",","q",",","r","]","]",",","[","[","s",",","t",",","u","]","[","v",",","w",",","x","]","]",",","[","[","a",",","b",",","c","]","[","d",",","e",",","f","]","]","The","re-appeared","ads","(","a~f",")","will","have","similar","but","differetn","features","vs","their","prev","appearance","."]
docstr: This method generates the reward shift_k for all arms at the beginning of this program only once.
  When a batch of arms (with sample_batch_idx) show up, index their shift_k.
docstr_tok: ["This","method","generates","the","reward","shift_k","for","all","arms","at","the","beginning","of","this","program","only","once",".","When","a","batch","of","arms","(","with","sample_batch_idx",")","show","up",",","index","their","shift_k","."]
code: def gen_all_arms_reward_shifts(self):
      """
      This method generates the reward shift_k for all arms at the beginning of this program only once.
      When a batch of arms (with sample_batch_idx) show up, index their shift_k.
      """
      reward_shifts = (
          torch.randn(self._num_arms_all) * self.sigma_shift
          + self.mu_shift
      )
      return reward_shifts
  
imports: import random
  from dataclasses import replace
  from typing import Any, Optional, Tuple
  import reagent.core.types
  import torch
  from reagent.core.types import CBInput
  from reagent.gym.policies.policy import Policy
  from reagent.gym.policies.samplers.discrete_sampler import GreedyActionSampler
  from reagent.gym.types import Sampler, Scorer
  from reagent.models.linear_regression import LinearRegressionUCB
  from reagent.training.cb.base_trainer import BaseCBTrainerWithEval
  from reagent.training.cb.linucb_trainer import LinUCBTrainer
  from reagent.training.parameters import LinUCBTrainerParameters
label_desc: Use image node_id 7 for calling the DynamicBanditEnv obj's underlying member method code with example usage: obj.gen_all_arms_reward_shifts() and returns: reward_shifts