class_docstr: The evaluation environment for RL-based algorithms.
  
  Currently the environment mocks an environment of multiple choices.
  In each step, it returns an observation with the current action history and the number of choices in this step.
  The environment is "done" when the step count reaches the length of choices.
  
  The reward is always 0 as it's processed in :class:`TuningTrajectoryGenerator`.
  The environment isn't responsible for the actual configuration evaluation, neither will it invoke any training.
  
  It only supports search space with categorical choices for now.
  When there are multiple categorical choices with different number of choices,
  the environment will pad the choices to make them have the same length, and put the real number of choices in the observation.
  The policy returned by :func:`default_policy_fn` will ignore the padded choices.
  
  Environment is stateful. One environment instance should not be used in more than one trajectories at the same time.
  
  Parameters
  ----------
  num_choices
      The number of choices in each step.
class_docstr_tok: ["The","evaluation","environment","for","RL-based","algorithms",".","Currently","the","environment","mocks","an","environment","of","multiple","choices",".","In","each","step",",","it","returns","an","observation","with","the","current","action","history","and","the","number","of","choices","in","this","step",".","The","environment","is","``","done","''","when","the","step","count","reaches","the","length","of","choices",".","The","reward","is","always","0","as","it","'s","processed","in",":","class",":","`","TuningTrajectoryGenerator","`",".","The","environment","is","n't","responsible","for","the","actual","configuration","evaluation",",","neither","will","it","invoke","any","training",".","It","only","supports","search","space","with","categorical","choices","for","now",".","When","there","are","multiple","categorical","choices","with","different","number","of","choices",",","the","environment","will","pad","the","choices","to","make","them","have","the","same","length",",","and","put","the","real","number","of","choices","in","the","observation",".","The","policy","returned","by",":","func",":","`","default_policy_fn","`","will","ignore","the","padded","choices",".","Environment","is","stateful",".","One","environment","instance","should","not","be","used","in","more","than","one","trajectories","at","the","same","time",".","Parameters","--","--","--","--","--","num_choices","The","number","of","choices","in","each","step","."]
code: def __init__(self, num_choices: list[int]):
      self.num_choices = num_choices
      self.max_num_choices = max(num_choices)
      self.num_steps = len(self.num_choices)
  
imports: from __future__ import annotations
  from copy import deepcopy
  from typing import Tuple, Callable
  import gym
  import numpy
  import torch
  import torch.nn
  import torch.nn.functional
  from typing_extensions import TypedDict
  from gym import spaces
  from tianshou.data import ReplayBuffer, Batch, to_torch, to_numpy
  from tianshou.policy import BasePolicy, PPOPolicy
  from nni.mutable import Categorical, CategoricalMultiple, Mutable, MutableAnnotation, Sample
label_desc: Use image node_id 1 to create a new TuningEnvironment object from inherited base classes: gym.Env with example: obj = TuningEnvironment(num_choices)