class_docstr: The search space that is proposed in `AutoFormer <https://arxiv.org/abs/2107.00651>`__.
  There are four searchable variables: depth, embedding dimension, heads number and MLP ratio.
  
  Parameters
  ----------
  search_embed_dim
      The search space of embedding dimension. Use a list to specify search range.
  search_mlp_ratio
      The search space of MLP ratio. Use a list to specify search range.
  search_num_heads
      The search space of number of heads. Use a list to specify search range.
  search_depth
      The search space of depth. Use a list to specify search range.
  img_size
      Size of input image.
  patch_size
      Size of image patch.
  in_channels
      Number of channels of the input image.
  num_labels
      Number of classes for classifier.
  qkv_bias
      Whether to use bias item in the qkv embedding.
  drop_rate
      Drop rate of the MLP projection in MSA and FFN.
  attn_drop_rate
      Drop rate of attention.
  drop_path_rate
      Drop path rate.
  pre_norm
      Whether to use pre_norm. Otherwise post_norm is used.
  global_pooling
      Whether to use global pooling to generate the image representation. Otherwise the cls_token is used.
  absolute_position
      Whether to use absolute positional embeddings.
  qk_scale
      The scaler on score map in self-attention.
  rpe
      Whether to use relative position encoding.
docstr: Load the related supernet checkpoints.
  
  Thanks to the weight entangling strategy that AutoFormer uses,
  AutoFormer releases a few trained supernet that allows thousands of subnets to be very well-trained.
  Under different constraints, different subnets can be found directly from the supernet, and used without any fine-tuning.
  
  Parameters
  ----------
  name
      Search space size, must be one of {'random-one-shot-tiny', 'random-one-shot-small', 'random-one-shot-base'}.
  download
      Whether to download supernet weights.
  progress
      Whether to display the download progress.
  
  Returns
  -------
  The loaded supernet.
docstr_tok: ["Load","the","related","supernet","checkpoints",".","Thanks","to","the","weight","entangling","strategy","that","AutoFormer","uses",",","AutoFormer","releases","a","few","trained","supernet","that","allows","thousands","of","subnets","to","be","very","well-trained",".","Under","different","constraints",",","different","subnets","can","be","found","directly","from","the","supernet",",","and","used","without","any","fine-tuning",".","Parameters","--","--","--","--","--","name","Search","space","size",",","must","be","one","of","{","'random-one-shot-tiny","'",",","'random-one-shot-small","'",",","'random-one-shot-base","'","}",".","download","Whether","to","download","supernet","weights",".","progress","Whether","to","display","the","download","progress",".","Returns","--","--","--","-","The","loaded","supernet","."]
code: def load_pretrained_supernet(
      cls, name: str, download: bool = True, progress: bool = True
  ) -> "AutoFormer":
      """
      Load the related supernet checkpoints.
  
      Thanks to the weight entangling strategy that AutoFormer uses,
      AutoFormer releases a few trained supernet that allows thousands of subnets to be very well-trained.
      Under different constraints, different subnets can be found directly from the supernet, and used without any fine-tuning.
  
      Parameters
      ----------
      name
          Search space size, must be one of {'random-one-shot-tiny', 'random-one-shot-small', 'random-one-shot-base'}.
      download
          Whether to download supernet weights.
      progress
          Whether to display the download progress.
  
      Returns
      -------
      The loaded supernet.
      """
      legal = [
          "random-one-shot-tiny",
          "random-one-shot-small",
          "random-one-shot-base",
      ]
      if name not in legal:
          raise ValueError(
              f"Unsupported name: {name}. It should be one of {legal}."
          )
      name = name[16:]
  
      # RandomOneShot is the only supported strategy for now.
      from nni.nas.strategy import RandomOneShot
  
      init_kwargs = cls.preset(name)
      with frozen_context.bypass():
          model_space = cls(**init_kwargs)
      model_space = RandomOneShot().mutate_model(model_space)
      weight_file = load_pretrained_weight(
          f"autoformer-{name}-supernet",
          download=download,
          progress=progress,
      )
      pretrained_weights = torch.load(weight_file)
      model_space.load_state_dict(pretrained_weights)
      return model_space
  
imports: from __future__ import annotations
  from typing import Tuple, cast, Any, Dict
  import torch
  import torch.nn
  import torch.nn.functional
  import nni
  from nni.mutable import Categorical, MutableExpression, frozen_context, ensure_frozen, Mutable
  from nni.nas.nn.pytorch import ParametrizedModule, ModelSpace, MutableModule, Repeat, MutableConv2d, MutableLinear, MutableLayerNorm
  from nni.nas.space import current_model
  from nni.nas.oneshot.pytorch.supermodule.operation import MixedOperation
  from nni.nas.profiler.pytorch.flops import FlopsResult
  from nni.nas.profiler.pytorch.utils import MutableShape, ShapeTensor, profiler_leaf_module
  from .utils.pretrained import load_pretrained_weight
  from .utils.nn import DropPath
label_desc: Use image node_id 4 for calling the AutoFormer obj's underlying member method code with example usage: obj.load_pretrained_supernet(cls, name, download, progress) and returns: model_space