class_docstr: The search space that is proposed in `AutoFormer <https://arxiv.org/abs/2107.00651>`__.
  There are four searchable variables: depth, embedding dimension, heads number and MLP ratio.
  
  Parameters
  ----------
  search_embed_dim
      The search space of embedding dimension. Use a list to specify search range.
  search_mlp_ratio
      The search space of MLP ratio. Use a list to specify search range.
  search_num_heads
      The search space of number of heads. Use a list to specify search range.
  search_depth
      The search space of depth. Use a list to specify search range.
  img_size
      Size of input image.
  patch_size
      Size of image patch.
  in_channels
      Number of channels of the input image.
  num_labels
      Number of classes for classifier.
  qkv_bias
      Whether to use bias item in the qkv embedding.
  drop_rate
      Drop rate of the MLP projection in MSA and FFN.
  attn_drop_rate
      Drop rate of attention.
  drop_path_rate
      Drop path rate.
  pre_norm
      Whether to use pre_norm. Otherwise post_norm is used.
  global_pooling
      Whether to use global pooling to generate the image representation. Otherwise the cls_token is used.
  absolute_position
      Whether to use absolute positional embeddings.
  qk_scale
      The scaler on score map in self-attention.
  rpe
      Whether to use relative position encoding.
code: def forward(self, x):
      B = x.shape[0]
      x = self.patch_embed(x)
      x = x.permute(0, 2, 3, 1).view(B, self.patches_num, -1)
      x = self.cls_token(x)
      x = self.pos_embed(x)
      x = self.blocks(x)
      x = self.norm(x)
  
      if self.global_pooling:
          x = torch.mean(x[:, 1:], dim=1)
      else:
          x = x[:, 0]
  
      x = self.head(x)
  
      return x
  
imports: from __future__ import annotations
  from typing import Tuple, cast, Any, Dict
  import torch
  import torch.nn
  import torch.nn.functional
  import nni
  from nni.mutable import Categorical, MutableExpression, frozen_context, ensure_frozen, Mutable
  from nni.nas.nn.pytorch import ParametrizedModule, ModelSpace, MutableModule, Repeat, MutableConv2d, MutableLinear, MutableLayerNorm
  from nni.nas.space import current_model
  from nni.nas.oneshot.pytorch.supermodule.operation import MixedOperation
  from nni.nas.profiler.pytorch.flops import FlopsResult
  from nni.nas.profiler.pytorch.utils import MutableShape, ShapeTensor, profiler_leaf_module
  from .utils.pretrained import load_pretrained_weight
  from .utils.nn import DropPath
label_desc: Use image node_id 6 for calling the AutoFormer obj's underlying member method code with example usage: obj.forward(x) and returns: x