class_docstr: ViT transformer
  ViT Transformer, classifier is a single Linear layer for finetune,
  For training from scratch, two layer mlp should be used.
  Classification is done using cls_token.
  Args:
      image_size: int, input image size, default: 224
      patch_size: int, patch size, default: 16
      in_channels: int, input image channels, default: 3
      num_classes: int, number of classes for classification, default: 1000
      embed_dim: int, embedding dimension (patch embed out dim), default: 768
      depth: int, number ot transformer blocks, default: 12
      num_heads: int, number of attention heads, default: 12
      mlp_ratio: float, ratio of mlp hidden dim to embed dim(mlp in dim), default: 4.0
      qkv_bias: bool, If True, enable qkv(nn.Linear) layer with bias, default: True
      dropout: float, dropout rate for linear layers, default: 0.
      attention_dropout: float, dropout rate for attention layers default: 0.
      droppath: float, droppath rate for droppath layers, default: 0.
code: def __init__(
      self,
      image_size=224,
      patch_size=16,
      in_channels=3,
      num_classes=768,
      embed_dim=768,
      depth=12,
      num_heads=12,
      attn_head_size=None,
      mlp_ratio=4,
      qkv_bias=True,
      dropout=0.0,
      attention_dropout=0.0,
      droppath=0.0,
      train_from_scratch=False,
  ):
      super().__init__()
      # create patch embedding with positional embedding
      self.patch_embedding = PatchEmbedding(
          image_size, patch_size, in_channels, embed_dim, dropout
      )
      # create multi head self-attention layers
      self.encoder = Encoder(
          embed_dim,
          num_heads,
          depth,
          attn_head_size,
          qkv_bias,
          mlp_ratio,
          dropout,
          attention_dropout,
          droppath,
      )
  
      # classifier head (for training from scracth)
      if train_from_scratch:
          w_attr_1, b_attr_1 = self._init_weights()
          w_attr_2, b_attr_2 = self._init_weights()
          self.classifier = nn.Sequential(
              nn.Linear(
                  embed_dim,
                  embed_dim,
                  weight_attr=w_attr_1,
                  bias_attr=b_attr_1,
              ),
              nn.ReLU(),
              nn.Dropout(dropout),
              nn.Linear(
                  embed_dim,
                  num_classes,
                  weight_attr=w_attr_2,
                  bias_attr=b_attr_2,
              ),
              nn.Dropout(dropout),
          )
      else:
          # classifier head (for finetuning)
          w_attr_1, b_attr_1 = self._init_weights()
          self.classifier = nn.Linear(
              embed_dim,
              num_classes,
              weight_attr=w_attr_1,
              bias_attr=b_attr_1,
          )
  
imports: import copy
  import paddle
  import paddle.nn
  from disco_diffusion_ernievil_base.vit_b_16x.ernievil2.transformers.droppath import DropPath
label_desc: Use image node_id 1 to create a new VisualTransformer object from inherited base classes: nn with example: obj = VisualTransformer(image_size, patch_size, in_channels, num_classes, embed_dim, depth, num_heads, attn_head_size, mlp_ratio, qkv_bias, dropout, attention_dropout, droppath, train_from_scratch)