class_docstr: Vision Transformer w/ Distillation Token and Head
  
  Distillation token & head support for `DeiT: Data-efficient Image Transformers`
      - https://arxiv.org/abs/2012.12877
code: def __init__(self, *args, **kwargs):
      weight_init = kwargs.pop("weight_init", "")
      super().__init__(*args, **kwargs, weight_init="skip")
      assert self.global_pool in ("token",)
  
      self.num_prefix_tokens = 2
      self.dist_token = nn.Parameter(torch.zeros(1, 1, self.embed_dim))
      self.pos_embed = nn.Parameter(
          torch.zeros(
              1,
              self.patch_embed.num_patches + self.num_prefix_tokens,
              self.embed_dim,
          )
      )
      self.head_dist = (
          nn.Linear(self.embed_dim, self.num_classes)
          if self.num_classes > 0
          else nn.Identity()
      )
      self.distilled_training = (
          False  # must set this True to train w/ distillation token
      )
  
      self.init_weights(weight_init)
  
imports: from functools import partial
  from typing import Sequence, Union
  import torch
  from torch import nn
  from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD
  from timm.layers import resample_abs_pos_embed
  from timm.models.vision_transformer import VisionTransformer, trunc_normal_, checkpoint_filter_fn
  from ._builder import build_model_with_cfg
  from ._manipulate import checkpoint_seq
  from ._registry import generate_default_cfgs, register_model, register_model_deprecations
label_desc: Use image node_id 1 to create a new VisionTransformerDistilled object from inherited base classes: VisionTransformer with example: obj = VisionTransformerDistilled()