class_docstr: Vision Transformer w/ Distillation Token and Head
  
  Distillation token & head support for `DeiT: Data-efficient Image Transformers`
      - https://arxiv.org/abs/2012.12877
code: def forward_head(self, x, pre_logits: bool = False) -> torch.Tensor:
      x, x_dist = x[:, 0], x[:, 1]
      if pre_logits:
          return (x + x_dist) / 2
      x = self.head(x)
      x_dist = self.head_dist(x_dist)
      if (
          self.distilled_training
          and self.training
          and not torch.jit.is_scripting()
      ):
          # only return separate classification predictions when training in distilled mode
          return x, x_dist
      else:
          # during standard train / finetune, inference average the classifier predictions
          return (x + x_dist) / 2
  
imports: from functools import partial
  from typing import Sequence, Union
  import torch
  from torch import nn
  from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD
  from timm.layers import resample_abs_pos_embed
  from timm.models.vision_transformer import VisionTransformer, trunc_normal_, checkpoint_filter_fn
  from ._builder import build_model_with_cfg
  from ._manipulate import checkpoint_seq
  from ._registry import generate_default_cfgs, register_model, register_model_deprecations
label_desc: Use image node_id 8 for calling the VisionTransformerDistilled obj's underlying member method code with example usage: obj.forward_head(x, pre_logits) and returns: unknown, x,  x_dist, unknown