Table 11 Finetuning Evaluation hyper-parameters.
Hyper-parameter K400 SSv2
data
num_segments 1
num_frames 16
sampling_rate 4
resolution 224
model
model_name ViT-L/16 ViT-H/16 ViT-L/16 ViT-H/16
drop_path 0.1 0.2 0.2 0.2
head_drop_rate 0. 0. 0.5 0.5
optimization
batch_size 256 1024 256 256
epochs 35 25 15 15
opt adamw
opt_eps 0.00000001
momentum 0.9
weight_decay 0.05
lr 0.002 0.0005 0.0005 0.0005
layer_decay 0.75 0.75 0.75 0.75
warmup_lr 1e-6 1e-8 1e-6 1e-6
min_lr 1e-6 1e-5 1.5e-4 1.5e-3
warmup_epochs 5
augmentations
color_jitter 0.4
horizontal_flip True True False False
num_sample 2
aa rand-m7-n4-mstd0.5-inc1
smoothing 0.1
train_interpolation bicubic
test_num_segment 5 5 2 2
test_num_crop 3 3 3 3
erase
prob 0.25
mode pixel
count 1
split False
mixup
mixup 0.8
cutmix 1.0
mixup_prob 1.0
mixup_switch_prob 0.5
mixup_mode batch
20