102.4 102.6 102.8 103 103.2 103.4
74
74.5
75
SOTA fine-tuned task-specific
model on SSv 2 (MVD)
V-JEPA
ViT-L/16
VideoMAE
ViT-L/16
Hiera
Hiera-L
OmniMAE
ViT-L/16
Samples Seen (M)
Something-Something-v2 End-to-End Fine-Tuning
Video Feature Pred.
Video Pixel Pred.
Figure 4 SSv2 fine-tuning performance vs. Samples Seen.We
report SSv2 fine-tuning for V-JEPA and pixel-reconstruction
baselines using a ViT-L/16 or Hiera-L architecture. V-JEPA
outperforms all pixel-reconstruction methods using a ViT-
L/16 and matches the Hiera-L performance while seeing
significantly less samples during pretraining.
ageNet; hence,V-JEPA achieves comparable ImageNet
performance despite only pretraining on video.
Under the fine-tuning protocol,V-JEPAalso achieves the
best performance of any model trained with a ViT-L/16,
and matches the performance of the Hiera-L on SSv2,
whichbenefitsfromahierachicalprior (Ryali etal.,2023).
The V-JEPA models achieve this result while processing
significantly fewer samples during pretraining (Figure 4),
demonstrating the efficiency of feature prediction as a
learning principle.
5.2 Comparison with State-of-the-Art
Next, in Table 6, we inspect how theV-JEPA models
pretrained on video stack up next to the largest state-
of-the-art self-supervised image and video models when
freezing the backbone encoder and training an attentive
probe on top. Our image pretrained baselines include
OpenCLIP (Cherti et al., 2023), DINOv2 (Oquab et al.,
2023), and I-JEPA (Assran et al., 2023). The Open-
CLIP model is trained with a contrastive image-text
alignment objective, DINOv2 and I-JEPA are trained
with self-supervision. These models are known to excel
in their frozen-evaluation performance (Oquab et al.,
2023); i.e., their ability to produce visual features that
can be applied to many downstream tasks simultane-
ously, without end-to-end fine-tuning, and thus pro-
vide highly competitive baselines. Our video pretrained
baselines include VideoMAE (Tong et al., 2022), Omni-
MAE (Girdhar et al., 2023), Hiera (Ryali et al., 2023),
VideoMAEv2 (Wang et al., 2023a), and MVD (Wang
et al., 2023b). The OpenCLIP, DINOv2 and Video-
MAEv2 models are parameterized as Giant/Gigantic
vision transformer architectures containing over 1B pa-
rameters trained on large-scale image or video datasets.
Comparison with video models. Compared to
large-scale video baselines, theV-JEPA models outper-
form all previous models on every downstream video
50 100 150 200 250 300 350
60
65
70
75 V-JEPA
ViT-H/16384
VideoMAE
ViT-H/16
VideoMAEv2
ViT-g/14
Pretraining Time (Hrs.)
Something-Something-v2 Frozen Evaluation
Video Feature Pred.
Video Pixel Pred.
Figure 5 SSv2 frozen-evaluation performance vs. Pretraining
Time. Wallclock times for all methods are measured on a
single GPU with a batch size of 10 clips, using the official
codebases for VideoMAE and VideoMAEv2, and linearly
extrapolated assuming a global batch size of 2400 samples.
However, note that the SSv2 accuracies of video pixel pre-
diction methods are actually obtained with small batch sizes
and significantly longer training schedules. V-JEPA out-
performs pixel-reconstruction methods while training signifi-
cantly faster.
and image task with notable margin (see Table 6). Our
H/16 model outperforms the largest publicly available
VideoMAE, VideoMAEv2, OmniMAE, MVD, and Hiera
models by at least+5 points in motion understanding
(Something-Something-v2), +2 points in action recogni-
tion (Kinetics-400),+5 points on action detection (AVA),
+1 point on object recognition (ImageNet-1K),+2 points
in scene recognition (Places205), and+0.2 points on fine-
grained recognition (iNaturalist). Moreover, when com-
paring pretraining wallclock time in Figure 5, we see that
V-JEPA achieves this performance with a roughly2×
speedup compared to the large pixel prediction models.
Comparison with image models. On tasks that re-
quire a fine-grained understanding of motion (Something-