other baseline models as reported in Appendix E.
4.4 Prediction Task: Predictingy from x
We conduct an ablation on the masking strategy used in
V-JEPA pretraining. We examine the following masking
strategies: random-tube[r] in which x is obtained by
removing a random fractionr of tubes (spatial patches
extended across the entire temporal duration) from the
video, causal multi-block[p]in whichx is restricted to
the firstp frames of the 16-frame video, which are then
masked with a random set of spatio-temporal blocks,
and multi-block in whichx obtained by masking a ran-
dom set of spatio-temporal blocks from the entire video.
Spatio-temporal blocks are sampled using the parame-
ters described in Section 3.2; an ablation on the size and
quantity of masked spatio-temporal blocks is provided
in Appendix E.4.
Table 4 indicates that the best results are obtained by
sampling x using a multi-block strategy, wherein the
network is forced to make predictions after removing
large continuous blocks in the video. Whenx is only
sampled from the first few frames of the video, as in
the causal multi-blockstrategy, we observe a decrease
in downstream performances. Finally, therandom-tube
strategy, wherein 90% of the tubes in the video are ran-
domly masked, leads to features of low-semantic quality
when combined with our feature prediction objective.
6