of the masked regions (see Section 3). To qualitatively in-
vestigate the grounding of the feature-space predictions,
we freeze the pretrained encoder and predictor networks
and train a conditional diffusion decoder to map the
V-JEPA predictions to interpretable pixels. Notably, the
decoder is only fed the representations predicted for the
missing regions of the video, and does not have access
to the unmasked regions of the video (see Figure 6a).
Given a masked video, we use theV-JEPA pretrained
models to predict the representations of the missing
regions, and then use the decoder to project the rep-
resentations to pixel space. Figure 6b shows decoder
outputs for various random seeds. Qualities that are
common across samples represent information that is
contained in the predictor representation.
Figure 6b shows that theV-JEPA feature predictions
are indeed grounded, and exhibit spatio-temporal con-
sistency with the unmasked regions of the video. Specif-
ically, the samples in Figure 6b show that theV-JEPA
predictor correctly captures positional uncertainty and
produces a variety of visual objects at various locations
with consistent motion. Some of the samples also demon-
strate an understanding of object-permanence, as the
visual objects remain consistent after partial occlusion.
9