text: C Pretraining details 
  
  In section, we report V-JEPA pretraining details. Table 8 summarizes the main hyperparameters used during 
  pretraining. 
  
  Architectures. We use Vision Transformer (Dosovitskiy et al., 2020) (ViT) architectures for the x-encoder and 
  y-encoder. We train three V-JEPA encoders: a ViT-L/16224, a ViT-H/16224 and a ViT-H/16384. All three encoders 
  take as input a short video clip of 16 frames with a temporal stride of 4 between consecutive frames. The subscripts, 
  224 and 384, indicate the spatial resolution of the video clip. V-JEPA flattens the video clip into a sequence of 
  2 (see Figure 7). For all three models, the predictor is 
  non-overlapping spatio-temporal patches of size 16 
  × 
  designed as a narrow ViT architecture, consisting of 12 transformer blocks with an embedding dimension of 384. For 
  simplicity, we keep the number of self-attention heads in the predictor equal to that of the backbone used for the 
  context-encoder/target-encoder. V-JEPA is pretrained without using a [cls] token. 
  
  16 
  × 