text: D Evaluation details 
  
  D.1 Frozen classification 
  
  Attentive Probing. Given an input video, xL, the V-JEPA target encoder Eθ( 
  ) outputs a sequence of L tokens, 
  Rd. To pool this sequence of tokens into a single feature vector, we apply a 
  Eθ(xL) = (s1, . . . , sL), where si 
  lightweight non-linear cross-attention block which replace the self-attention operation of a transformer block with 
  cross attention. Specifically, the cross-attention performs the following computation: 
  
  · 
  
  L 
  (cid:88) 
  
  exp(q⊤Wksi) 
  j exp(q⊤Wksj) 
  (cid:80) 
  i=1 
  
  where Wk, Wv 
  Rd is a learnable query token. The output of the 
  cross-attention is then added back to the query token (residual connection), and then fed into two-layer MLP with a 
  single GeLU activation, followed by a LayerNorm, and finally a linear classifier. The parameters of the cross-attention 
  block are jointly learned with that of the linear classifier for the downstream task, while the encoder parameters 
  are kept frozen. Note that, in practice, we actually use an attentive probe with 12 heads, each of dimension 12. In 
  Appendix E we show that baselines benefit from the attentive probing protocol. 
  
  Rd×d are the key and value matrices, and q 
  ∈  ∈ 