text: Theoretical motivation. A theoretical motivation for 
  the effectiveness of this collapse prevention strategy was 
  proposed in Grill et al. (2020) for the BYOL method. We 
  provide a simple adaptation of their analysis for our ℓ1 
  loss. For ease of exposition, we will disregard the effect of 
  the conditioning variable z and consider one dimensional 
  representations. Denote the representation Eθ(y) by 
  a random variable Y . The optimal predictor under 
  equation (1) is thus given by the following functional 
  expression, 
  
  Figure 2 Joint-Embedding Predictive Architectures are 
  trained to predict the representation of an input y from 
  the representation of another input x. The additional vari- 
  able z provides the predictor with information about the 
  transformation that computes y from x. 
  
  Our goal is to explore the effectiveness of feature pre- 
  diction as a stand-alone objective for learning visual 
  representations from video. To that end, we use a 
  joint-embedding predictive architecture (JEPA) (LeCun, 
  2022); see Figure 2. The main idea behind a JEPA is 
  to learn by predicting the representation of an input y 
  from the representation of another input x. The basic 
  architecture is made up of an encoder, Eθ( 
  ), which com- 
  · 
  putes the representation of the inputs, and a predictor, 
  ), which predicts the representation of y from the 
  Pϕ( 
  · 
  representation of x, conditioned on a variable z indicat- 
  ing the transformation (or corruption) between x and 
  y. Conditioning on z enables the generation of distinct 
  predictions for various transformations of x. 