text: minimizeθ,ϕ 
  Pϕ(Eθ(x), ∆y) 
  ∥  1, 
  Eθ(y) 
  ∥  − 
  
  would admit a trivial solution, where the encoder out- 
  puts a constant representation, regardless of its input. 
  In practice, we use the following modified objective to 
  prevent representation collapse, 
  
  minimizeθ,ϕ  sg(Eθ(y)) 
  (1) 
  Pϕ(Eθ(x), ∆y) 
  1, 
  ∥  ∥  − 
  
  ) denotes a stop-gradient operation, which 
  where sg( 
  · 
  does not backpropagate through its argument, and Eθ( 
  ) 
  · 
  ). 
  is an exponential moving average of the network Eθ( 
  · 
  The use of an exponential-moving average feature ex- 
  tractor along with a stop-gradient and a predictor has 
  been used as a collapse prevention strategy for image pre- 
  training (Grill et al., 2020), and studied empirically (Xie 
  et al., 2021) and theoretically (Tian et al., 2021). In 
  fact, the objective in equation (1) is similar to the loss 
  of Assran et al. (2023) used for image pretraining, but 
  we modify it to use an ℓ1 regression, which we found to 
  be more stable. 