text: Self-Supervised Learning from Static Images 
  
  Other approaches for learning from static images leverage unsupervised objectives. Initial works on self-supervised 
  approaches are based on sparse coding or hand-crafted pretext tasks, such as colorization (Larsson et al., 2016, 2017), 
  rotation prediction (Gidaris et al., 2020), and jigsaws (Noroozi and Favaro, 2016). More recent approaches leverage 
  invariance-based objectives by training a visual encoder to be invariant to hand-crafted image transformations (Wu 
  et al., 2018; Chen et al., 2020). 
  
  Another family of methods learn representations using denoising autoencoders (Vincent et al., 2008); image inpainting 
  is one popular instantiation of this idea (Pathak et al., 2016). More recently, masked autoencoders (He et al., 2021) 
  train an encoder-decoder transformer to predict missing pixels of a masked image. Follow-up work addresses the 
  indeterminism of pixel reconstruction by exploring instantiations of masked image modeling in latent space (Baevski 
  et al., 2022b; Assran et al., 2023; Baevski et al., 2022a). These approaches can be seen as applications of the 
  predictive feature principle in the image modality. 