text: Optimization. For all the tasks, we use AdamW optimizer with a cosine scheduler (no warmup) that decays the 
  learning rate from 0.001 to 0. We use a fixed weight-decay of 0.01 and apply simple data augmentations (random 
  resized crops and horizontal flips) during training of the attentive probe, except on image tasks, where we apply 
  AutoAugment (Dogus Cubuk et al., 2019). Table 9 reports the hyperparameters for each downstream evaluation. 
  
  Extension to multiple clips. Unless stated otherwise, our attentive probe takes 8 clips of 16 frames as input on 
  Kinetics, and 2 clips of 16 frames on Something-Somethingv2 to increase the temporal coverage of the video. 
  
  18 