text: 3.3 Network Parameterization 
  
  We use a Vision Transformer (ViT) (Dosovitskiy et al., 
  2020; Arnab et al., 2021) as our video backbone. To 
  process a video with a transformer network, we split the 
  video clip into a 3D grid of L spatio-temporal patches, 
  where a patch consists of a 16 
  16 pixel block spanning 
  2 consecutive frames; we refer to these spatio-temporal 
  patches as tokens. This sequence of tokens is then di- 
  rectly processed by the stack of transformer blocks. In- 
  
  × 