 sliced version of attention that performs the computation in steps instead of all at once.
Attention slicing is useful even if a batch size of just 1 is used - as long
  as the model uses more than one attention head. If there is more than one
  attention head the *QK^T* attention matrix can be computed sequentially for
  each head which can save a significant amount of memory.

To perform the attention computation sequentially over each head,