
At present, the only attention mask supported by fused kernel implementation is the causal mask commonly used for training. To specify the causal mask in custom kernels, it must be specified with the is_causal boolean and attn_mask must be None. 
Support for Nested Tensors is still under development.  Specifically, in PyTorch 2.0, only the sdpa_math kernel supports training with Nested Tensors. Also, PyTorch 2.0 does not support Nested Tensors as part of code being compiled with torch.compile(). 
The SDPA operator does not support returning averaged attention weights because computing them defeats the optimizations that enabled fused kernels to execute more efficiently.  The argument need_weights for torch.nn.MultiheadAttention's forward function defaults to True. In order to use the fused kernels, need_weights needs to be set to need_weights=False.
