

training is disabled (using ".eval()")


"add_bias_kv" is "False"


"add_zero_attn" is "False"


"batch_first" is "True" and the input is batched


"kdim" and "vdim" are equal to "embed_dim"


if a NestedTensor is passed, neither "key_padding_mask" nor
     "attn_mask" is passed


autocast is disabled


If the optimized implementation is in use, a NestedTensor can be
   passed for "query"/"key"/"value" to represent padding more
   efficiently than using a padding mask. In this case, a NestedTensor
   will be returned, and an additional speedup proportional to the
   fraction of the input that is padding can be expected.
Parameters:
      * embed_dim -- Total dimension of the model.
  * **num_heads** -- Number of parallel attention heads. Note that
    "embed_dim" will be split across "num_heads" (i.e. each head
    will have dimension "embed_dim // num_heads").

  * **dropout** -- Dropout probability on "attn_output_weights".
