class_docstr: Implement the scaled dot product attention with softmax.
  Arguments
  ---------
      softmax_scale: The temperature to use for the softmax attention.
                    (default: 1/sqrt(d_keys) where d_keys is computed at
                    runtime)
      attention_dropout: The dropout rate to apply to the attention
                         (default: 0.0)
code: def __init__(
      self,
      causal=False,
      softmax_scale=None,
      attention_dropout=0.0,
      device=None,
      dtype=None,
  ):
      super().__init__()
      self.causal = causal
      self.softmax_scale = softmax_scale
      self.dropout_p = attention_dropout
  
imports: import math
  from functools import partial
  import torch
  import torch.nn
  import torch.nn.functional
  from einops import rearrange
label_desc: Use image node_id 1 to create a new SelfAttention object from inherited base classes: nn with example: obj = SelfAttention(causal, softmax_scale, attention_dropout, device, dtype)