class_docstr: SelfAttention Layer
  
  Given $X\in \mathbb{R}^{n   imes in_feature}$, the attention is calculated by: $a=softmax(W_2tanh(W_1X))$, where
  $W_1 \in \mathbb{R}^{hidden         imes in_feature}$, $W_2 \in \mathbb{R}^{out_feature     imes hidden}$.
  The final output is $y=aX$ where $y \in \mathbb{R}^{n       imes out_feature}$.
  
  Parameters
  ----------
  in_features: int
      Dimension of input features
  out_features: int
      Dimension of output features
  hidden_size: int
      Dimension of hidden layer
class_docstr_tok: ["SelfAttention","Layer","Given","$","X\\in","\\mathbb","{","R","}","^","{","n","imes","in_feature","}","$",",","the","attention","is","calculated","by",":","$","a=softmax","(","W_2tanh","(","W_1X",")",")","$",",","where","$","W_1","\\in","\\mathbb","{","R","}","^","{","hidden","imes","in_feature","}","$",",","$","W_2","\\in","\\mathbb","{","R","}","^","{","out_feature","imes","hidden","}","$",".","The","final","output","is","$","y=aX","$","where","$","y","\\in","\\mathbb","{","R","}","^","{","n","imes","out_feature","}","$",".","Parameters","--","--","--","--","--","in_features",":","int","Dimension","of","input","features","out_features",":","int","Dimension","of","output","features","hidden_size",":","int","Dimension","of","hidden","layer"]
code: def __init__(self, in_features, out_features, hidden_size=128):
      super(SelfAttention, self).__init__()
      self.w1 = torch.nn.Parameter(
          torch.FloatTensor(hidden_size, in_features)
      )
      self.w2 = torch.nn.Parameter(
          torch.FloatTensor(out_features, hidden_size)
      )
      self.reset_parameters()
  
file_path: deepchem/deepchem/models/torch_models/attention.py
imports: import math
  from typing import Optional
label_desc: Use image node_id 1 to create a new SelfAttention object from inherited base classes: nn with example: obj = SelfAttention(in_features, out_features, hidden_size)