class_docstr: Vanilla transformer block from the original paper "Attention is all you need"
  Detailed analysis: https://theaisummer.com/transformer/
class_docstr_tok: ["Vanilla","transformer","block","from","the","original","paper","``","Attention","is","all","you","need","''","Detailed","analysis",":","https",":","\/\/theaisummer.com\/transformer\/"]
docstr: Args:
      dim: token's vector length
      heads: number of heads
      dim_head: if none dim/heads is used
      dim_linear_block: the inner projection dim
      dropout: probability of droppping values
      mhsa: if provided you can change the vanilla self-attention block
      prenorm: if the layer norm will be applied before the mhsa or after
code: def __init__(
      self,
      dim,
      heads=8,
      dim_head=None,
      dim_linear_block=1024,
      dropout=0.1,
      activation=nn.GELU,
      mhsa=None,
      prenorm=False,
  ):
      """
      Args:
          dim: token's vector length
          heads: number of heads
          dim_head: if none dim/heads is used
          dim_linear_block: the inner projection dim
          dropout: probability of droppping values
          mhsa: if provided you can change the vanilla self-attention block
          prenorm: if the layer norm will be applied before the mhsa or after
      """
      super().__init__()
      self.mhsa = (
          mhsa
          if mhsa is not None
          else MultiHeadSelfAttention(
              dim=dim, heads=heads, dim_head=dim_head
          )
      )
      self.prenorm = prenorm
      self.drop = nn.Dropout(dropout)
      self.norm_1 = nn.LayerNorm(dim)
      self.norm_2 = nn.LayerNorm(dim)
  
      self.linear = nn.Sequential(
          nn.Linear(dim, dim_linear_block),
          activation(),  # nn.ReLU or nn.GELU
          nn.Dropout(dropout),
          nn.Linear(dim_linear_block, dim),
          nn.Dropout(dropout),
      )
  
imports: from typing import List, Tuple
  import numpy
  import paddle
  import paddle.nn
  from paddle import Tensor
  from medicalseg.cvlibs import manager
label_desc: Use image node_id 1 to create a new TransformerBlock object from inherited base classes: nn with example: obj = TransformerBlock(dim, heads, dim_head, dim_linear_block, dropout, activation, mhsa, prenorm)