class_docstr: Transformer block described in ViT.
  
  Paper: https://arxiv.org/abs/2010.11929
  Based on: https://github.com/lucidrains/vit-pytorch
  
  Args:
      dim: input dimension.
      depth: depth for transformer block.
      heads: number of heads in multi-head attention layer.
      dim_head: head size.
      mlp_dim: dimension of the FeedForward layer.
      dropout: dropout ratio, defaults to 0.
class_docstr_tok: ["Transformer","block","described","in","ViT",".","Paper",":","https",":","\/\/arxiv.org\/abs\/2010.11929","Based","on",":","https",":","\/\/github.com\/lucidrains\/vit-pytorch","Args",":","dim",":","input","dimension",".","depth",":","depth","for","transformer","block",".","heads",":","number","of","heads","in","multi-head","attention","layer",".","dim_head",":","head","size",".","mlp_dim",":","dimension","of","the","FeedForward","layer",".","dropout",":","dropout","ratio",",","defaults","to","0","."]
code: def __init__(
      self,
      dim: int,
      depth: int,
      heads: int,
      dim_head: int,
      mlp_dim: int,
      dropout: float = 0.0,
  ) -> None:
      super().__init__()
      self.layers = nn.ModuleList([])
      for _ in range(depth):
          self.layers.append(
              nn.ModuleList(
                  [
                      PreNorm(
                          dim, Attention(dim, heads, dim_head, dropout)
                      ),
                      PreNorm(dim, FeedForward(dim, mlp_dim, dropout)),
                  ]
              )
          )
  
imports: from typing import Any, Dict, Tuple
  import torch
  from torch import nn
  from kornia.core import Module, Tensor
label_desc: Use image node_id 1 to create a new Transformer object from inherited base classes: Module with example: obj = Transformer(dim, depth, heads, dim_head, mlp_dim, dropout)