class_docstr: Implements the DETR transformer.
  
  Following the official DETR implementation, this module copy-paste
  from torch.nn.Transformer with modifications:
  
      * positional encodings are passed in MultiheadAttention
      * extra LN at the end of encoder is removed
      * decoder returns a stack of activations from all decoding layers
  
  See `paper: End-to-End Object Detection with Transformers
  <https://arxiv.org/pdf/2005.12872>`_ for details.
  
  Args:
      embed_dims (int): The feature dimension.
      num_heads (int): Parallel attention heads. Same as
          `nn.MultiheadAttention`.
      num_encoder_layers (int): Number of `TransformerEncoderLayer`.
      num_decoder_layers (int): Number of `TransformerDecoderLayer`.
      feedforward_channels (int): The hidden dimension for FFNs used in both
          encoder and decoder.
      dropout (float): Probability of an element to be zeroed. Default 0.0.
      act_cfg (dict): Activation config for FFNs used in both encoder
          and decoder. Defalut ReLU.
      norm_cfg (dict): Config dict for normalization used in both encoder
          and decoder. Default layer normalization.
      num_fcs (int): The number of fully-connected layers in FFNs, which is
          used for both encoder and decoder.
      pre_norm (bool): Whether the normalization layer is ordered
          first in the encoder and decoder. Default False.
      return_intermediate_dec (bool): Whether to return the intermediate
          output from each TransformerDecoderLayer or only the last
          TransformerDecoderLayer. Default False. If False, the returned
          `hs` has shape [num_decoder_layers, bs, num_query, embed_dims].
          If True, the returned `hs` will have shape [1, bs, num_query,
          embed_dims].
class_docstr_tok: ["Implements","the","DETR","transformer",".","Following","the","official","DETR","implementation",",","this","module","copy-paste","from","torch.nn.Transformer","with","modifications",":","*","positional","encodings","are","passed","in","MultiheadAttention","*","extra","LN","at","the","end","of","encoder","is","removed","*","decoder","returns","a","stack","of","activations","from","all","decoding","layers","See","`","paper",":","End-to-End","Object","Detection","with","Transformers","<","https",":","\/\/arxiv.org\/pdf\/2005.12872",">","`","_","for","details",".","Args",":","embed_dims","(","int",")",":","The","feature","dimension",".","num_heads","(","int",")",":","Parallel","attention","heads",".","Same","as","`","nn.MultiheadAttention","`",".","num_encoder_layers","(","int",")",":","Number","of","`","TransformerEncoderLayer","`",".","num_decoder_layers","(","int",")",":","Number","of","`","TransformerDecoderLayer","`",".","feedforward_channels","(","int",")",":","The","hidden","dimension","for","FFNs","used","in","both","encoder","and","decoder",".","dropout","(","float",")",":","Probability","of","an","element","to","be","zeroed",".","Default","0.0.","act_cfg","(","dict",")",":","Activation","config","for","FFNs","used","in","both","encoder","and","decoder",".","Defalut","ReLU",".","norm_cfg","(","dict",")",":","Config","dict","for","normalization","used","in","both","encoder","and","decoder",".","Default","layer","normalization",".","num_fcs","(","int",")",":","The","number","of","fully-connected","layers","in","FFNs",",","which","is","used","for","both","encoder","and","decoder",".","pre_norm","(","bool",")",":","Whether","the","normalization","layer","is","ordered","first","in","the","encoder","and","decoder",".","Default","False",".","return_intermediate_dec","(","bool",")",":","Whether","to","return","the","intermediate","output","from","each","TransformerDecoderLayer","or","only","the","last","TransformerDecoderLayer",".","Default","False",".","If","False",",","the","returned","`","hs","`","has","shape","[","num_decoder_layers",",","bs",",","num_query",",","embed_dims","]",".","If","True",",","the","returned","`","hs","`","will","have","shape","[","1",",","bs",",","num_query",",","embed_dims","]","."]
code: def __init__(
      self,
      embed_dims=512,
      num_heads=8,
      num_encoder_layers=6,
      num_decoder_layers=6,
      feedforward_channels=2048,
      dropout=0.0,
      act_cfg=dict(type="ReLU", inplace=True),
      norm_cfg=dict(type="LN"),
      num_fcs=2,
      pre_norm=False,
      return_intermediate_dec=False,
  ):
      super(Transformer, self).__init__()
      self.embed_dims = embed_dims
      self.num_heads = num_heads
      self.num_encoder_layers = num_encoder_layers
      self.num_decoder_layers = num_decoder_layers
      self.feedforward_channels = feedforward_channels
      self.dropout = dropout
      self.act_cfg = act_cfg
      self.norm_cfg = norm_cfg
      self.num_fcs = num_fcs
      self.pre_norm = pre_norm
      self.return_intermediate_dec = return_intermediate_dec
      if self.pre_norm:
          encoder_order = ("norm", "selfattn", "norm", "ffn")
          decoder_order = (
              "norm",
              "selfattn",
              "norm",
              "multiheadattn",
              "norm",
              "ffn",
          )
      else:
          encoder_order = ("selfattn", "norm", "ffn", "norm")
          decoder_order = (
              "selfattn",
              "norm",
              "multiheadattn",
              "norm",
              "ffn",
              "norm",
          )
      self.encoder = TransformerEncoder(
          num_encoder_layers,
          embed_dims,
          num_heads,
          feedforward_channels,
          dropout,
          encoder_order,
          act_cfg,
          norm_cfg,
          num_fcs,
      )
      self.decoder = TransformerDecoder(
          num_decoder_layers,
          embed_dims,
          num_heads,
          feedforward_channels,
          dropout,
          decoder_order,
          act_cfg,
          norm_cfg,
          num_fcs,
          return_intermediate_dec,
      )
  
imports: import torch
  import torch.nn
  from mmcv.cnn import Linear, build_activation_layer, build_norm_layer, xavier_init
  from .builder import TRANSFORMER
label_desc: Use image node_id 1 to create a new Transformer object from inherited base classes: nn with example: obj = Transformer(embed_dims, num_heads, num_encoder_layers, num_decoder_layers, feedforward_channels, dropout, act_cfg, norm_cfg, num_fcs, pre_norm, return_intermediate_dec)