docstr: Transformer with adapters (small bottleneck layers) 
docstr_tok: ["Transformer","with","adapters","(","small","bottleneck","layers",")"]
code: def __init__(
      self,
      adapters_dim,
      embed_dim,
      hidden_dim,
      num_embeddings,
      num_max_positions,
      num_heads,
      num_layers,
      dropout,
      causal,
  ):
      """Transformer with adapters (small bottleneck layers)"""
      super().__init__(
          embed_dim,
          hidden_dim,
          num_embeddings,
          num_max_positions,
          num_heads,
          num_layers,
          dropout,
          causal,
      )
      self.adapters_1 = torch.nn.ModuleList()
      self.adapters_2 = torch.nn.ModuleList()
      for _ in range(num_layers):
          self.adapters_1.append(
              torch.nn.Sequential(
                  torch.nn.Linear(embed_dim, adapters_dim),
                  torch.nn.ReLU(),
                  torch.nn.Linear(adapters_dim, embed_dim),
              )
          )
  
          self.adapters_2.append(
              torch.nn.Sequential(
                  torch.nn.Linear(embed_dim, adapters_dim),
                  torch.nn.ReLU(),
                  torch.nn.Linear(adapters_dim, embed_dim),
              )
          )
  
imports: import torch
  from pytorch_pretrained_bert import BertTokenizer
  from transfer_nlp.plugins.config import register_plugin
label_desc: Use image node_id 1 to create a new TransformerWithAdapters object from inherited base classes: Transformer with example: obj = TransformerWithAdapters(adapters_dim, embed_dim, hidden_dim, num_embeddings, num_max_positions, num_heads, num_layers, dropout, causal)