class_docstr: This is the configuration class to store the configuration of a [`DistilBertModel`] or a [`TFDistilBertModel`]. It
  is used to instantiate a DistilBERT model according to the specified arguments, defining the model architecture.
  Instantiating a configuration with the defaults will yield a similar configuration to that of the DistilBERT
  [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) architecture.
  
  Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
  documentation from [`PretrainedConfig`] for more information.
  
  Args:
      vocab_size (`int`, *optional*, defaults to 30522):
          Vocabulary size of the DistilBERT model. Defines the number of different tokens that can be represented by
          the `inputs_ids` passed when calling [`DistilBertModel`] or [`TFDistilBertModel`].
      max_position_embeddings (`int`, *optional*, defaults to 512):
          The maximum sequence length that this model might ever be used with. Typically set this to something large
          just in case (e.g., 512 or 1024 or 2048).
      sinusoidal_pos_embds (`boolean`, *optional*, defaults to `False`):
          Whether to use sinusoidal positional embeddings.
      n_layers (`int`, *optional*, defaults to 6):
          Number of hidden layers in the Transformer encoder.
      n_heads (`int`, *optional*, defaults to 12):
          Number of attention heads for each attention layer in the Transformer encoder.
      dim (`int`, *optional*, defaults to 768):
          Dimensionality of the encoder layers and the pooler layer.
      hidden_dim (`int`, *optional*, defaults to 3072):
          The size of the "intermediate" (often named feed-forward) layer in the Transformer encoder.
      dropout (`float`, *optional*, defaults to 0.1):
          The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
      attention_dropout (`float`, *optional*, defaults to 0.1):
          The dropout ratio for the attention probabilities.
      activation (`str` or `Callable`, *optional*, defaults to `"gelu"`):
          The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
          `"relu"`, `"silu"` and `"gelu_new"` are supported.
      initializer_range (`float`, *optional*, defaults to 0.02):
          The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
      qa_dropout (`float`, *optional*, defaults to 0.1):
          The dropout probabilities used in the question answering model [`DistilBertForQuestionAnswering`].
      seq_classif_dropout (`float`, *optional*, defaults to 0.2):
          The dropout probabilities used in the sequence classification and the multiple choice model
          [`DistilBertForSequenceClassification`].
  
  Examples:
  
  ```python
  >>> from transformers import DistilBertConfig, DistilBertModel
  
  >>> # Initializing a DistilBERT configuration
  >>> configuration = DistilBertConfig()
  
  >>> # Initializing a model (with random weights) from the configuration
  >>> model = DistilBertModel(configuration)
  
  >>> # Accessing the model configuration
  >>> configuration = model.config
  ```
class_docstr_tok: ["This","is","the","configuration","class","to","store","the","configuration","of","a","[","`","DistilBertModel","`","]","or","a","[","`","TFDistilBertModel","`","]",".","It","is","used","to","instantiate","a","DistilBERT","model","according","to","the","specified","arguments",",","defining","the","model","architecture",".","Instantiating","a","configuration","with","the","defaults","will","yield","a","similar","configuration","to","that","of","the","DistilBERT","[","distilbert-base-uncased","]","(","https",":","\/\/huggingface.co\/distilbert-base-uncased",")","architecture",".","Configuration","objects","inherit","from","[","`","PretrainedConfig","`","]","and","can","be","used","to","control","the","model","outputs",".","Read","the","documentation","from","[","`","PretrainedConfig","`","]","for","more","information",".","Args",":","vocab_size","(","`","int","`",",","*","optional","*",",","defaults","to","30522",")",":","Vocabulary","size","of","the","DistilBERT","model",".","Defines","the","number","of","different","tokens","that","can","be","represented","by","the","`","inputs_ids","`","passed","when","calling","[","`","DistilBertModel","`","]","or","[","`","TFDistilBertModel","`","]",".","max_position_embeddings","(","`","int","`",",","*","optional","*",",","defaults","to","512",")",":","The","maximum","sequence","length","that","this","model","might","ever","be","used","with",".","Typically","set","this","to","something","large","just","in","case","(","e.g.",",","512","or","1024","or","2048",")",".","sinusoidal_pos_embds","(","`","boolean","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","use","sinusoidal","positional","embeddings",".","n_layers","(","`","int","`",",","*","optional","*",",","defaults","to","6",")",":","Number","of","hidden","layers","in","the","Transformer","encoder",".","n_heads","(","`","int","`",",","*","optional","*",",","defaults","to","12",")",":","Number","of","attention","heads","for","each","attention","layer","in","the","Transformer","encoder",".","dim","(","`","int","`",",","*","optional","*",",","defaults","to","768",")",":","Dimensionality","of","the","encoder","layers","and","the","pooler","layer",".","hidden_dim","(","`","int","`",",","*","optional","*",",","defaults","to","3072",")",":","The","size","of","the","``","intermediate","''","(","often","named","feed-forward",")","layer","in","the","Transformer","encoder",".","dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.1",")",":","The","dropout","probability","for","all","fully","connected","layers","in","the","embeddings",",","encoder",",","and","pooler",".","attention_dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.1",")",":","The","dropout","ratio","for","the","attention","probabilities",".","activation","(","`","str","`","or","`","Callable","`",",","*","optional","*",",","defaults","to","`","``","gelu","''","`",")",":","The","non-linear","activation","function","(","function","or","string",")","in","the","encoder","and","pooler",".","If","string",",","`","``","gelu","''","`",",","`","``","relu","''","`",",","`","``","silu","''","`","and","`","``","gelu_new","''","`","are","supported",".","initializer_range","(","`","float","`",",","*","optional","*",",","defaults","to","0.02",")",":","The","standard","deviation","of","the","truncated_normal_initializer","for","initializing","all","weight","matrices",".","qa_dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.1",")",":","The","dropout","probabilities","used","in","the","question","answering","model","[","`","DistilBertForQuestionAnswering","`","]",".","seq_classif_dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.2",")",":","The","dropout","probabilities","used","in","the","sequence","classification","and","the","multiple","choice","model","[","`","DistilBertForSequenceClassification","`","]",".","Examples",":","``","`","python",">",">",">","from","transformers","import","DistilBertConfig",",","DistilBertModel",">",">",">","#","Initializing","a","DistilBERT","configuration",">",">",">","configuration","=","DistilBertConfig","(",")",">",">",">","#","Initializing","a","model","(","with","random","weights",")","from","the","configuration",">",">",">","model","=","DistilBertModel","(","configuration",")",">",">",">","#","Accessing","the","model","configuration",">",">",">","configuration","=","model.config","``","`"]
code: def __init__(
      self,
      vocab_size=30522,
      max_position_embeddings=512,
      sinusoidal_pos_embds=False,
      n_layers=6,
      n_heads=12,
      dim=768,
      hidden_dim=4 * 768,
      dropout=0.1,
      attention_dropout=0.1,
      activation="gelu",
      initializer_range=0.02,
      qa_dropout=0.1,
      seq_classif_dropout=0.2,
      pad_token_id=0,
      **kwargs,
  ):
      self.vocab_size = vocab_size
      self.max_position_embeddings = max_position_embeddings
      self.sinusoidal_pos_embds = sinusoidal_pos_embds
      self.n_layers = n_layers
      self.n_heads = n_heads
      self.dim = dim
      self.hidden_dim = hidden_dim
      self.dropout = dropout
      self.attention_dropout = attention_dropout
      self.activation = activation
      self.initializer_range = initializer_range
      self.qa_dropout = qa_dropout
      self.seq_classif_dropout = seq_classif_dropout
      super().__init__(**kwargs, pad_token_id=pad_token_id)
  
file_path: transformers/src/transformers/models/distilbert/configuration_distilbert.py
imports: from collections import OrderedDict
  from typing import Mapping
  from ...configuration_utils import PretrainedConfig
  from ...onnx import OnnxConfig
  from ...utils import logging
label_desc: Use image node_id 1 to create a new DistilBertConfig object from inherited base classes: PretrainedConfig with example: obj = DistilBertConfig(vocab_size, max_position_embeddings, sinusoidal_pos_embds, n_layers, n_heads, dim, hidden_dim, dropout, attention_dropout, activation, initializer_range, qa_dropout, seq_classif_dropout, pad_token_id)