class_docstr: This is the configuration class to store the configuration of a [`IdeficsModel`]. It is used to instantiate an
  Idefics model according to the specified arguments, defining the model architecture. Instantiating a configuration
  with the defaults will yield a similar configuration to that of the Idefics-9B.
  e.g. [HuggingFaceM4/idefics-9b](https://huggingface.co/HuggingFaceM4/idefics-9b)
  Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
  documentation from [`PretrainedConfig`] for more information.
  Args:
      additional_vocab_size (`int`, *optional`, defaults to 0):
          Additional vocabulary size of the model, typically for the special "<img>" token. Additional vocab tokens
          are always trainable whereas regular vocab tokens can be frozen or not.
      vocab_size (`int`, *optional*, defaults to 32000):
          Vocabulary size of the Idefics model. Defines the number of different tokens that can be represented by the
          `inputs_ids` passed when calling [`~IdeficsModel`]
      hidden_size (`int`, *optional*, defaults to 4096):
          Dimension of the hidden representations.
      intermediate_size (`int`, *optional*, defaults to 11008):
          Dimension of the MLP representations.
      num_hidden_layers (`int`, *optional*, defaults to 32):
          Number of hidden layers in the Transformer encoder.
      num_attention_heads (`int`, *optional*, defaults to 32):
          Number of attention heads for each attention layer in the Transformer encoder.
      dropout (`float`, *optional*, defaults to 0.0):
          The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
      hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
          The non-linear activation function (function or string) in the decoder.
      initializer_range (`float`, *optional*, defaults to 0.02):
          The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
      alpha_initializer (`str`, *optional*, defaults to `"zeros"`):
          Initialization type for the alphas.
      alphas_initializer_range (`float`, *optional*, defaults to 0.0):
          The standard deviation of the truncated_normal_initializer for initializing the alphas in the Gated Cross
          Attention.
      alpha_type (`str`, *optional*, defaults to `"float"`):
          Whether the gating alphas should be vectors or single floats.
      rms_norm_eps (`float`, *optional*, defaults to 1e-6):
          The epsilon used by the rms normalization layers.
      use_cache (`bool`, *optional*, defaults to `True`):
          Whether or not the model should return the last key/values attentions (not used by all models). Only
          relevant if `config.is_decoder=True`.
      pad_token_id (`int`, *optional*, defaults to 0)
          Padding token id.
      bos_token_id (`int`, *optional*, defaults to 1)
          Beginning of stream token id.
      eos_token_id (`int`, *optional*, defaults to 2)
          End of stream token id.
      tie_word_embeddings(`bool`, *optional*, defaults to `False`):
          Whether to tie weight embeddings
      cross_layer_interval (`int`, *optional*, default to 1)
          Interval for cross attention (from text to image) layers.
      qk_layer_norms (`bool`, *optional*, defaults to `False`): Whether to add layer norm after q and k
      freeze_text_layers (`bool`, *optional*, defaults to `True`): Whether to freeze text layers
      freeze_text_module_exceptions (`bool`, *optional*, defaults to `[]`):
          Exceptions to freezing text layers when `freeze_text_layers` is `True`
      freeze_lm_head (`bool`, *optional*, defaults to `False`): Whether to freeze lm head
      freeze_vision_layers (`bool`, *optional*, defaults to `True`):  Whether to freeze vision layers
      freeze_vision_module_exceptions (`bool`, *optional*, defaults to `[]`):
          Exceptions to freezing vision layers when `freeze_vision_layers` is `True`
      use_resampler (`bool`, *optional*, defaults to `False`): Whether to use the Resampler
      vision_config (`IdeficsVisionConfig`,  *optional*): Custom vision config or dict
      perceiver_config (`IdeficsPerceiverConfig`,  *optional*): Custom perceiver config or dict
  Example:
  ```python
  >>> from transformers import IdeficsModel, IdeficsConfig
  >>> # Initializing a Idefics idefics-9b style configuration
  >>> configuration = IdeficsConfig()
  >>> # Initializing a model from the idefics-9b style configuration
  >>> model = IdeficsModel(configuration)
  >>> # Accessing the model configuration
  >>> configuration = model.config
  ```
class_docstr_tok: ["This","is","the","configuration","class","to","store","the","configuration","of","a","[","`","IdeficsModel","`","]",".","It","is","used","to","instantiate","an","Idefics","model","according","to","the","specified","arguments",",","defining","the","model","architecture",".","Instantiating","a","configuration","with","the","defaults","will","yield","a","similar","configuration","to","that","of","the","Idefics-9B",".","e.g",".","[","HuggingFaceM4\/idefics-9b","]","(","https",":","\/\/huggingface.co\/HuggingFaceM4\/idefics-9b",")","Configuration","objects","inherit","from","[","`","PretrainedConfig","`","]","and","can","be","used","to","control","the","model","outputs",".","Read","the","documentation","from","[","`","PretrainedConfig","`","]","for","more","information",".","Args",":","additional_vocab_size","(","`","int","`",",","*","optional","`",",","defaults","to","0",")",":","Additional","vocabulary","size","of","the","model",",","typically","for","the","special","``","<","img",">","''","token",".","Additional","vocab","tokens","are","always","trainable","whereas","regular","vocab","tokens","can","be","frozen","or","not",".","vocab_size","(","`","int","`",",","*","optional","*",",","defaults","to","32000",")",":","Vocabulary","size","of","the","Idefics","model",".","Defines","the","number","of","different","tokens","that","can","be","represented","by","the","`","inputs_ids","`","passed","when","calling","[","`","~IdeficsModel","`","]","hidden_size","(","`","int","`",",","*","optional","*",",","defaults","to","4096",")",":","Dimension","of","the","hidden","representations",".","intermediate_size","(","`","int","`",",","*","optional","*",",","defaults","to","11008",")",":","Dimension","of","the","MLP","representations",".","num_hidden_layers","(","`","int","`",",","*","optional","*",",","defaults","to","32",")",":","Number","of","hidden","layers","in","the","Transformer","encoder",".","num_attention_heads","(","`","int","`",",","*","optional","*",",","defaults","to","32",")",":","Number","of","attention","heads","for","each","attention","layer","in","the","Transformer","encoder",".","dropout","(","`","float","`",",","*","optional","*",",","defaults","to","0.0",")",":","The","dropout","probability","for","all","fully","connected","layers","in","the","embeddings",",","encoder",",","and","pooler",".","hidden_act","(","`","str","`","or","`","function","`",",","*","optional","*",",","defaults","to","`","``","silu","''","`",")",":","The","non-linear","activation","function","(","function","or","string",")","in","the","decoder",".","initializer_range","(","`","float","`",",","*","optional","*",",","defaults","to","0.02",")",":","The","standard","deviation","of","the","truncated_normal_initializer","for","initializing","all","weight","matrices",".","alpha_initializer","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","zeros","''","`",")",":","Initialization","type","for","the","alphas",".","alphas_initializer_range","(","`","float","`",",","*","optional","*",",","defaults","to","0.0",")",":","The","standard","deviation","of","the","truncated_normal_initializer","for","initializing","the","alphas","in","the","Gated","Cross","Attention",".","alpha_type","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","float","''","`",")",":","Whether","the","gating","alphas","should","be","vectors","or","single","floats",".","rms_norm_eps","(","`","float","`",",","*","optional","*",",","defaults","to","1e-6",")",":","The","epsilon","used","by","the","rms","normalization","layers",".","use_cache","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","or","not","the","model","should","return","the","last","key\/values","attentions","(","not","used","by","all","models",")",".","Only","relevant","if","`","config.is_decoder=True","`",".","pad_token_id","(","`","int","`",",","*","optional","*",",","defaults","to","0",")","Padding","token","id",".","bos_token_id","(","`","int","`",",","*","optional","*",",","defaults","to","1",")","Beginning","of","stream","token","id",".","eos_token_id","(","`","int","`",",","*","optional","*",",","defaults","to","2",")","End","of","stream","token","id",".","tie_word_embeddings","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","tie","weight","embeddings","cross_layer_interval","(","`","int","`",",","*","optional","*",",","default","to","1",")","Interval","for","cross","attention","(","from","text","to","image",")","layers",".","qk_layer_norms","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","add","layer","norm","after","q","and","k","freeze_text_layers","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","to","freeze","text","layers","freeze_text_module_exceptions","(","`","bool","`",",","*","optional","*",",","defaults","to","`","[","]","`",")",":","Exceptions","to","freezing","text","layers","when","`","freeze_text_layers","`","is","`","True","`","freeze_lm_head","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","freeze","lm","head","freeze_vision_layers","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","to","freeze","vision","layers","freeze_vision_module_exceptions","(","`","bool","`",",","*","optional","*",",","defaults","to","`","[","]","`",")",":","Exceptions","to","freezing","vision","layers","when","`","freeze_vision_layers","`","is","`","True","`","use_resampler","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","to","use","the","Resampler","vision_config","(","`","IdeficsVisionConfig","`",",","*","optional","*",")",":","Custom","vision","config","or","dict","perceiver_config","(","`","IdeficsPerceiverConfig","`",",","*","optional","*",")",":","Custom","perceiver","config","or","dict","Example",":","``","`","python",">",">",">","from","transformers","import","IdeficsModel",",","IdeficsConfig",">",">",">","#","Initializing","a","Idefics","idefics-9b","style","configuration",">",">",">","configuration","=","IdeficsConfig","(",")",">",">",">","#","Initializing","a","model","from","the","idefics-9b","style","configuration",">",">",">","model","=","IdeficsModel","(","configuration",")",">",">",">","#","Accessing","the","model","configuration",">",">",">","configuration","=","model.config","``","`"]
code: def __init__(
      self,
      vocab_size=32000,
      additional_vocab_size=0,
      hidden_size=4096,
      intermediate_size=11008,
      num_hidden_layers=32,
      num_attention_heads=32,
      dropout=0.0,
      hidden_act="silu",
      initializer_range=0.02,
      alpha_initializer="zeros",
      alphas_initializer_range=0.0,
      alpha_type="float",
      rms_norm_eps=1e-6,
      use_cache=True,
      pad_token_id=0,
      bos_token_id=1,
      eos_token_id=2,
      tie_word_embeddings=False,
      cross_layer_interval=1,
      qk_layer_norms=False,
      freeze_text_layers=True,
      freeze_text_module_exceptions=[],
      freeze_lm_head=False,
      freeze_vision_layers=True,
      freeze_vision_module_exceptions=[],
      use_resampler=False,
      vision_config=None,
      perceiver_config=None,
      **kwargs,
  ):
      self.vocab_size = vocab_size
      self.additional_vocab_size = additional_vocab_size
      self.hidden_size = hidden_size
      self.intermediate_size = intermediate_size
      self.num_hidden_layers = num_hidden_layers
      self.num_attention_heads = num_attention_heads
      self.dropout = dropout
      self.hidden_act = hidden_act
      self.initializer_range = initializer_range
      self.alpha_initializer = alpha_initializer
      self.alphas_initializer_range = alphas_initializer_range
      self.alpha_type = alpha_type
      self.rms_norm_eps = rms_norm_eps
      self.use_cache = use_cache
  
      self.cross_layer_interval = cross_layer_interval
      self.qk_layer_norms = qk_layer_norms
      self.freeze_vision_layers = freeze_vision_layers
  
      self.freeze_text_layers = freeze_text_layers
      self.freeze_text_module_exceptions = freeze_text_module_exceptions
      self.freeze_vision_module_exceptions = (
          freeze_vision_module_exceptions
      )
      self.freeze_lm_head = freeze_lm_head
  
      self.use_resampler = use_resampler
  
      if perceiver_config is None:
          self.perceiver_config = IdeficsPerceiverConfig()
      elif isinstance(perceiver_config, dict):
          self.perceiver_config = IdeficsPerceiverConfig(
              **perceiver_config
          )
      elif isinstance(perceiver_config, IdeficsPerceiverConfig):
          self.perceiver_config = perceiver_config
  
      if vision_config is None:
          self.vision_config = IdeficsVisionConfig()
      elif isinstance(vision_config, dict):
          self.vision_config = IdeficsVisionConfig(**vision_config)
      elif isinstance(vision_config, IdeficsVisionConfig):
          self.vision_config = vision_config
  
      super().__init__(
          pad_token_id=pad_token_id,
          bos_token_id=bos_token_id,
          eos_token_id=eos_token_id,
          tie_word_embeddings=tie_word_embeddings,
          **kwargs,
      )
  
file_path: text-generation-inference/server/text_generation_server/models/custom_modeling/idefics_config.py
imports: import copy
  from transformers import PretrainedConfig
label_desc: Use image node_id 1 to create a new IdeficsConfig object from inherited base classes: PretrainedConfig with example: obj = IdeficsConfig(vocab_size, additional_vocab_size, hidden_size, intermediate_size, num_hidden_layers, num_attention_heads, dropout, hidden_act, initializer_range, alpha_initializer, alphas_initializer_range, alpha_type, rms_norm_eps, use_cache, pad_token_id, bos_token_id, eos_token_id, tie_word_embeddings, cross_layer_interval, qk_layer_norms, freeze_text_layers, freeze_text_module_exceptions, freeze_lm_head, freeze_vision_layers, freeze_vision_module_exceptions, use_resampler, vision_config, perceiver_config)