class_docstr: [`FlavaConfig`] is the configuration class to store the configuration of a [`FlavaModel`]. It is used to
  instantiate FLAVA model according to the specified arguments, defining the text model, image model, image codebook
  and multimodal model configs. Instantiating a configuration with the defaults will yield a similar configuration to
  that of the FLAVA [facebook/flava-full](https://huggingface.co/facebook/flava-full) architecture.
  
  Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
  documentation from [`PretrainedConfig`] for more information.
  
  Args:
      text_config_dict (`dict`, *optional*):
          Dictionary of configuration options used to initialize [`FlavaTextConfig`].
      image_config_dict (`dict`, *optional*):
          Dictionary of configuration options used to initialize [`FlavaImageConfig`].
      multimodal_config_dict (`dict`, *optional*):
          Dictionary of configuration options used to initialize [`FlavaMultimodalConfig`].
      hidden_size (`int`, *optional*, defaults to 768):
          Dimensionality of the encoder layers and the pooler layer.
      layer_norm_eps (`float`, *optional*, defaults to 1e-12):
          The epsilon used by the layer normalization layers.
      projection_dim (`int`, *optional*, defaults to 512):
          Dimentionality of text and image projection layers.
      logit_scale_init_value (`float`, *optional*, defaults to 2.6592):
          The inital value of the *logit_scale* paramter. Default is used as per the original FLAVA/CLIP
          implementation.
      initializer_range (`float`, *optional*, defaults to 0.02):
          The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
      ce_ignore_index (`int`, *optional*, defaults to -100):
          Cross entropy index to ignore.
      mim_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to MIM (Masked Image Modeling) unimodal loss
      mlm_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to MLM (Masked Language Modeling) unimodal loss
      global_contrastive_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to global contrastive cross-alignment loss.
      itm_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to image-text matching multimodal loss.
      mmm_image_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to MMM loss's image part.
      mmm_text_weight (`float`, *optional*, defaults to 1.0):
          Weight to be assigned to MMM loss's text part.
      global_backprop_contrastive (`bool`, *optional*, defaults to `True`):
          Whether to use global backpropgation through all workers in contrastive loss.
      skip_unmasked_multimodal_encoder (`bool`, *optional*, defaults to `True`):
          Whether to skip running unmasked multimodal encoder whose outputs are not used by FLAVA losses.
      return_loss (`bool`, *optional*, defaults to `True`):
          Whether to return loss or not
  
      kwargs (*optional*):
          Dictionary of keyword arguments.
  
  Example:
  
  ```python
  >>> from transformers import FlavaConfig, FlavaModel, FlavaForPreTraining
  
  >>> # Initializing a FlavaConfig with style configuration
  >>> configuration = FlavaConfig()
  
  >>> # Initializing a FlavaModel and FlavaForPreTraining model (with random weights) from the style configuration
  >>> model = FlavaModel(configuration)
  >>> model_pre = FlavaForPreTraining(configuration)
  
  >>> # Accessing the model configuration
  >>> configuration = model.config
  >>> configuration_pre = model_pre.config
  ```
class_docstr_tok: ["[","`","FlavaConfig","`","]","is","the","configuration","class","to","store","the","configuration","of","a","[","`","FlavaModel","`","]",".","It","is","used","to","instantiate","FLAVA","model","according","to","the","specified","arguments",",","defining","the","text","model",",","image","model",",","image","codebook","and","multimodal","model","configs",".","Instantiating","a","configuration","with","the","defaults","will","yield","a","similar","configuration","to","that","of","the","FLAVA","[","facebook\/flava-full","]","(","https",":","\/\/huggingface.co\/facebook\/flava-full",")","architecture",".","Configuration","objects","inherit","from","[","`","PretrainedConfig","`","]","and","can","be","used","to","control","the","model","outputs",".","Read","the","documentation","from","[","`","PretrainedConfig","`","]","for","more","information",".","Args",":","text_config_dict","(","`","dict","`",",","*","optional","*",")",":","Dictionary","of","configuration","options","used","to","initialize","[","`","FlavaTextConfig","`","]",".","image_config_dict","(","`","dict","`",",","*","optional","*",")",":","Dictionary","of","configuration","options","used","to","initialize","[","`","FlavaImageConfig","`","]",".","multimodal_config_dict","(","`","dict","`",",","*","optional","*",")",":","Dictionary","of","configuration","options","used","to","initialize","[","`","FlavaMultimodalConfig","`","]",".","hidden_size","(","`","int","`",",","*","optional","*",",","defaults","to","768",")",":","Dimensionality","of","the","encoder","layers","and","the","pooler","layer",".","layer_norm_eps","(","`","float","`",",","*","optional","*",",","defaults","to","1e-12",")",":","The","epsilon","used","by","the","layer","normalization","layers",".","projection_dim","(","`","int","`",",","*","optional","*",",","defaults","to","512",")",":","Dimentionality","of","text","and","image","projection","layers",".","logit_scale_init_value","(","`","float","`",",","*","optional","*",",","defaults","to","2.6592",")",":","The","inital","value","of","the","*","logit_scale","*","paramter",".","Default","is","used","as","per","the","original","FLAVA\/CLIP","implementation",".","initializer_range","(","`","float","`",",","*","optional","*",",","defaults","to","0.02",")",":","The","standard","deviation","of","the","truncated_normal_initializer","for","initializing","all","weight","matrices",".","ce_ignore_index","(","`","int","`",",","*","optional","*",",","defaults","to","-100",")",":","Cross","entropy","index","to","ignore",".","mim_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","MIM","(","Masked","Image","Modeling",")","unimodal","loss","mlm_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","MLM","(","Masked","Language","Modeling",")","unimodal","loss","global_contrastive_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","global","contrastive","cross-alignment","loss",".","itm_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","image-text","matching","multimodal","loss",".","mmm_image_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","MMM","loss","'s","image","part",".","mmm_text_weight","(","`","float","`",",","*","optional","*",",","defaults","to","1.0",")",":","Weight","to","be","assigned","to","MMM","loss","'s","text","part",".","global_backprop_contrastive","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","to","use","global","backpropgation","through","all","workers","in","contrastive","loss",".","skip_unmasked_multimodal_encoder","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","to","skip","running","unmasked","multimodal","encoder","whose","outputs","are","not","used","by","FLAVA","losses",".","return_loss","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","to","return","loss","or","not","kwargs","(","*","optional","*",")",":","Dictionary","of","keyword","arguments",".","Example",":","``","`","python",">",">",">","from","transformers","import","FlavaConfig",",","FlavaModel",",","FlavaForPreTraining",">",">",">","#","Initializing","a","FlavaConfig","with","style","configuration",">",">",">","configuration","=","FlavaConfig","(",")",">",">",">","#","Initializing","a","FlavaModel","and","FlavaForPreTraining","model","(","with","random","weights",")","from","the","style","configuration",">",">",">","model","=","FlavaModel","(","configuration",")",">",">",">","model_pre","=","FlavaForPreTraining","(","configuration",")",">",">",">","#","Accessing","the","model","configuration",">",">",">","configuration","=","model.config",">",">",">","configuration_pre","=","model_pre.config","``","`"]
docstr: Instantiate a [`FlavaConfig`] (or a derived class) from flava text model configuration, flava image model
  configuration, flava multimodal model and flava codebook model configuration.
  
  Returns:
      [`FlavaConfig`]: An instance of a configuration object
docstr_tok: ["Instantiate","a","[","`","FlavaConfig","`","]","(","or","a","derived","class",")","from","flava","text","model","configuration",",","flava","image","model","configuration",",","flava","multimodal","model","and","flava","codebook","model","configuration",".","Returns",":","[","`","FlavaConfig","`","]",":","An","instance","of","a","configuration","object"]
code: def from_configs(
      cls,
      image_config: FlavaImageConfig,
      text_config: FlavaTextConfig,
      multimodal_config: FlavaMultimodalConfig,
      image_codebook_config: FlavaImageCodebookConfig,
      **kwargs,
  ):
      r"""
      Instantiate a [`FlavaConfig`] (or a derived class) from flava text model configuration, flava image model
      configuration, flava multimodal model and flava codebook model configuration.
  
      Returns:
          [`FlavaConfig`]: An instance of a configuration object
      """
  
      return cls(
          image_config_dict=image_config.to_dict(),
          text_config_dict=text_config.to_dict(),
          multimodal_config_dict=multimodal_config.to_dict(),
          image_codebook_config_dict=image_codebook_config.to_dict(),
          **kwargs,
      )
  
imports: import copy
  import os
  from typing import Any, Dict, Union
  from ...configuration_utils import PretrainedConfig
  from ...utils import logging
label_desc: Use image node_id 2 for calling the FlavaConfig obj's underlying member method code with example usage: obj.from_configs(cls, image_config, text_config, multimodal_config, image_codebook_config) and returns: cls