class_docstr: PyTorch implementation containing all the modelling needed for your NLP task. Combines a language
  model and a prediction head. Allows for gradient flow back to the language model component.
class_docstr_tok: ["PyTorch","implementation","containing","all","the","modelling","needed","for","your","NLP","task",".","Combines","a","language","model","and","a","prediction","head",".","Allows","for","gradient","flow","back","to","the","language","model","component","."]
docstr: :param language_model: Any model that turns token ids into vector representations
  :type language_model: LanguageModel
  :param prediction_heads: A list of models that take embeddings and return logits for a given task
  :type prediction_heads: list
  :param embeds_dropout_prob: The probability that a value in the embeddings returned by the
     language model will be zeroed.
  :param embeds_dropout_prob: float
  :param lm_output_types: How to extract the embeddings from the final layer of the language model. When set
                          to "per_token", one embedding will be extracted per input token. If set to
                          "per_sequence", a single embedding will be extracted to represent the full
                          input sequence. Can either be a single string, or a list of strings,
                          one for each prediction head.
  :type lm_output_types: list or str
  :param device: The device on which this model will operate. Either "cpu" or "cuda".
  :param loss_aggregation_fn: Function to aggregate the loss of multiple prediction heads.
                              Input: loss_per_head (list of tensors), global_step (int), batch (dict)
                              Output: aggregated loss (tensor)
                              Default is a simple sum:
                              `lambda loss_per_head, global_step=None, batch=None: sum(tensors)`
                              However, you can pass more complex functions that depend on the
                              current step (e.g. for round-robin style multitask learning) or the actual
                              content of the batch (e.g. certain labels)
                              Note: The loss at this stage is per sample, i.e one tensor of
                              shape (batchsize) per prediction head.
  :type loss_aggregation_fn: function
code: def __init__(
      self,
      language_model,
      prediction_heads,
      embeds_dropout_prob,
      lm_output_types,
      device,
      loss_aggregation_fn=None,
  ):
      """
      :param language_model: Any model that turns token ids into vector representations
      :type language_model: LanguageModel
      :param prediction_heads: A list of models that take embeddings and return logits for a given task
      :type prediction_heads: list
      :param embeds_dropout_prob: The probability that a value in the embeddings returned by the
         language model will be zeroed.
      :param embeds_dropout_prob: float
      :param lm_output_types: How to extract the embeddings from the final layer of the language model. When set
                              to "per_token", one embedding will be extracted per input token. If set to
                              "per_sequence", a single embedding will be extracted to represent the full
                              input sequence. Can either be a single string, or a list of strings,
                              one for each prediction head.
      :type lm_output_types: list or str
      :param device: The device on which this model will operate. Either "cpu" or "cuda".
      :param loss_aggregation_fn: Function to aggregate the loss of multiple prediction heads.
                                  Input: loss_per_head (list of tensors), global_step (int), batch (dict)
                                  Output: aggregated loss (tensor)
                                  Default is a simple sum:
                                  `lambda loss_per_head, global_step=None, batch=None: sum(tensors)`
                                  However, you can pass more complex functions that depend on the
                                  current step (e.g. for round-robin style multitask learning) or the actual
                                  content of the batch (e.g. certain labels)
                                  Note: The loss at this stage is per sample, i.e one tensor of
                                  shape (batchsize) per prediction head.
      :type loss_aggregation_fn: function
      """
  
      super(AdaptiveModel, self).__init__()
      self.device = device
      self.language_model = language_model.to(device)
      self.lm_output_dims = language_model.get_output_dims()
      self.prediction_heads = nn.ModuleList(
          [ph.to(device) for ph in prediction_heads]
      )
      self.fit_heads_to_lm()
      # set shared weights for LM finetuning
      for head in self.prediction_heads:
          if head.model_type == "language_modelling":
              head.set_shared_weights(
                  language_model.model.embeddings.word_embeddings.weight
              )
      self.dropout = nn.Dropout(embeds_dropout_prob)
      self.lm_output_types = (
          [lm_output_types]
          if isinstance(lm_output_types, str)
          else lm_output_types
      )
      self.log_params()
      # default loss aggregation function is a simple sum (without using any of the optional params)
      if not loss_aggregation_fn:
          loss_aggregation_fn = loss_per_head_sum
      self.loss_aggregation_fn = loss_aggregation_fn
  
imports: import copy
  import json
  import logging
  import os
  from pathlib import Path
  import multiprocessing
  import numpy
  import torch
  from torch import nn
  from transformers import AutoConfig
  from transformers.convert_graph_to_onnx import convert, quantize
  from farm.data_handler.processor import Processor
  from farm.modeling.language_model import LanguageModel
  from farm.modeling.prediction_head import PredictionHead, pick_single_fn
  from farm.utils import MLFlowLogger, stack
  import farm.conversion.transformers
label_desc: Use image node_id 1 to create a new AdaptiveModel object from inherited base classes: nn, BaseAdaptiveModel with example: obj = AdaptiveModel(language_model, prediction_heads, embeds_dropout_prob, lm_output_types, device, loss_aggregation_fn)