class_docstr: Here, we use three networks to predict the start scores, end scores and answerable scores.
  
  We formulate p(start, end, answerable | contextual_embedding) as the product of the
  following three terms:
  
  - p(start | contextual_embedding)
  - p(end | start, contextual_embedding)
  - p(answerable | contextual_embedding)
  
  In the inference phase, we are able to use beam search to do the inference.
  
  use_segmentation is used to mark whether we segment the input sentence. In RoBERTa and XLMR,
  this flag is set to True, then the QA model no longer accept `token_types` as valid input.
  
  - use_segmentation=True:
      tokens :      <CLS> Question <SEP> Context <SEP>
      token_typess:  0       0       0      1      1
  
  - use_segmentation=False:
      tokens :      <CLS> Question <SEP> Context <SEP>
      token_typess:  None
class_docstr_tok: ["Here",",","we","use","three","networks","to","predict","the","start","scores",",","end","scores","and","answerable","scores",".","We","formulate","p","(","start",",","end",",","answerable","|","contextual_embedding",")","as","the","product","of","the","following","three","terms",":","-","p","(","start","|","contextual_embedding",")","-","p","(","end","|","start",",","contextual_embedding",")","-","p","(","answerable","|","contextual_embedding",")","In","the","inference","phase",",","we","are","able","to","use","beam","search","to","do","the","inference",".","use_segmentation","is","used","to","mark","whether","we","segment","the","input","sentence",".","In","RoBERTa","and","XLMR",",","this","flag","is","set","to","True",",","then","the","QA","model","no","longer","accept","`","token_types","`","as","valid","input",".","-","use_segmentation=True",":","tokens",":","<","CLS",">","Question","<","SEP",">","Context","<","SEP",">","token_typess",":","0","0","0","1","1","-","use_segmentation=False",":","tokens",":","<","CLS",">","Question","<","SEP",">","Context","<","SEP",">","token_typess",":","None"]
code: def __init__(
      self,
      backbone,
      units=768,
      layer_norm_eps=1e-12,
      dropout_prob=0.1,
      activation="tanh",
      weight_initializer=None,
      bias_initializer=None,
      use_segmentation=True,
  ):
      super().__init__()
      self.backbone = backbone
      self.use_segmentation = use_segmentation
      self.start_scores = nn.Dense(
          1,
          flatten=False,
          weight_initializer=weight_initializer,
          bias_initializer=bias_initializer,
      )
      self.end_scores = nn.HybridSequential()
      self.end_scores.add(
          nn.Dense(
              units,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
      self.end_scores.add(get_activation(activation))
      self.end_scores.add(nn.LayerNorm(epsilon=layer_norm_eps))
      self.end_scores.add(
          nn.Dense(
              1,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
      self.answerable_scores = nn.HybridSequential()
      self.answerable_scores.add(
          nn.Dense(
              units,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
      self.answerable_scores.add(get_activation(activation))
      self.answerable_scores.add(nn.Dropout(dropout_prob))
      self.answerable_scores.add(
          nn.Dense(
              2,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
      self.quantized_backbone = None
  
file_path: gluon-nlp/docs/model_zoo/question_answering/models.py
imports: import mxnet
  from mxnet import np, npx
  from mxnet.gluon import nn, HybridBlock
  from mxnet.util import use_np
  from gluonnlp.layers import get_activation
  from gluonnlp.op import select_vectors_by_position
  from gluonnlp.attention_cell import masked_logsoftmax, masked_softmax
label_desc: Use image node_id 1 to create a new ModelForQAConditionalV1 object from inherited base classes: HybridBlock with example: obj = ModelForQAConditionalV1(backbone, units, layer_norm_eps, dropout_prob, activation, weight_initializer, bias_initializer, use_segmentation)