class_docstr: It is slightly different from the traditional mask language model which recover the
  masked word (find the matched word in dictionary). The Object of Discriminator in
  Electra is 'replaced token detection' that is a binary classification task to
  predicts every token whether it is an original or a replacement.
class_docstr_tok: ["It","is","slightly","different","from","the","traditional","mask","language","model","which","recover","the","masked","word","(","find","the","matched","word","in","dictionary",")",".","The","Object","of","Discriminator","in","Electra","is","'replaced","token","detection","'","that","is","a","binary","classification","task","to","predicts","every","token","whether","it","is","an","original","or","a","replacement","."]
docstr: Parameters
  ----------
  backbone_cfg
  weight_initializer
  bias_initializer
docstr_tok: ["Parameters","--","--","--","--","--","backbone_cfg","weight_initializer","bias_initializer"]
code: def __init__(
      self, backbone_cfg, weight_initializer=None, bias_initializer=None
  ):
      """
  
      Parameters
      ----------
      backbone_cfg
      weight_initializer
      bias_initializer
      """
      super().__init__()
      self.backbone_model = ElectraModel.from_cfg(backbone_cfg)
      if weight_initializer is None:
          weight_initializer = self.backbone_model.weight_initializer
      if bias_initializer is None:
          bias_initializer = self.backbone_model.bias_initializer
      self.rtd_encoder = nn.HybridSequential()
      # Extra non-linear layer
      self.rtd_encoder.add(
          nn.Dense(
              units=self.backbone_model.units,
              in_units=self.backbone_model.units,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
      self.rtd_encoder.add(
          get_activation(self.backbone_model.activation)
      )
      self.rtd_encoder.add(
          nn.Dense(
              units=1,
              in_units=self.backbone_model.units,
              flatten=False,
              weight_initializer=weight_initializer,
              bias_initializer=bias_initializer,
          )
      )
  
imports: import os
  from typing import Tuple, Optional, List
  import mxnet
  from mxnet import use_np, np, npx
  from mxnet.gluon import HybridBlock, nn
  from .base import BACKBONE_REGISTRY
  from .transformer import TransformerEncoderLayer
  from ..op import gumbel_softmax, select_vectors_by_position, add_vectors_by_position, update_vectors_by_position
  from ..base import get_model_zoo_home_dir, get_repo_model_zoo_url, get_model_zoo_checksum_dir
  from ..layers import PositionalEmbedding, get_activation
  from ..initializer import TruncNorm
  from ..utils.config import CfgNode
  from ..utils.misc import load_checksum_stats, download
  from ..utils.registry import Registry
  from ..attention_cell import gen_self_attn_mask
  from ..data.tokenizers import HuggingFaceWordPieceTokenizer
label_desc: Use image node_id 1 to create a new ElectraDiscriminator object from inherited base classes: HybridBlock with example: obj = ElectraDiscriminator(backbone_cfg, weight_initializer, bias_initializer)