class_docstr: It is slightly different from the traditional mask language model which recover the
  masked word (find the matched word in dictionary). The Object of Discriminator in
  Electra is 'replaced token detection' that is a binary classification task to
  predicts every token whether it is an original or a replacement.
class_docstr_tok: ["It","is","slightly","different","from","the","traditional","mask","language","model","which","recover","the","masked","word","(","find","the","matched","word","in","dictionary",")",".","The","Object","of","Discriminator","in","Electra","is","'replaced","token","detection","'","that","is","a","binary","classification","task","to","predicts","every","token","whether","it","is","an","original","or","a","replacement","."]
docstr: Getting the scores of the replaced token detection of the whole sentence
  based on the corrupted tokens produced from a generator.
  
  Parameters
  ----------
  inputs
      - layout = 'NT'
          Shape (batch_size, seq_length)
      - layout = 'TN'
          Shape (seq_length, batch_size)
  
  token_types
      If the inputs contain two sequences, we will set different token types for the first
       sentence and the second sentence.
  
      - layout = 'NT'
          Shape (batch_size, seq_length)
      - layout = 'TN'
          Shape (seq_length, batch_size)
  
  valid_length
      The valid length of each sequence
      Shape (batch_size,)
  
  Returns
  -------
  contextual_embedding
      - layout = 'NT'
          Shape (batch_size, seq_length, units).
      - layout = 'TN'
          Shape (seq_length, batch_size, units).
  pooled_out
      Shape (batch_size, units)
  rtd_scores
      - layout = 'NT'
          Shape (batch_size, seq_length)
      - layout = 'TN'
          Shape (seq_length, batch_size)
docstr_tok: ["Getting","the","scores","of","the","replaced","token","detection","of","the","whole","sentence","based","on","the","corrupted","tokens","produced","from","a","generator",".","Parameters","--","--","--","--","--","inputs","-","layout","=","'NT'","Shape","(","batch_size",",","seq_length",")","-","layout","=","'TN'","Shape","(","seq_length",",","batch_size",")","token_types","If","the","inputs","contain","two","sequences",",","we","will","set","different","token","types","for","the","first","sentence","and","the","second","sentence",".","-","layout","=","'NT'","Shape","(","batch_size",",","seq_length",")","-","layout","=","'TN'","Shape","(","seq_length",",","batch_size",")","valid_length","The","valid","length","of","each","sequence","Shape","(","batch_size",",",")","Returns","--","--","--","-","contextual_embedding","-","layout","=","'NT'","Shape","(","batch_size",",","seq_length",",","units",")",".","-","layout","=","'TN'","Shape","(","seq_length",",","batch_size",",","units",")",".","pooled_out","Shape","(","batch_size",",","units",")","rtd_scores","-","layout","=","'NT'","Shape","(","batch_size",",","seq_length",")","-","layout","=","'TN'","Shape","(","seq_length",",","batch_size",")"]
code: def forward(self, inputs, token_types, valid_length):
      """Getting the scores of the replaced token detection of the whole sentence
      based on the corrupted tokens produced from a generator.
  
      Parameters
      ----------
      inputs
          - layout = 'NT'
              Shape (batch_size, seq_length)
          - layout = 'TN'
              Shape (seq_length, batch_size)
  
      token_types
          If the inputs contain two sequences, we will set different token types for the first
           sentence and the second sentence.
  
          - layout = 'NT'
              Shape (batch_size, seq_length)
          - layout = 'TN'
              Shape (seq_length, batch_size)
  
      valid_length
          The valid length of each sequence
          Shape (batch_size,)
  
      Returns
      -------
      contextual_embedding
          - layout = 'NT'
              Shape (batch_size, seq_length, units).
          - layout = 'TN'
              Shape (seq_length, batch_size, units).
      pooled_out
          Shape (batch_size, units)
      rtd_scores
          - layout = 'NT'
              Shape (batch_size, seq_length)
          - layout = 'TN'
              Shape (seq_length, batch_size)
      """
      contextual_embeddings, pooled_out = self.backbone_model(
          inputs, token_types, valid_length
      )
      rtd_scores = self.rtd_encoder(contextual_embeddings).squeeze(-1)
      return contextual_embeddings, pooled_out, rtd_scores
  
imports: import os
  from typing import Tuple, Optional, List
  import mxnet
  from mxnet import use_np, np, npx
  from mxnet.gluon import HybridBlock, nn
  from .base import BACKBONE_REGISTRY
  from .transformer import TransformerEncoderLayer
  from ..op import gumbel_softmax, select_vectors_by_position, add_vectors_by_position, update_vectors_by_position
  from ..base import get_model_zoo_home_dir, get_repo_model_zoo_url, get_model_zoo_checksum_dir
  from ..layers import PositionalEmbedding, get_activation
  from ..initializer import TruncNorm
  from ..utils.config import CfgNode
  from ..utils.misc import load_checksum_stats, download
  from ..utils.registry import Registry
  from ..attention_cell import gen_self_attn_mask
  from ..data.tokenizers import HuggingFaceWordPieceTokenizer
label_desc: Use image node_id 2 for calling the ElectraDiscriminator obj's underlying member method code with example usage: obj.forward(inputs, token_types, valid_length) and returns: contextual_embeddings,  pooled_out,  rtd_scores