ELECTRA pretraining. The E-CLM model was implemented in Python (v. 3.6.5) using Keras (v. 2.2.0, ) with the TensorFlow GPU backend (v. 1.9.0, ). The ELECTRA model was implemented with the same architecture as that of the generative CLM, i.e., as a recurrent neural network with LSTM cells. Model training was performed with the Adam optimizer (learning rate = 10 -3 , 50 epochs) and the binary cross-entropy loss function.