To probe the effect of the pretraining scheme on the predictions, we added only a single feedforward layer to the pretrained CLM and E-CLM for bioactivity prediction. This additional network layer consisted of three neurons, one for each of the three bioactivity classes. It was added to fine-tune the entire network for bioactivity prediction . To mitigate the class data imbalance, we applied oversampling to the classes with fewer data (i.e., the "inactive" and "highly active" classes) .