bert-base-cased-finetuned-AddedTokens-HMGCR-IC50s-V1

This model is a fine-tuned version of bert-base-cased on 905 HMGCR IC50 values from bindingDB.org. Molecules with counter ions were included twice, once with and once without counter-ions.

It achieves the following results on the evaluation set:

• Loss: 0.7278
• Accuracy: 0.7929
• F1: 0.7931

Model description

More information needed

Intended uses & limitations

Can classify HMGCR IC50 values as < 50 nM, < 500 nM, and > 500 nM. See Confusion matrix below:

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.9314	1.0	25	0.8466	0.7071	0.6371
0.7535	2.0	50	0.7025	0.7357	0.6634
0.6292	3.0	75	0.6237	0.7714	0.6956
0.5464	4.0	100	0.6162	0.7571	0.7137
0.5068	5.0	125	0.5730	0.7857	0.7185
0.4516	6.0	150	0.5872	0.7643	0.7312
0.3971	7.0	175	0.6004	0.7643	0.7578
0.3768	8.0	200	0.6253	0.7714	0.7739
0.3353	9.0	225	0.6280	0.7786	0.7522
0.3439	10.0	250	0.6299	0.7714	0.7613
0.3087	11.0	275	0.6569	0.7786	0.7719
0.2979	12.0	300	0.6308	0.7714	0.7753
0.2561	13.0	325	0.6596	0.7786	0.7786
0.2703	14.0	350	0.6646	0.7786	0.7808
0.2504	15.0	375	0.7125	0.7857	0.7913
0.2397	16.0	400	0.6893	0.7786	0.7770
0.2152	17.0	425	0.7278	0.7929	0.7931
0.2066	18.0	450	0.6947	0.7857	0.7895
0.2133	19.0	475	0.7202	0.7714	0.7756
0.202	20.0	500	0.7167	0.7857	0.7887

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.0
• Tokenizers 0.21.0