alem-147/poison-distilled-2

models

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: -1.0568
• Accuracy: 0.4219

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.0517	1.0	130	-0.1478	0.4361
-0.2131	2.0	260	-0.7916	0.4737
-0.6796	3.0	390	-1.1818	0.3910
-1.0059	4.0	520	-1.5061	0.4737
-1.4067	5.0	650	-2.4894	0.3835
-1.6574	6.0	780	-1.4229	0.4211
-1.8941	7.0	910	-1.9573	0.3684
-2.2115	8.0	1040	-1.1340	0.2857
-2.4642	9.0	1170	-0.4890	0.3383
-2.5106	10.0	1300	-3.0730	0.4586
-2.8623	11.0	1430	-1.3475	0.4737
-2.8928	12.0	1560	-3.6841	0.4286
-3.2526	13.0	1690	-0.3445	0.3158
-3.3179	14.0	1820	-4.3698	0.4511
-3.5768	15.0	1950	-1.2273	0.3534
-3.4264	16.0	2080	-0.9398	0.3383
-3.7022	17.0	2210	-4.5067	0.3835
-3.7768	18.0	2340	-1.5536	0.3308
-3.7878	19.0	2470	-4.1282	0.4737
-4.0069	20.0	2600	-1.5285	0.4361

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3