631f7acb-c01f-4431-ae69-6d7482b82fae

This model is a fine-tuned version of echarlaix/tiny-random-mistral on the None dataset. It achieves the following results on the evaluation set:

• Loss: 10.3067

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.000218
• train_batch_size: 4
• eval_batch_size: 4
• seed: 180
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 50
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0005	1	10.3792
20.6306	0.0268	50	10.3272
20.6148	0.0537	100	10.3178
20.5953	0.0805	150	10.3140
20.6146	0.1074	200	10.3135
20.5962	0.1342	250	10.3117
20.5843	0.1610	300	10.3096
20.5765	0.1879	350	10.3079
20.5839	0.2147	400	10.3069
20.5816	0.2415	450	10.3069
20.5832	0.2684	500	10.3067

Framework versions

• PEFT 0.13.2
• Transformers 4.46.0
• Pytorch 2.5.0+cu124
• Datasets 3.0.1
• Tokenizers 0.20.1