58666903-8b23-4431-98d1-929f087da646

This model is a fine-tuned version of fxmarty/tiny-dummy-qwen2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.9124

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.000203
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 50
• training_steps: 400

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0026	1	11.9306
11.922	0.1306	50	11.9198
11.9205	0.2613	100	11.9179
11.9173	0.3919	150	11.9155
11.9157	0.5225	200	11.9141
11.9147	0.6532	250	11.9134
11.9141	0.7838	300	11.9131
11.9139	0.9144	350	11.9127
12.0354	1.0457	400	11.9124

Framework versions

• PEFT 0.13.2
• Transformers 4.46.0
• Pytorch 2.5.0+cu124
• Datasets 3.0.1
• Tokenizers 0.20.1