932b6959-0eeb-4126-a7dd-9776f4f4ff51

This model is a fine-tuned version of fxmarty/tiny-dummy-qwen2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.8795

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.000201
• train_batch_size: 4
• eval_batch_size: 4
• seed: 10
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 50
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0008	1	11.9276
11.9072	0.0394	50	11.9099
11.8757	0.0788	100	11.8917
11.8724	0.1182	150	11.8869
11.8638	0.1577	200	11.8842
11.8626	0.1971	250	11.8817
11.8597	0.2365	300	11.8808
11.8598	0.2759	350	11.8801
11.8622	0.3153	400	11.8792
11.8628	0.3547	450	11.8796
11.8598	0.3942	500	11.8795

Framework versions

• PEFT 0.13.2
• Transformers 4.46.0
• Pytorch 2.5.0+cu124
• Datasets 3.0.1
• Tokenizers 0.20.1