mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-5hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.2369
• Cer: 0.0801

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
9.5784	1.0	183	inf	0.9998	0.9646
1.9211	2.0	366	inf	0.2887	0.0983
0.6977	3.0	549	inf	0.2641	0.0915
0.6391	4.0	732	inf	0.2568	0.0893
0.59	5.0	915	inf	0.2502	0.0874
0.5654	6.0	1098	inf	0.2489	0.0865
0.5544	7.0	1281	inf	0.2459	0.0853
0.5385	8.0	1464	inf	0.2443	0.0846
0.5256	9.0	1647	inf	0.2442	0.0841
0.5152	10.0	1830	inf	0.2420	0.0837
0.5116	11.0	2013	inf	0.2411	0.0833
0.497	12.0	2196	inf	0.2414	0.0829
0.4969	13.0	2379	inf	0.2400	0.0826
0.4887	14.0	2562	inf	0.2396	0.0824
0.4849	15.0	2745	inf	0.2394	0.0820
0.473	16.0	2928	inf	0.2396	0.0820
0.4732	17.0	3111	inf	0.2403	0.0819
0.4648	18.0	3294	inf	0.2396	0.0817
0.4608	19.0	3477	inf	0.2397	0.0817
0.4583	20.0	3660	inf	0.2397	0.0815
0.4579	21.0	3843	inf	0.2402	0.0817
0.447	22.0	4026	inf	0.2379	0.0813
0.4442	23.0	4209	inf	0.2373	0.0809
0.439	24.0	4392	inf	0.2373	0.0809
0.4381	25.0	4575	inf	0.2376	0.0809
0.4386	26.0	4758	inf	0.2377	0.0807
0.423	27.0	4941	inf	0.2360	0.0804
0.428	28.0	5124	inf	0.2366	0.0803
0.425	29.0	5307	inf	0.2368	0.0804
0.4169	30.0	5490	inf	0.2376	0.0803
0.4253	31.0	5673	inf	0.2367	0.0802
0.4173	32.0	5856	inf	0.2371	0.0802
0.414	33.0	6039	inf	0.2362	0.0800
0.414	34.0	6222	inf	0.2367	0.0803
0.4064	35.0	6405	inf	0.2373	0.0801
0.402	36.0	6588	inf	0.2370	0.0802
0.4035	37.0	6771	inf	0.2369	0.0801

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0