w2v-bert-2.0-lg-CV-Fleurs-1hrs-v1

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1416
• Wer: 0.4566
• Cer: 0.0978

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
6.4287	1.0	33	4.3733	1.0	0.8746
3.5018	2.0	66	3.0412	1.0	0.9993
2.9634	3.0	99	2.8308	1.0	0.9648
2.3326	4.0	132	1.4299	0.9958	0.3733
0.949	5.0	165	0.7168	0.7194	0.1571
0.5686	6.0	198	0.5983	0.5879	0.1301
0.4608	7.0	231	0.5696	0.5459	0.1237
0.3589	8.0	264	0.5918	0.5246	0.1112
0.305	9.0	297	0.6142	0.5282	0.1176
0.2595	10.0	330	0.5571	0.5178	0.1139
0.2392	11.0	363	0.5865	0.4942	0.1079
0.1864	12.0	396	0.5887	0.4890	0.1057
0.1481	13.0	429	0.6205	0.5112	0.1118
0.1286	14.0	462	0.6221	0.4831	0.1072
0.112	15.0	495	0.7048	0.5106	0.1116
0.0939	16.0	528	0.7129	0.4956	0.1075
0.0749	17.0	561	0.7164	0.4872	0.1082
0.0681	18.0	594	0.7782	0.4828	0.1046
0.0582	19.0	627	0.8116	0.4929	0.1069
0.0483	20.0	660	0.8012	0.5072	0.1088
0.0376	21.0	693	0.8372	0.4958	0.1067
0.0354	22.0	726	0.8349	0.4797	0.1051
0.0258	23.0	759	0.9033	0.5013	0.1088
0.0233	24.0	792	0.8219	0.4690	0.1024
0.0181	25.0	825	0.9054	0.4694	0.1017
0.0199	26.0	858	0.8698	0.5092	0.1082
0.0159	27.0	891	0.9403	0.4846	0.1064
0.0142	28.0	924	0.9794	0.4625	0.0997
0.01	29.0	957	0.9501	0.4759	0.1024
0.0084	30.0	990	1.0099	0.4625	0.0993
0.0057	31.0	1023	1.0293	0.4622	0.1002
0.0069	32.0	1056	1.0173	0.4729	0.1022
0.0082	33.0	1089	1.0217	0.4683	0.1005
0.0052	34.0	1122	0.9799	0.4684	0.1032
0.0084	35.0	1155	1.0559	0.4753	0.1045
0.0045	36.0	1188	1.0602	0.4627	0.0998
0.0039	37.0	1221	1.0979	0.4755	0.1024
0.0028	38.0	1254	1.0779	0.4633	0.1007
0.0041	39.0	1287	1.0735	0.4685	0.1009
0.0059	40.0	1320	1.0740	0.4576	0.0990
0.0032	41.0	1353	1.0726	0.4528	0.0976
0.006	42.0	1386	1.1226	0.4667	0.1004
0.0047	43.0	1419	1.0967	0.4556	0.0981
0.0045	44.0	1452	1.0532	0.4616	0.1009
0.0042	45.0	1485	1.0789	0.4570	0.0990
0.0035	46.0	1518	1.1087	0.4530	0.0981
0.0023	47.0	1551	1.1016	0.4601	0.0994
0.002	48.0	1584	1.1111	0.4485	0.0979
0.0025	49.0	1617	1.0998	0.4596	0.1005
0.0064	50.0	1650	1.1135	0.4546	0.0987
0.0028	51.0	1683	1.1165	0.4627	0.1006
0.0062	52.0	1716	1.1230	0.4609	0.1002
0.0093	53.0	1749	1.0876	0.4803	0.1025
0.0045	54.0	1782	1.0904	0.4609	0.0991
0.0017	55.0	1815	1.1188	0.4528	0.0979
0.0018	56.0	1848	1.1259	0.4509	0.0978
0.0038	57.0	1881	1.1149	0.4626	0.1002
0.0025	58.0	1914	1.1349	0.4544	0.0989
0.001	59.0	1947	1.1473	0.4653	0.0994
0.0009	60.0	1980	1.1416	0.4566	0.0978

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0