Hubert-kakeiken-W-enhanced_A_one_set

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_ENHANCED_A_ONE_SET - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0276
• Wer: 0.9988
• Cer: 1.0167

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
27.4586	1.0	820	10.8551	1.0	1.1283
9.1714	2.0	1640	7.5436	1.0	1.1284
6.9896	3.0	2460	4.2157	1.0	1.1284
3.6374	4.0	3280	2.9699	1.0	1.1284
2.5862	5.0	4100	2.2529	1.0	1.1284
2.0604	6.0	4920	0.8419	0.9997	1.0955
0.5707	7.0	5740	0.3563	0.9996	1.0568
0.2899	8.0	6560	0.2087	0.9990	1.0339
0.2295	9.0	7380	0.1561	0.9993	1.0337
0.1701	10.0	8200	0.1098	0.9988	1.0259
0.1403	11.0	9020	0.1463	0.9990	1.0319
0.1255	12.0	9840	0.0919	0.9988	1.0286
0.12	13.0	10660	0.0487	0.9990	1.0209
0.11	14.0	11480	0.0698	0.9988	1.0268
0.1101	15.0	12300	0.0412	0.9988	1.0195
0.1006	16.0	13120	0.0629	0.9993	1.0235
0.0999	17.0	13940	0.1519	0.9990	1.0329
0.0914	18.0	14760	0.0398	0.9988	1.0211
0.0883	19.0	15580	0.0489	0.9988	1.0236
0.0727	20.0	16400	0.0381	0.9988	1.0184
0.0848	21.0	17220	0.0436	0.9990	1.0201
0.0689	22.0	18040	0.0494	0.9988	1.0207
0.0672	23.0	18860	0.0290	0.9988	1.0183
0.0629	24.0	19680	0.0397	0.9988	1.0184
0.0608	25.0	20500	0.0395	0.9988	1.0187
0.0556	26.0	21320	0.0327	0.9990	1.0177
0.0519	27.0	22140	0.0291	0.9988	1.0172
0.0471	28.0	22960	0.0318	0.9988	1.0176
0.0429	29.0	23780	0.0371	0.9988	1.0178
0.0435	30.0	24600	0.0284	0.9988	1.0173
0.039	31.0	25420	0.0293	0.9988	1.0182
0.0331	32.0	26240	0.0287	0.9988	1.0171
0.0331	33.0	27060	0.0265	0.9988	1.0162
0.0318	34.0	27880	0.0259	0.9988	1.0165
0.0269	35.0	28700	0.0254	0.9988	1.0163
0.0281	36.0	29520	0.0254	0.9988	1.0162
0.0231	37.0	30340	0.0268	0.9988	1.0167
0.025	38.0	31160	0.0266	0.9988	1.0167
0.0262	39.0	31980	0.0266	0.9988	1.0164
0.0265	39.9518	32760	0.0274	0.9988	1.0166

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0