Hubert-kakeiken-W-jp_style_room

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_JP_STYLE_ROOM - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0240
• Wer: 0.9988
• Cer: 1.0162

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
26.5829	1.0	820	10.3742	1.0	1.1285
8.6907	2.0	1640	7.2536	1.0	1.1284
6.6878	3.0	2460	4.1513	1.0	1.1284
3.6421	4.0	3280	3.1288	1.0	1.1285
2.7853	5.0	4100	2.5107	1.0	1.1284
2.2972	6.0	4920	1.2105	1.0	1.0308
0.9952	7.0	5740	0.5433	0.9997	1.0731
0.4783	8.0	6560	0.2819	0.9990	1.0332
0.3964	9.0	7380	0.1789	0.9994	1.0327
0.3013	10.0	8200	0.1114	0.9988	1.0219
0.2448	11.0	9020	0.1206	0.9990	1.0295
0.2275	12.0	9840	0.0860	0.9990	1.0228
0.2231	13.0	10660	0.0771	0.9988	1.0226
0.2159	14.0	11480	0.0864	0.9990	1.0278
0.2008	15.0	12300	0.0486	0.9988	1.0206
0.1909	16.0	13120	0.0485	0.9990	1.0217
0.1779	17.0	13940	0.0446	0.9988	1.0192
0.1757	18.0	14760	0.0463	0.9988	1.0240
0.1674	19.0	15580	0.0446	0.9988	1.0215
0.1565	20.0	16400	0.0346	0.9988	1.0182
0.1554	21.0	17220	0.0442	0.9988	1.0199
0.1484	22.0	18040	0.0472	0.9988	1.0220
0.1429	23.0	18860	0.0372	0.9988	1.0187
0.137	24.0	19680	0.0404	0.9988	1.0184
0.1353	25.0	20500	0.0498	0.9988	1.0189
0.1245	26.0	21320	0.0344	0.9990	1.0190
0.1199	27.0	22140	0.0362	0.9988	1.0190
0.1138	28.0	22960	0.0386	0.9988	1.0172
0.1098	29.0	23780	0.0284	0.9988	1.0173
0.1065	30.0	24600	0.0284	0.9988	1.0178
0.1004	31.0	25420	0.0268	0.9988	1.0169
0.095	32.0	26240	0.0291	0.9988	1.0174
0.0893	33.0	27060	0.0275	0.9988	1.0166
0.0882	34.0	27880	0.0259	0.9988	1.0166
0.0852	35.0	28700	0.0246	0.9988	1.0166
0.0814	36.0	29520	0.0250	0.9988	1.0165
0.078	37.0	30340	0.0254	0.9988	1.0163
0.0794	38.0	31160	0.0242	0.9988	1.0162
0.0805	39.0	31980	0.0248	0.9988	1.0163
0.0774	39.9518	32760	0.0249	0.9988	1.0164

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0