Hubert-kakeiken-W-5_impulses

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_5_IMPULSES - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0225
• Wer: 0.9994
• Cer: 1.0155

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.819	1.0	4097	2.5780	1.0	1.1284
0.3096	2.0	8194	0.0899	0.9995	1.0198
0.2159	3.0	12291	0.0483	0.9994	1.0215
0.1898	4.0	16388	0.0310	0.9994	1.0181
0.1655	5.0	20485	0.0341	0.9995	1.0194
0.1591	6.0	24582	0.0302	0.9995	1.0180
0.1465	7.0	28679	0.0371	0.9994	1.0186
0.1411	8.0	32776	0.0246	0.9994	1.0176
0.1289	9.0	36873	0.0305	0.9995	1.0184
0.1309	10.0	40970	0.0195	0.9994	1.0161
0.1282	11.0	45067	0.0242	0.9994	1.0183
0.1195	12.0	49164	0.0293	0.9994	1.0176
0.1183	13.0	53261	0.0171	0.9995	1.0162
0.1121	14.0	57358	0.0201	0.9994	1.0159
0.108	15.0	61455	0.0176	0.9994	1.0162
0.1094	16.0	65552	0.0158	0.9994	1.0163
0.1055	17.0	69649	0.0199	0.9995	1.0163
0.0998	18.0	73746	0.0228	0.9994	1.0159
0.0991	19.0	77843	0.0175	0.9994	1.0163
0.0974	20.0	81940	0.0231	0.9994	1.0164
0.0904	21.0	86037	0.0141	0.9994	1.0155
0.0883	22.0	90134	0.0177	0.9994	1.0156
0.0888	23.0	94231	0.0211	0.9994	1.0156
0.0839	24.0	98328	0.0205	0.9994	1.0161
0.0833	25.0	102425	0.0193	0.9994	1.0155
0.083	26.0	106522	0.0202	0.9994	1.0156
0.08	27.0	110619	0.0195	0.9994	1.0156
0.0754	28.0	114716	0.0228	0.9994	1.0157
0.0761	29.0	118813	0.0262	0.9994	1.0162
0.0729	30.0	122910	0.0245	0.9994	1.0158
0.0718	31.0	127007	0.0266	0.9994	1.0162
0.0715	32.0	131104	0.0215	0.9994	1.0154
0.0707	33.0	135201	0.0231	0.9994	1.0155
0.0682	34.0	139298	0.0228	0.9994	1.0156
0.0673	35.0	143395	0.0228	0.9994	1.0155
0.0665	36.0	147492	0.0238	0.9994	1.0155
0.0656	37.0	151589	0.0219	0.9995	1.0155
0.0678	38.0	155686	0.0224	0.9994	1.0154
0.0656	39.0	159783	0.0227	0.9994	1.0155
0.0654	40.0	163880	0.0224	0.9994	1.0154

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0