arielcerdap/whisper-large-v3-stutter-optimized

Whisper Large V3 Optimized for Stuttered Speech

This model is a fine-tuned version of openai/whisper-large-v3 on the TimeStamped dataset. It achieves the following results on the evaluation set:

• Loss: 1.8521
• Wer: 10.3918
• Wer Ortho: 5.5937
• Cer: 5.5914

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 8000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Wer Ortho	Cer
1.4877	5.8187	500	1.6643	12.9475	7.3346	7.4041
1.4373	11.6316	1000	1.6887	14.1410	9.0894	9.1010
1.4112	17.4444	1500	1.7115	10.0203	5.5033	5.5149
1.4089	23.2573	2000	1.7320	9.7838	5.4036	5.4129
1.4085	29.0702	2500	1.7222	10.0090	5.3503	5.3572
1.4176	34.8889	3000	1.7498	11.8442	7.0101	7.0101
1.4162	40.7018	3500	1.7794	11.7654	6.9683	6.9683
1.4045	46.5146	4000	1.7699	12.1031	7.1074	7.1051
1.401	52.3275	4500	1.7946	12.0919	7.1468	7.1445
1.4007	58.1404	5000	1.8018	9.9527	5.3966	5.3943
1.4145	63.9591	5500	1.8229	10.2342	5.5079	5.5056
1.4142	69.7719	6000	1.8371	10.3468	5.5566	5.5543
1.414	75.5848	6500	1.8430	10.3580	5.5682	5.5659
1.3998	81.3977	7000	1.8494	10.3805	5.5867	5.5844
1.3997	87.2105	7500	1.8516	10.3918	5.5960	5.5937
1.3997	93.0234	8000	1.8521	10.3918	5.5937	5.5914

Framework versions

• Transformers 4.48.3
• Pytorch 2.6.0+cu124
• Datasets 3.4.1
• Tokenizers 0.21.1