|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
base_model: openai/whisper-tiny |
|
tags: |
|
- whisper-event |
|
- generated_from_trainer |
|
datasets: |
|
- asierhv/composite_corpus_eu_v2.1 |
|
metrics: |
|
- wer |
|
model-index: |
|
- name: Whisper Tiny Basque |
|
results: |
|
- task: |
|
name: Automatic Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: Mozilla Common Voice 18.0 |
|
type: mozilla-foundation/common_voice_18_0 |
|
metrics: |
|
- name: Wer |
|
type: wer |
|
value: 13.56 |
|
language: |
|
- eu |
|
--- |
|
|
|
# Whisper Tiny Basque |
|
|
|
This model is a fine-tuned version of [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) specifically for Basque (eu) language Automatic Speech Recognition (ASR). It was trained on the [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1) dataset, which is a composite corpus designed to improve Basque ASR performance. |
|
|
|
**Key improvements and results compared to the base model:** |
|
|
|
* **Significant WER reduction:** The fine-tuned model achieves a Word Error Rate (WER) of 14.8495 on the validation set of the `asierhv/composite_corpus_eu_v2.1` dataset, demonstrating improved accuracy compared to the base `whisper-tiny` model for Basque. |
|
* **Performance on Common Voice:** When evaluated on the Mozilla Common Voice 18.0 dataset, the model achieved a WER of 13.56. This demonstrates the model's ability to generalize to other Basque speech datasets. |
|
|
|
## Model description |
|
|
|
This model leverages the power of the Whisper architecture, originally developed by OpenAI, and adapts it to the specific nuances of the Basque language. By fine-tuning the `whisper-tiny` model on a comprehensive Basque speech corpus, it learns to accurately transcribe spoken Basque. The `whisper-tiny` model is the smallest of the whisper models, providing a good balance between speed and accuracy. |
|
|
|
## Intended uses & limitations |
|
|
|
**Intended uses:** |
|
|
|
* Automatic transcription of Basque speech. |
|
* Development of Basque speech-based applications. |
|
* Research on Basque speech processing. |
|
* Accessibility tools for Basque speakers. |
|
|
|
**Limitations:** |
|
|
|
* Performance may vary depending on the quality of the audio input (e.g., background noise, recording quality). |
|
* The model might struggle with highly dialectal or informal speech. |
|
* While the model shows improved performance, it may still produce errors, especially with complex sentences or uncommon words. |
|
* The model is based on the small version of whisper, and thus, accuracy may be improved with larger models. |
|
|
|
## Training and evaluation data |
|
|
|
* **Training dataset:** [asierhv/composite_corpus_eu_v2.1](https://huggingface.co/datasets/asierhv/composite_corpus_eu_v2.1). This dataset is a composite corpus of Basque speech data, designed to improve the performance of Basque ASR systems. |
|
* **Evaluation Dataset:** The `test` portion of `asierhv/composite_corpus_eu_v2.1`. |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
|
|
* **learning_rate:** 3.75e-05 |
|
* **train_batch_size:** 32 |
|
* **eval_batch_size:** 16 |
|
* **seed:** 42 |
|
* **optimizer:** AdamW with betas=(0.9, 0.999) and epsilon=1e-08 |
|
* **lr_scheduler_type:** linear |
|
* **lr_scheduler_warmup_steps:** 1000 |
|
* **training_steps:** 10000 |
|
* **mixed_precision_training:** Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | WER | |
|
|---------------|-------|-------|-----------------|----------| |
|
| 0.586 | 0.1 | 1000 | 0.6249 | 34.1639 | |
|
| 0.3145 | 0.2 | 2000 | 0.5048 | 25.2591 | |
|
| 0.225 | 0.3 | 3000 | 0.4839 | 22.0557 | |
|
| 0.3003 | 0.4 | 4000 | 0.4540 | 20.3072 | |
|
| 0.132 | 0.5 | 5000 | 0.4574 | 19.0146 | |
|
| 0.1588 | 0.6 | 6000 | 0.4380 | 17.8219 | |
|
| 0.1841 | 0.7 | 7000 | 0.4395 | 16.6667 | |
|
| 0.143 | 0.8 | 8000 | 0.3719 | 15.4490 | |
|
| 0.0967 | 0.9 | 9000 | 0.3685 | 15.1368 | |
|
| 0.1059 | 1.0 | 10000 | 0.3719 | 14.8495 | |
|
|
|
### Framework versions |
|
|
|
* Transformers 4.49.0.dev0 |
|
* Pytorch 2.6.0+cu124 |
|
* Datasets 3.3.1.dev0 |
|
* Tokenizers 0.21.0 |
