Hugging Face's logo

asr-africa
/
W2V2-Bert_DigitalUmuganda_Afrivoice_Shona_1hr_v2

Automatic Speech Recognition
Transformers
Safetensors
wav2vec2-bert
Generated from Trainer
Inference Endpoints
Model card Files Community
W2V2-Bert_DigitalUmuganda_Afrivoice_Shona_1hr_v2 / README.md
Beijuka's picture
Beijuka
End of training
910a92c verified
preview code
|
raw
history blame contribute delete
3.74 kB
metadata
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: W2V2-Bert_DigitalUmuganda_Afrivoice_Shona_1hr_v2
    results: []

W2V2-Bert_DigitalUmuganda_Afrivoice_Shona_1hr_v2

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4556
  • Model Preparation Time: 0.0127
  • Wer: 0.3692
  • Cer: 0.0718

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Model Preparation Time Wer Cer
7.2051 0.9778 22 6.1914 0.0127 1.0 0.9176
4.1072 2.0 45 3.1571 0.0127 1.0 0.9905
3.0778 2.9778 67 2.8928 0.0127 1.0 0.9418
2.6835 4.0 90 2.5263 0.0127 1.0 0.7968
2.0498 4.9778 112 1.1299 0.0127 0.9935 0.3234
0.6781 6.0 135 0.4324 0.0127 0.5731 0.0952
0.3809 6.9778 157 0.3307 0.0127 0.4004 0.0667
0.2728 8.0 180 0.2900 0.0127 0.3482 0.0571
0.2301 8.9778 202 0.2700 0.0127 0.3382 0.0549
0.1725 10.0 225 0.2791 0.0127 0.3510 0.0541
0.1417 10.9778 247 0.2745 0.0127 0.3643 0.0551
0.1158 12.0 270 0.3095 0.0127 0.3256 0.0534
0.1125 12.9778 292 0.2983 0.0127 0.3407 0.0555
0.0854 14.0 315 0.3183 0.0127 0.3505 0.0540
0.078 14.9778 337 0.3576 0.0127 0.3409 0.0549
0.0661 16.0 360 0.3444 0.0127 0.3550 0.0567
0.0736 16.9778 382 0.3750 0.0127 0.3570 0.0595
0.0584 18.0 405 0.3570 0.0127 0.3738 0.0583
0.0503 18.9778 427 0.4068 0.0127 0.3740 0.0612
0.0498 20.0 450 0.3769 0.0127 0.3545 0.0581
0.0549 20.9778 472 0.3819 0.0127 0.3620 0.0574
0.049 22.0 495 0.3945 0.0127 0.3632 0.0582

Framework versions

  • Transformers 4.44.1
  • Pytorch 2.2.0+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1