# Wav2Vec2-Base-Vietnamese-270h
Fine-tuned Wav2Vec2 model on Vietnamese Speech Recognition task using about 270h labelled data combined from multiple datasets including [Common Voice](https://huggingface.co/datasets/common_voice), [VIVOS](https://huggingface.co/datasets/vivos), [VLSP2020](https://vlsp.org.vn/vlsp2020/eval/asr). The model was fine-tuned using SpeechBrain toolkit with a custom tokenizer. For a better experience, we encourage you to learn more about [SpeechBrain](https://speechbrain.github.io/).  
When using this model, make sure that your speech input is sampled at 16kHz.  
Please refer to [huggingface blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) or [speechbrain](https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice/ASR/CTC) on how to fine-tune Wav2Vec2 model on a specific language.

### Benchmark WER result:
| | [VIVOS](https://huggingface.co/datasets/vivos) | [COMMON VOICE 7.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) | [COMMON VOICE 8.0](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0) |
|---|---|---|---|
|without LM| 8.23 | 12.15 | 12.15 |
|with 4-grams LM| 3.70 | 5.57 | 5.76 |

The language model was trained using [OSCAR](https://huggingface.co/datasets/oscar-corpus/OSCAR-2109) dataset on about 32GB of crawled text.

### Install SpeechBrain
To use this model, you should install speechbrain > 0.5.10

### Usage
The model can be used directly (without a language model) as follows:
```python
from speechbrain.pretrained import EncoderASR

model = EncoderASR.from_hparams(source="dragonSwing/wav2vec2-base-vn-270h", savedir="pretrained_models/asr-wav2vec2-vi")
model.transcribe_file('dragonSwing/wav2vec2-base-vn-270h/example.mp3')
# Output: được hồ chí minh coi là một động lực lớn của sự phát triển đất nước
```

### Inference on GPU
To perform inference on the GPU, add  `run_opts={"device":"cuda"}`  when calling the `from_hparams` method.

### Evaluation
The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.
```python
import torch
import torchaudio
from datasets import load_dataset, load_metric, Audio
from transformers import Wav2Vec2FeatureExtractor
from speechbrain.pretrained import EncoderASR
import re
test_dataset = load_dataset("mozilla-foundation/common_voice_8_0", "vi", split="test", use_auth_token=True)
test_dataset = test_dataset.cast_column("audio", Audio(sampling_rate=16_000))
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
wer = load_metric("wer")
extractor = Wav2Vec2FeatureExtractor.from_pretrained("dragonSwing/wav2vec2-base-vn-270h")
model = EncoderASR.from_hparams(source="dragonSwing/wav2vec2-base-vn-270h", savedir="pretrained_models/asr-wav2vec2-vi", run_opts={'device': device})
chars_to_ignore_regex = r'[,?.!\-;:"“%\'�]'
# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
  audio = batch["audio"]
  batch["target_text"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
  batch['speech'] = audio['array']
  return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)

def evaluate(batch):
  # For padding inputs only
  inputs = extractor(
    batch['speech'], 
    sampling_rate=16000, 
    return_tensors="pt", 
    padding=True, 
    do_normalize=False
  ).input_values
  input_lens = torch.ones(inputs.shape[0])
  pred_str, pred_tokens = model.transcribe_batch(inputs, input_lens)
  batch["pred_strings"] = pred_str
  
  return batch
result = test_dataset.map(evaluate, batched=True, batch_size=1)
print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["target_text"])))
```
**Test Result**: 12.155553%

#### Citation
```
@misc{SB2021,
    author = {Ravanelli, Mirco and Parcollet, Titouan and Rouhe, Aku and Plantinga, Peter and Rastorgueva, Elena and Lugosch, Loren and Dawalatabad, Nauman and Ju-Chieh, Chou and Heba, Abdel and Grondin, Francois and Aris, William and Liao, Chien-Feng and Cornell, Samuele and Yeh, Sung-Lin and Na, Hwidong and Gao, Yan and Fu, Szu-Wei and Subakan, Cem and De Mori, Renato and Bengio, Yoshua },
    title = {SpeechBrain},
    year = {2021},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\\\\url{https://github.com/speechbrain/speechbrain}},
  }
```

#### About SpeechBrain
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.  
Website: [https://speechbrain.github.io](https://speechbrain.github.io/)  
GitHub: [https://github.com/speechbrain/speechbrain](https://github.com/speechbrain/speechbrain)
{"language": "vi", "license": "cc-by-nc-4.0", "tags": ["audio", "speech", "speechbrain", "Transformer"], "datasets": ["vivos", "common_voice"], "metrics": ["wer"], "pipeline_tag": "automatic-speech-recognition", "widget": [{"example_title": "Example 1", "src": "https://huggingface.co/dragonSwing/wav2vec2-base-vn-270h/raw/main/example.mp3"}, {"example_title": "Example 2", "src": "https://huggingface.co/dragonSwing/wav2vec2-base-vn-270h/raw/main/example2.mp3"}], "model-index": [{"name": "Wav2vec2 Base Vietnamese 270h", "results": [{"task": {"type": "automatic-speech-recognition", "name": "Speech Recognition"}, "dataset": {"name": "Common Voice vi", "type": "common_voice", "args": "vi"}, "metrics": [{"type": "wer", "value": 9.66, "name": "Test WER"}]}, {"task": {"type": "automatic-speech-recognition", "name": "Speech Recognition"}, "dataset": {"name": "Common Voice 7.0", "type": "mozilla-foundation/common_voice_7_0", "args": "vi"}, "metrics": [{"type": "wer", "value": 5.57, "name": "Test WER"}]}, {"task": {"type": "automatic-speech-recognition", "name": "Speech Recognition"}, "dataset": {"name": "Common Voice 8.0", "type": "mozilla-foundation/common_voice_8_0", "args": "vi"}, "metrics": [{"type": "wer", "value": 5.76, "name": "Test WER"}]}, {"task": {"type": "automatic-speech-recognition", "name": "Speech Recognition"}, "dataset": {"name": "VIVOS", "type": "vivos", "args": "vi"}, "metrics": [{"type": "wer", "value": 3.7, "name": "Test WER"}]}]}]}
speechbrain wav2vec2 audio speech Transformer automatic-speech-recognition vi dataset:vivos dataset:common_voice license:cc-by-nc-4.0 model-index has_space region:us
TAGS
#speechbrain #wav2vec2 #audio #speech #Transformer #automatic-speech-recognition #vi #dataset-vivos #dataset-common_voice #license-cc-by-nc-4.0 #model-index #has_space #region-us 

Wav2Vec2-Base-Vietnamese-270h
=============================


Fine-tuned Wav2Vec2 model on Vietnamese Speech Recognition task using about 270h labelled data combined from multiple datasets including Common Voice, VIVOS, VLSP2020. The model was fine-tuned using SpeechBrain toolkit with a custom tokenizer. For a better experience, we encourage you to learn more about SpeechBrain.  

When using this model, make sure that your speech input is sampled at 16kHz.  

Please refer to huggingface blog or speechbrain on how to fine-tune Wav2Vec2 model on a specific language.


### Benchmark WER result:



The language model was trained using OSCAR dataset on about 32GB of crawled text.


### Install SpeechBrain


To use this model, you should install speechbrain > 0.5.10


### Usage


The model can be used directly (without a language model) as follows:


### Inference on GPU


To perform inference on the GPU, add 'run\_opts={"device":"cuda"}' when calling the 'from\_hparams' method.


### Evaluation


The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.


Test Result: 12.155553%


#### About SpeechBrain


SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.  

Website: URL  

GitHub: URL



### Benchmark WER result:



The language model was trained using OSCAR dataset on about 32GB of crawled text. ### Install SpeechBrain


To use this model, you should install speechbrain > 0.5.10 ### Usage


The model can be used directly (without a language model) as follows: ### Inference on GPU


To perform inference on the GPU, add 'run\_opts={"device":"cuda"}' when calling the 'from\_hparams' method. ### Evaluation


The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.


Test Result: 12.155553% #### About SpeechBrain


SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.  

Website: URL  

GitHub: URL
TAGS
#speechbrain #wav2vec2 #audio #speech #Transformer #automatic-speech-recognition #vi #dataset-vivos #dataset-common_voice #license-cc-by-nc-4.0 #model-index #has_space #region-us 
 ### Benchmark WER result:



The language model was trained using OSCAR dataset on about 32GB of crawled text. ### Install SpeechBrain


To use this model, you should install speechbrain > 0.5.10 ### Usage


The model can be used directly (without a language model) as follows: ### Inference on GPU


To perform inference on the GPU, add 'run\_opts={"device":"cuda"}' when calling the 'from\_hparams' method. ### Evaluation


The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.


Test Result: 12.155553% #### About SpeechBrain


SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.  

Website: URL  

GitHub: URL
passage: TAGS
#speechbrain #wav2vec2 #audio #speech #Transformer #automatic-speech-recognition #vi #dataset-vivos #dataset-common_voice #license-cc-by-nc-4.0 #model-index #has_space #region-us 
### Benchmark WER result:



The language model was trained using OSCAR dataset on about 32GB of crawled text.### Install SpeechBrain


To use this model, you should install speechbrain > 0.5.10### Usage


The model can be used directly (without a language model) as follows:### Inference on GPU


To perform inference on the GPU, add 'run\_opts={"device":"cuda"}' when calling the 'from\_hparams' method.### Evaluation


The model can be evaluated as follows on the Vietnamese test data of Common Voice 8.0.


Test Result: 12.155553%#### About SpeechBrain


SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to be simple, extremely flexible, and user-friendly. Competitive or state-of-the-art performance is obtained in various domains.  

Website: URL  

GitHub: URL