Update README.md

README.md

@@ -5,8 +5,41 @@ language:
 metrics:
 - wer
 pipeline_tag: automatic-speech-recognition
+datasets:
+- MuST-C-en_ar
+library_name: fairseq
+tags:
+- audio 
+- automatic-speech-recognition
+- speech
+- speech2text
+- ASR
+- asr
+- ASR-punctuation-sensitive
+- encoder-decoder-for-asr
 ---
 
+# speecht5-asr-punctuation-sensitive
+This model is part of SotoMedia's Automatic Video Dubbing project, aiming to build first open source video dubbing technolgy across a 
+diverse range of languages. You can find more details about our project and our pibline [here](https://github.com/ElsebaiyMohamed/Modablag).
+
+## Description:
+The **speecht5-asr-punctuation-sensitive** model is an advanced Automatic Speech Recognition (ASR) system designed to transcribe spoken English 
+while maintaining a high level of awareness for punctuation. This model is trained to accurately recognize and preserve punctuation marks,
+enhancing the fidelity of transcriptions in scenarios where punctuation is crucial for conveying meaning. 
+
+- **Model type:** transformer encoder- decoder
+- **Language:** En
+- **Base model:** SpeechT5-ASR [checkpoint](https://huggingface.co/microsoft/speecht5_asr)
+- **Finetuning dataset:** [MuST-C-en_ar](https://www.kaggle.com/datasets/sebaeymohamed/must-c-en-ar) 
+
+## Key Features:
+Punctuation Sensitivity: The model is specifically engineered to be highly sensitive to punctuation nuances in spoken English, ensuring 
+accurate representation of the speaker's intended meaning.
+
+
+## Usage
+
 ```py
 # Use a pipeline as a high-level helper
 from transformers import pipeline
@@ -14,10 +47,99 @@ from transformers import pipeline
 pipe = pipeline("automatic-speech-recognition", model="seba3y/speecht5-asr-punctuation-sensitive")
 ```
 
+
 ```py
 # Load model directly
 from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
 
 processor = AutoProcessor.from_pretrained("seba3y/speecht5-asr-punctuation-sensitive")
 model = AutoModelForSpeechSeq2Seq.from_pretrained("seba3y/speecht5-asr-punctuation-sensitive")
+```
+
+## Fintuning & Evaluation Details
+
+### Dataset
+MuST-C is a multilingual speech translation corpus whose size and quality will facilitate the training of end-to-end systems 
+for SLT from English into several target languages. For each target language, MuST-C comprises several hundred hours of audio 
+recordings from English TED Talks, which are automatically aligned at the sentence level with their manual transcriptions and translations.
+
+**Datasplits:**
+- set: dev
+  - talks: 11
+  - sentences: 1073
+  - words src: 24274
+  - words tgt: 21387
+  - time: 8914.57s == 2h28m34s
+
+- set: tst-COMMON
+  - talks: 27
+  - sentences: 2019
+  - words src: 41955
+  - words tgt: 36443
+  - time: 14679.72s == 4h04m39s
+
+- set: tst-HE
+  - talks: 12
+  - sentences: 578
+  - words src: 13080
+  - words tgt: 10912
+  - time: 5211.84s == 1h26m51s
+
+- set: train
+  - talks: 2412
+  - sentences: 212085
+  - words src: 4520522
+  - words tgt: 4000457
+  - time: 1667744.17s == 463h15m44s
+
+
+#### Hyperparameters
+
+|Paramter|Value|
+|-|-|
+|per_device_train_batch_size|6|
+|per_device_eval_batch_size|10|
+|gradient_accumulation_steps|20|
+|eval_accumulation_steps|16|
+|dataloader_num_workers|2|
+|learning_rate|7e-5|
+|adafactor|True|
+|weight_decay|0.1|
+|max_grad_norm|0.9|
+|num_train_epochs|2.15|
+|warmup_steps|2000|
+|lr_scheduler_type|constant_with_warmup|
+|fp16|True|
+|gradient_checkpointing|True|
+|sortish_sampler|True|
+
+##### Results
+**Train loss:** 0.4429
+|Split|Word Error Rate (%)|
+|-|-|
+|dev|51.6|
+|tst-HE|40.2|
+|tst-COMMON|43.01|
+
+
+## Citation
+
+- MuST-C dataset
+```
+@InProceedings{mustc19, author = "Di Gangi, Mattia Antonino and Cattoni, Roldano and Bentivogli, Luisa and Negri, Matteo > and Turchi, Marco",
+ title = "{MuST-C: a Multilingual Speech Translation Corpus}",
+booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,
+ Volume 2 (Short Papers)", year = "2019", address = "Minneapolis, MN, USA", month = "June"}}
+```
+- SpeechT5-ASR
+```
+@inproceedings{ao-etal-2022-speecht5,
+    title = {{S}peech{T}5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing},
+    author = {Ao, Junyi and Wang, Rui and Zhou, Long and Wang, Chengyi and Ren, Shuo and Wu, Yu and Liu, Shujie and Ko, Tom and Li, Qing and Zhang, Yu and Wei, Zhihua and Qian, Yao and Li, Jinyu and Wei, Furu},
+    booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
+    month = {May},
+    year = {2022},
+    pages={5723--5738},
+}
+
 ```