Spaces:
Runtime error
Runtime error
update custom message
Browse files- constants.py +3 -1
constants.py
CHANGED
@@ -105,10 +105,12 @@ This opens way the to achieve standardized test set for most languages, enabling
|
|
105 |
|
106 |
Columns `Model`, `RTF`, and `Average WER` were sourced from [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) using the version from September 7, 2023.
|
107 |
|
108 |
-
Models are sorted by consistancy in
|
109 |
|
110 |
### Results
|
111 |
The CommonVoice Test provides a Word Error Rate (WER) within a 20-point margin of the average WER.
|
112 |
|
113 |
While not perfect, this indicates that CommonVoice can be a useful tool for quickly identifying a suitable ASR model for a wide range of languages in a programmatic manner. However, it's important to note that it is not sufficient as the sole criterion for choosing the most appropriate architecture. Further considerations may be needed depending on the specific requirements of your ASR application.
|
|
|
|
|
114 |
"""
|
|
|
105 |
|
106 |
Columns `Model`, `RTF`, and `Average WER` were sourced from [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) using the version from September 7, 2023.
|
107 |
|
108 |
+
Models are sorted by consistancy in theire results across testsets. (by increasing order of absolute delta between average WER and CommonVoice WER)
|
109 |
|
110 |
### Results
|
111 |
The CommonVoice Test provides a Word Error Rate (WER) within a 20-point margin of the average WER.
|
112 |
|
113 |
While not perfect, this indicates that CommonVoice can be a useful tool for quickly identifying a suitable ASR model for a wide range of languages in a programmatic manner. However, it's important to note that it is not sufficient as the sole criterion for choosing the most appropriate architecture. Further considerations may be needed depending on the specific requirements of your ASR application.
|
114 |
+
|
115 |
+
For more context, [here](https://gist.github.com/wasertech/400ca3dd61f2d6f7f4f5495afbb32ef3) is the output of my ASR server when running without any specified model to load for various languages. It tries to score the most suitable model for any given language. Since metrics are mostly self-reported, sometimes in different format, it consistently picks an unadequate model.
|
116 |
"""
|