Abdulmohsena commited on
Commit
7d3fac4
·
verified ·
1 Parent(s): 9df4ead

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -52,7 +52,12 @@ tokenizer.decode(generated_tokens[0], skip_special_tokens=True)
52
 
53
  - The language pairs are outside of quran is mostly translated by Google Translate. Thus, the quality of translation is dependant on the quality of Google's Translation from Classical Arabic to English.
54
  - The Metrics used in this model is bertscore/e5score. It is not even close to perfect in terms of alignment, but it is the best available metric for semantic translation. Thus, until a better subsitute appears, this is the main evaluation metric.
 
55
 
 
 
 
 
56
 
57
  ### Training Data
58
 
@@ -60,3 +65,4 @@ tokenizer.decode(generated_tokens[0], skip_special_tokens=True)
60
 
61
  ### Metrics
62
  - **COMET**: to pay more attention in representing the same meaning rather than focusing on individual words (Semantic Translation, not Syntactic Translation)
 
 
52
 
53
  - The language pairs are outside of quran is mostly translated by Google Translate. Thus, the quality of translation is dependant on the quality of Google's Translation from Classical Arabic to English.
54
  - The Metrics used in this model is bertscore/e5score. It is not even close to perfect in terms of alignment, but it is the best available metric for semantic translation. Thus, until a better subsitute appears, this is the main evaluation metric.
55
+ - Metrics used in general to evaluate the translation quality to Arabic are trained on Modern Standard Arabic, thus making them unaligned to the goals of the model.
56
 
57
+ ### Improvements
58
+
59
+ - A much better approach to generate a language pair out of classical Arabic text is to use GPT4o (at the time of this writing, that is the only model capable of understanding complex Arabic sentences).
60
+ - There should be evaluation metrics designed for the goal of this model. Currently, I have only created a binary classifier to classify a sentence if it is classical or not. It works as a score from 0 to 1, but it is not sufficient nor flexible, thus more work need to be done in evaluations.
61
 
62
  ### Training Data
63
 
 
65
 
66
  ### Metrics
67
  - **COMET**: to pay more attention in representing the same meaning rather than focusing on individual words (Semantic Translation, not Syntactic Translation)
68
+ - **[Fluency Score](https://huggingface.co/Abdulmohsena/Fluency_Score)**: A custom built metric to classify a metric if it is classical or not.