diff --git "a/README.md" "b/README.md" --- "a/README.md" +++ "b/README.md" @@ -6,7 +6,7 @@ tags: - generated_from_trainer - dataset_size:753920 - loss:MultipleNegativesRankingLoss -base_model: aubmindlab/bert-base-arabertv02 +base_model: egyllm/pretrained-arabert widget: - source_sentence: ': استجابة الحادث بعد حادث كشف عن أوجه القصور في الشركة' sentences: @@ -78,7 +78,7 @@ metrics: - pearson_max - spearman_max model-index: -- name: SentenceTransformer based on aubmindlab/bert-base-arabertv02 +- name: SentenceTransformer based on egyllm/pretrained-arabert results: - task: type: information-retrieval @@ -88,49 +88,49 @@ model-index: type: unknown metrics: - type: cosine_accuracy@1 - value: 0.7065 + value: 0.7175 name: Cosine Accuracy@1 - type: cosine_accuracy@3 - value: 0.84 + value: 0.841 name: Cosine Accuracy@3 - type: cosine_accuracy@5 - value: 0.877 + value: 0.878 name: Cosine Accuracy@5 - type: cosine_accuracy@10 - value: 0.9125 + value: 0.9155 name: Cosine Accuracy@10 - type: cosine_precision@1 - value: 0.7065 + value: 0.7175 name: Cosine Precision@1 - type: cosine_precision@3 - value: 0.28 + value: 0.28033333333333327 name: Cosine Precision@3 - type: cosine_precision@5 - value: 0.17540000000000003 + value: 0.17560000000000003 name: Cosine Precision@5 - type: cosine_precision@10 - value: 0.09125 + value: 0.09155 name: Cosine Precision@10 - type: cosine_recall@1 - value: 0.7065 + value: 0.7175 name: Cosine Recall@1 - type: cosine_recall@3 - value: 0.84 + value: 0.841 name: Cosine Recall@3 - type: cosine_recall@5 - value: 0.877 + value: 0.878 name: Cosine Recall@5 - type: cosine_recall@10 - value: 0.9125 + value: 0.9155 name: Cosine Recall@10 - type: cosine_ndcg@10 - value: 0.8126831693564126 + value: 0.8172358824512647 name: Cosine Ndcg@10 - type: cosine_mrr@10 - value: 0.7803275793650786 + value: 0.7856547619047611 name: Cosine Mrr@10 - type: cosine_map@100 - value: 0.7837897332371666 + value: 0.7890154491139222 name: Cosine Map@100 - task: type: semantic-similarity @@ -140,47 +140,47 @@ model-index: type: sts-dev metrics: - type: pearson_cosine - value: 0.8041663291384161 + value: 0.8015277726105404 name: Pearson Cosine - type: spearman_cosine - value: 0.8057219672338036 + value: 0.8038248041571585 name: Spearman Cosine - type: pearson_manhattan - value: 0.7938961056566315 + value: 0.7895258398435966 name: Pearson Manhattan - type: spearman_manhattan - value: 0.804480775014223 + value: 0.8012166855619245 name: Spearman Manhattan - type: pearson_euclidean - value: 0.7922157416357797 + value: 0.7893816883662468 name: Pearson Euclidean - type: spearman_euclidean - value: 0.8028596887695664 + value: 0.8029392819509334 name: Spearman Euclidean - type: pearson_dot - value: 0.7985215152878228 + value: 0.7952010752539163 name: Pearson Dot - type: spearman_dot - value: 0.7985137905715485 + value: 0.7982104142453529 name: Spearman Dot - type: pearson_max - value: 0.8041663291384161 + value: 0.8015277726105404 name: Pearson Max - type: spearman_max - value: 0.8057219672338036 + value: 0.8038248041571585 name: Spearman Max --- -# SentenceTransformer based on aubmindlab/bert-base-arabertv02 +# SentenceTransformer based on egyllm/pretrained-arabert -This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [aubmindlab/bert-base-arabertv02](https://huggingface.co/aubmindlab/bert-base-arabertv02). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. +This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [egyllm/pretrained-arabert](https://huggingface.co/egyllm/pretrained-arabert). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer -- **Base model:** [aubmindlab/bert-base-arabertv02](https://huggingface.co/aubmindlab/bert-base-arabertv02) -- **Maximum Sequence Length:** 512 tokens +- **Base model:** [egyllm/pretrained-arabert](https://huggingface.co/egyllm/pretrained-arabert) +- **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity @@ -197,7 +197,7 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [a ``` SentenceTransformer( - (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel + (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` @@ -266,23 +266,23 @@ You can finetune this model on your own dataset. * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) -| Metric | Value | -|:--------------------|:-----------| -| cosine_accuracy@1 | 0.7065 | -| cosine_accuracy@3 | 0.84 | -| cosine_accuracy@5 | 0.877 | -| cosine_accuracy@10 | 0.9125 | -| cosine_precision@1 | 0.7065 | -| cosine_precision@3 | 0.28 | -| cosine_precision@5 | 0.1754 | -| cosine_precision@10 | 0.0912 | -| cosine_recall@1 | 0.7065 | -| cosine_recall@3 | 0.84 | -| cosine_recall@5 | 0.877 | -| cosine_recall@10 | 0.9125 | -| cosine_ndcg@10 | 0.8127 | -| cosine_mrr@10 | 0.7803 | -| **cosine_map@100** | **0.7838** | +| Metric | Value | +|:--------------------|:----------| +| cosine_accuracy@1 | 0.7175 | +| cosine_accuracy@3 | 0.841 | +| cosine_accuracy@5 | 0.878 | +| cosine_accuracy@10 | 0.9155 | +| cosine_precision@1 | 0.7175 | +| cosine_precision@3 | 0.2803 | +| cosine_precision@5 | 0.1756 | +| cosine_precision@10 | 0.0916 | +| cosine_recall@1 | 0.7175 | +| cosine_recall@3 | 0.841 | +| cosine_recall@5 | 0.878 | +| cosine_recall@10 | 0.9155 | +| cosine_ndcg@10 | 0.8172 | +| cosine_mrr@10 | 0.7857 | +| **cosine_map@100** | **0.789** | #### Semantic Similarity * Dataset: `sts-dev` @@ -290,16 +290,16 @@ You can finetune this model on your own dataset. | Metric | Value | |:--------------------|:-----------| -| pearson_cosine | 0.8042 | -| **spearman_cosine** | **0.8057** | -| pearson_manhattan | 0.7939 | -| spearman_manhattan | 0.8045 | -| pearson_euclidean | 0.7922 | +| pearson_cosine | 0.8015 | +| **spearman_cosine** | **0.8038** | +| pearson_manhattan | 0.7895 | +| spearman_manhattan | 0.8012 | +| pearson_euclidean | 0.7894 | | spearman_euclidean | 0.8029 | -| pearson_dot | 0.7985 | -| spearman_dot | 0.7985 | -| pearson_max | 0.8042 | -| spearman_max | 0.8057 | +| pearson_dot | 0.7952 | +| spearman_dot | 0.7982 | +| pearson_max | 0.8015 | +| spearman_max | 0.8038 |