tshasan commited on
Commit
d86f700
·
verified ·
1 Parent(s): 5093161

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -16
README.md CHANGED
@@ -1,31 +1,83 @@
1
  ---
2
  license: cc-by-4.0
3
  tags:
4
- - multi-label-classification
5
- - text-classification
6
- - onnx
7
- - web-classification
8
- - firefox-ai
9
- - preview
10
  language:
11
- - multilingual
12
  datasets:
13
- - tshasan/multi-label-web-classification
14
  base_model: Alibaba-NLP/gte-modernbert-base
15
  pipeline_tag: text-classification
16
  ---
17
 
18
- # modernBERT-URLTITLE-classifier-preview
19
 
20
  ## Model Overview
21
 
22
- This is a **preview version** of a multi-label web classification model fine-tuned from `Alibaba-NLP/gte-modernbert-base`. It classifies websites into multiple categories based on their URLs and titles. The model supports 11 labels: `Uncatergorized`,`News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, and `Travel`.
 
 
 
 
 
 
 
23
 
24
- - **Developed by**: Taimur Hasan
25
- - **Model Type**: Multi-label Text Classification
26
- - **Status**: Preview (under active development
27
  ### Architecture
28
- - **Fine-tuning**: Unfroze the last 4 encoder layers and the pooler
 
29
  - **Problem Type**: Multi-label classification
30
- - **Output Labels**: 11 (`News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, `Travel`,`Uncatergorized`)
31
- - **Input Format**: Concatenated string: `"{url}:{title}"`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-4.0
3
  tags:
4
+ - multi-label-classification
5
+ - text-classification
6
+ - onnx
7
+ - web-classification
8
+ - firefox-ai
9
+ - preview
10
  language:
11
+ - multilingual
12
  datasets:
13
+ - tshasan/multi-label-web-classification
14
  base_model: Alibaba-NLP/gte-modernbert-base
15
  pipeline_tag: text-classification
16
  ---
17
 
18
+ # URL-TITLE-classifier-preview
19
 
20
  ## Model Overview
21
 
22
+ This is a **preview version** of a multi-label web classification model fine-tuned from [`Alibaba-NLP/gte-modernbert-base`](https://huggingface.co/Alibaba-NLP/gte-modernbert-base). It classifies websites into multiple categories based on their URLs and titles.
23
+
24
+ The model supports **11 labels**:
25
+ `Uncategorized`, `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, and `Travel`.
26
+
27
+ - **Developed by**: Taimur Hasan
28
+ - **Model Type**: Multi-label Text Classification
29
+ - **Status**: Preview (under active development)
30
 
 
 
 
31
  ### Architecture
32
+
33
+ - **Fine-tuning Strategy**: Unfroze the last 4 encoder layers and the pooler
34
  - **Problem Type**: Multi-label classification
35
+ - **Output Labels**:
36
+ - `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, `Travel`, `Uncategorized`
37
+ - **Input Format**: Concatenated string:
38
+ `"{url}:{title}"`
39
+
40
+ ---
41
+
42
+ ## Evaluation Metrics (Validation Data)
43
+
44
+ | Metric | Value |
45
+ |-----------------------|--------|
46
+ | **Loss** | 0.207 |
47
+ | **Hamming Loss** | 0.083 |
48
+ | **Exact Match** | 0.445 |
49
+ | **Precision (Micro)** | 0.917 |
50
+ | **Recall (Micro)** | 0.917 |
51
+ | **F1 Score (Micro)** | 0.917 |
52
+ | **Precision (Macro)** | 0.795 |
53
+ | **Recall (Macro)** | 0.598 |
54
+ | **F1 Score (Macro)** | 0.677 |
55
+ | **Precision (Weighted)** | 0.798 |
56
+ | **Recall (Weighted)** | 0.647 |
57
+ | **F1 Score (Weighted)** | 0.711 |
58
+ | **ROC AUC (Micro)** | 0.941 |
59
+ | **ROC AUC (Macro)** | 0.928 |
60
+ | **PR AUC (Micro)** | 0.815 |
61
+ | **PR AUC (Macro)** | 0.765 |
62
+ | **Jaccard (Micro)** | 0.848 |
63
+ | **Jaccard (Macro)** | 0.520 |
64
+
65
+ ### Per-Label F1 Scores
66
+
67
+ | Label | F1 Score |
68
+ |----------------|----------|
69
+ | News | 0.605 |
70
+ | Entertainment | 0.764 |
71
+ | Shop | 0.704 |
72
+ | Chat | 0.875 |
73
+ | Education | 0.763 |
74
+ | Government | 0.667 |
75
+ | Health | 0.574 |
76
+ | Technology | 0.738 |
77
+ | Work | 0.527 |
78
+ | Travel | 0.571 |
79
+ | Uncategorized | 0.657 |
80
+
81
+ ---
82
+
83
+ > **Note:** This model is in preview and may not generalize well outside of its training dataset. Feedback and contributions are welcome.