|
--- |
|
license: cc-by-4.0 |
|
tags: |
|
- multi-label-classification |
|
- text-classification |
|
- onnx |
|
- web-classification |
|
- firefox-ai |
|
- preview |
|
language: |
|
- multilingual |
|
datasets: |
|
- tshasan/multi-label-web-classification |
|
base_model: Alibaba-NLP/gte-modernbert-base |
|
pipeline_tag: text-classification |
|
library_name: transformers |
|
--- |
|
|
|
# URL-TITLE-classifier-preview |
|
|
|
## Model Overview |
|
|
|
This is a **preview version** of a multi-label web classification model fine-tuned from [`Alibaba-NLP/gte-modernbert-base`](https://huggingface.co/Alibaba-NLP/gte-modernbert-base). It classifies websites into multiple categories based on their URLs and titles. |
|
|
|
The model supports **11 labels**: |
|
`Uncategorized`, `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, and `Travel`. |
|
|
|
- **Developed by**: Taimur Hasan |
|
- **Model Type**: Multi-label Text Classification |
|
- **Status**: Preview (under active development) |
|
|
|
### Architecture |
|
|
|
- **Fine-tuning Strategy**: Unfroze the last 4 encoder layers and the pooler |
|
- **Problem Type**: Multi-label classification |
|
- **Output Labels**: |
|
- `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, `Travel`, `Uncategorized` |
|
- **Input Format**: Concatenated string: |
|
`"{url}:{title}"` |
|
|
|
--- |
|
|
|
## Evaluation Metrics (Validation Data) |
|
|
|
| Metric | Value | |
|
|-----------------------|--------| |
|
| **Loss** | 0.207 | |
|
| **Hamming Loss** | 0.083 | |
|
| **Exact Match** | 0.445 | |
|
| **Precision (Micro)** | 0.917 | |
|
| **Recall (Micro)** | 0.917 | |
|
| **F1 Score (Micro)** | 0.917 | |
|
| **Precision (Macro)** | 0.795 | |
|
| **Recall (Macro)** | 0.598 | |
|
| **F1 Score (Macro)** | 0.677 | |
|
| **Precision (Weighted)** | 0.798 | |
|
| **Recall (Weighted)** | 0.647 | |
|
| **F1 Score (Weighted)** | 0.711 | |
|
| **ROC AUC (Micro)** | 0.941 | |
|
| **ROC AUC (Macro)** | 0.928 | |
|
| **PR AUC (Micro)** | 0.815 | |
|
| **PR AUC (Macro)** | 0.765 | |
|
| **Jaccard (Micro)** | 0.848 | |
|
| **Jaccard (Macro)** | 0.520 | |
|
|
|
### Per-Label F1 Scores |
|
|
|
| Label | F1 Score | |
|
|----------------|----------| |
|
| News | 0.605 | |
|
| Entertainment | 0.764 | |
|
| Shop | 0.704 | |
|
| Chat | 0.875 | |
|
| Education | 0.763 | |
|
| Government | 0.667 | |
|
| Health | 0.574 | |
|
| Technology | 0.738 | |
|
| Work | 0.527 | |
|
| Travel | 0.571 | |
|
| Uncategorized | 0.657 | |
|
|
|
--- |
|
|
|
> **Note:** This model is in preview and may not generalize well outside of its training dataset. Feedback and contributions are welcome. |
