---
license: cc-by-4.0
tags:
- multi-label-classification
- text-classification
- onnx
- web-classification
- firefox-ai
- preview
language:
- multilingual
datasets:
- tshasan/multi-label-web-classification
base_model: Alibaba-NLP/gte-modernbert-base
pipeline_tag: text-classification
library_name: transformers
---

# URL-TITLE-classifier-preview

## Model Overview

This is a **preview version** of a multi-label web classification model fine-tuned from [`Alibaba-NLP/gte-modernbert-base`](https://huggingface.co/Alibaba-NLP/gte-modernbert-base). It classifies websites into multiple categories based on their URLs and titles. 

The model supports **11 labels**:  
`Uncategorized`, `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, and `Travel`.

- **Developed by**: Taimur Hasan  
- **Model Type**: Multi-label Text Classification  
- **Status**: Preview (under active development)

### Architecture

- **Fine-tuning Strategy**: Unfroze the last 4 encoder layers and the pooler
- **Problem Type**: Multi-label classification
- **Output Labels**:  
  - `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, `Travel`, `Uncategorized`
- **Input Format**: Concatenated string:  
  `"{url}:{title}"`

---

## Evaluation Metrics (Validation Data)

| Metric                | Value  |
|-----------------------|--------|
| **Loss**              | 0.207  |
| **Hamming Loss**      | 0.083  |
| **Exact Match**       | 0.445  |
| **Precision (Micro)** | 0.917  |
| **Recall (Micro)**    | 0.917  |
| **F1 Score (Micro)**  | 0.917  |
| **Precision (Macro)** | 0.795  |
| **Recall (Macro)**    | 0.598  |
| **F1 Score (Macro)**  | 0.677  |
| **Precision (Weighted)** | 0.798 |
| **Recall (Weighted)**    | 0.647 |
| **F1 Score (Weighted)**  | 0.711 |
| **ROC AUC (Micro)**      | 0.941 |
| **ROC AUC (Macro)**      | 0.928 |
| **PR AUC (Micro)**       | 0.815 |
| **PR AUC (Macro)**       | 0.765 |
| **Jaccard (Micro)**      | 0.848 |
| **Jaccard (Macro)**      | 0.520 |

### Per-Label F1 Scores

| Label           | F1 Score |
|----------------|----------|
| News           | 0.605    |
| Entertainment  | 0.764    |
| Shop           | 0.704    |
| Chat           | 0.875    |
| Education      | 0.763    |
| Government     | 0.667    |
| Health         | 0.574    |
| Technology     | 0.738    |
| Work           | 0.527    |
| Travel         | 0.571    |
| Uncategorized  | 0.657    |

---

> **Note:** This model is in preview and may not generalize well outside of its training dataset. Feedback and contributions are welcome.