File size: 2,178 Bytes
3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 b859054 3c70f97 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
# 🧠 ClipSegMultiClass
Multiclass semantic segmentation using CLIP + CLIPSeg.
Fine-tuned version of [`CIDAS/clipseg-rd64-refined`](https://huggingface.co/CIDAS/clipseg-rd64-refined)
Supports multiple classes in a single forward pass.
---
## 🔬 Model
**Name:** [`BioMike/clipsegmulticlass_v1`](https://huggingface.co/BioMike/clipsegmulticlass_v1)
**Repository:** [github.com/BioMikeUkr/clipsegmulticlass](https://github.com/BioMikeUkr/clipsegmulticlass)
**Base:** `CIDAS/clipseg-rd64-refined`
**Classes:** `["background", "Pig", "Horse", "Sheep"]`
**Image Size:** 352×352
**Trained on:** OpenImages segmentation subset (custom fruit/animal dataset)
---
## 📊 Evaluation
| Model | Precision | Recall | F1 Score | Accuracy |
|-----------------------------|-----------|---------|----------|----------|
| CIDAS/clipseg-rd64-refined | 0.5239 | 0.2114 | 0.2882 | 0.2665 |
| BioMike/clipsegmulticlass_v1| 0.7460 | 0.5035 | 0.6009 | 0.6763 |
---
## 🎮 Demo
👉 Try it online:
[Hugging Face Space 🚀](https://huggingface.co/spaces/BioMike/clipsegmulticlass)
---
## 📦 Usage
```python
from PIL import Image
import torch
import matplotlib.pyplot as plt
import numpy as np
from model import ClipSegMultiClassModel
from config import ClipSegMultiClassConfig
# Load model
model = ClipSegMultiClassModel.from_pretrained("trained_clipseg_multiclass").to("cuda").eval()
config = model.config # contains label2color
# Load image
image = Image.open("pigs.jpg").convert("RGB")
# Run inference
mask = model.predict(image) # shape: [1, H, W]
# Visualize
def visualize_mask(mask_tensor: torch.Tensor, label2color: dict):
if mask_tensor.dim() == 3:
mask_tensor = mask_tensor.squeeze(0)
mask_np = mask_tensor.cpu().numpy().astype(np.uint8) # [H, W]
h, w = mask_np.shape
color_mask = np.zeros((h, w, 3), dtype=np.uint8)
for class_idx, color in label2color.items():
color_mask[mask_np == class_idx] = color
return color_mask
color_mask = visualize_mask(mask, config.label2color)
plt.imshow(color_mask)
plt.axis("off")
plt.title("Predicted Segmentation Mask")
plt.show()
|