ibm-esa-geospatial
/

Llama3-MS-CLIP-base

Zero-Shot Image Classification

earth-observation

Model card Files Files and versions

blumenstiel commited on 29 days ago

Commit

89b2a0b

·

1 Parent(s): 997a3f5

Update README

Files changed (1) hide show

README.md +2 -3

README.md CHANGED Viewed

@@ -20,7 +20,7 @@ It is trained on one million image-text pairs from the SSL4EO-S12-v1.1 dataset w
 The CLIP model consists of two encoders for text and images. We extended the RGB patch embeddings to multispectral input and initialized the weights of the additional input channels with zeros. During the continual pre-training, the images and texts of each batch are encoded and combined. The loss increases the similarity of matching pairs while decreasing other combinations.
-![Architecture](assets/clip_architecture.png)
 ## Evaluation
@@ -29,8 +29,7 @@ The following Figure compares our model with the OpenCLIP baselines and other EO
 We applied a smoothed min-max scaling and annotated the lowest and highest scores.
 Our multispectral CLIP model is outperforming other RGB-based models on most benchmarks.
-![Benchmarking](assets/benchmarking.png)
 ## Usage

 The CLIP model consists of two encoders for text and images. We extended the RGB patch embeddings to multispectral input and initialized the weights of the additional input channels with zeros. During the continual pre-training, the images and texts of each batch are encoded and combined. The loss increases the similarity of matching pairs while decreasing other combinations.
+![Architecture](https://raw.githubusercontent.com/IBM/MS-CLIP/main/assets/clip_architecture.png)
 ## Evaluation
 We applied a smoothed min-max scaling and annotated the lowest and highest scores.
 Our multispectral CLIP model is outperforming other RGB-based models on most benchmarks.
+![Benchmarking](https://raw.githubusercontent.com/IBM/MS-CLIP/main/assets/benchmarking.png)
 ## Usage