Hugging Face's logo

qualcomm
/
Segment-Anything-Model

Image Segmentation
PyTorch
LiteRT
ONNX
foundation
android
Model card Files Community
qaihm-bot commited on
Commit
8964976
·
verified ·
1 Parent(s): 2bc55b7

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +44 -26
README.md CHANGED
@@ -15,7 +15,7 @@ tags:
15
 
16
  Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.
17
 
18
- This model is an implementation of Segment-Anything-Model found [here](https://github.com/facebookresearch/segment-anything).
19
  This repository provides scripts to run Segment-Anything-Model on Qualcomm® devices.
20
  More details on model performance across various devices, can be found
21
  [here](https://aihub.qualcomm.com/models/sam).
@@ -30,15 +30,27 @@ More details on model performance across various devices, can be found
30
  - Number of parameters (SAMDecoder): 5.11M
31
  - Model size (SAMDecoder): 19.6 MB
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
 
35
 
36
- | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
37
- | ---|---|---|---|---|---|---|---|
38
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 29.972 ms | 4 - 12 MB | FP16 | NPU | [SAMDecoder.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite)
39
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 11293.293 ms | 38 - 215 MB | FP32 | CPU | [SAMEncoder.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite)
40
-
41
-
42
 
43
  ## Installation
44
 
@@ -94,23 +106,25 @@ device. This script does the following:
94
  ```bash
95
  python -m qai_hub_models.models.sam.export
96
  ```
97
-
98
  ```
99
- Profile Job summary of SAMDecoder
100
- --------------------------------------------------
101
- Device: SA8255 (Proxy) (13)
102
- Estimated Inference Time: 29.91 ms
103
- Estimated Peak Memory Range: 3.82-11.20 MB
104
- Compute Units: NPU (337) | Total (337)
105
-
106
- Profile Job summary of SAMEncoder
107
- --------------------------------------------------
108
- Device: SA8255 (Proxy) (13)
109
- Estimated Inference Time: 11339.80 ms
110
- Estimated Peak Memory Range: 123.86-127.24 MB
111
- Compute Units: GPU (36),CPU (782) | Total (818)
112
-
113
-
 
 
 
114
  ```
115
 
116
 
@@ -252,15 +266,19 @@ provides instructions on how to use the `.so` shared library in an Android appl
252
  Get more details on Segment-Anything-Model's performance across various devices [here](https://aihub.qualcomm.com/models/sam).
253
  Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
254
 
 
255
  ## License
256
- - The license for the original implementation of Segment-Anything-Model can be found
257
- [here](https://github.com/facebookresearch/segment-anything/blob/main/LICENSE).
258
- - The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
 
259
 
260
  ## References
261
  * [Segment Anything](https://arxiv.org/abs/2304.02643)
262
  * [Source Model Implementation](https://github.com/facebookresearch/segment-anything)
263
 
 
 
264
  ## Community
265
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
266
  * For questions or feedback please [reach out to us](mailto:[email protected]).
 
15
 
16
  Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.
17
 
18
+ This model is an implementation of Segment-Anything-Model found [here]({source_repo}).
19
  This repository provides scripts to run Segment-Anything-Model on Qualcomm® devices.
20
  More details on model performance across various devices, can be found
21
  [here](https://aihub.qualcomm.com/models/sam).
 
30
  - Number of parameters (SAMDecoder): 5.11M
31
  - Model size (SAMDecoder): 19.6 MB
32
 
33
+ | Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
34
+ |---|---|---|---|---|---|---|---|---|
35
+ | SAMDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 29.098 ms | 2 - 20 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
36
+ | SAMDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 20.232 ms | 2 - 227 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
37
+ | SAMDecoder | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 28.959 ms | 4 - 12 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
38
+ | SAMDecoder | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 29.061 ms | 4 - 25 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
39
+ | SAMDecoder | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 28.99 ms | 4 - 47 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
40
+ | SAMDecoder | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 29.004 ms | 4 - 7 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
41
+ | SAMDecoder | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 32.396 ms | 4 - 222 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
42
+ | SAMDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 20.466 ms | 2 - 157 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
43
+ | SAMEncoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 11323.51 ms | 0 - 272 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
44
+ | SAMEncoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 8300.484 ms | 123 - 1639 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
45
+ | SAMEncoder | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 10870.158 ms | 124 - 286 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
46
+ | SAMEncoder | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 10178.345 ms | 121 - 124 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
47
+ | SAMEncoder | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 11283.428 ms | 120 - 125 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
48
+ | SAMEncoder | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 10102.843 ms | 121 - 125 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
49
+ | SAMEncoder | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 13526.091 ms | 131 - 1692 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
50
+ | SAMEncoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 6334.196 ms | 98 - 1573 MB | FP32 | CPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoder.tflite) |
51
 
52
 
53
 
 
 
 
 
 
 
54
 
55
  ## Installation
56
 
 
106
  ```bash
107
  python -m qai_hub_models.models.sam.export
108
  ```
 
109
  ```
110
+ Profiling Results
111
+ ------------------------------------------------------------
112
+ SAMDecoder
113
+ Device : Samsung Galaxy S23 (13)
114
+ Runtime : TFLITE
115
+ Estimated inference time (ms) : 29.1
116
+ Estimated peak memory usage (MB): [2, 20]
117
+ Total # Ops : 337
118
+ Compute Unit(s) : NPU (337 ops)
119
+
120
+ ------------------------------------------------------------
121
+ SAMEncoder
122
+ Device : Samsung Galaxy S23 (13)
123
+ Runtime : TFLITE
124
+ Estimated inference time (ms) : 11323.5
125
+ Estimated peak memory usage (MB): [0, 272]
126
+ Total # Ops : 818
127
+ Compute Unit(s) : GPU (36 ops) CPU (782 ops)
128
  ```
129
 
130
 
 
266
  Get more details on Segment-Anything-Model's performance across various devices [here](https://aihub.qualcomm.com/models/sam).
267
  Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
268
 
269
+
270
  ## License
271
+ * The license for the original implementation of Segment-Anything-Model can be found [here](https://github.com/facebookresearch/segment-anything/blob/main/LICENSE).
272
+ * The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
273
+
274
+
275
 
276
  ## References
277
  * [Segment Anything](https://arxiv.org/abs/2304.02643)
278
  * [Source Model Implementation](https://github.com/facebookresearch/segment-anything)
279
 
280
+
281
+
282
  ## Community
283
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
284
  * For questions or feedback please [reach out to us](mailto:[email protected]).