Spaces:
Running
Running
Update utils.py
Browse files
utils.py
CHANGED
|
@@ -57,9 +57,9 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
|
|
| 57 |
|
| 58 |
## ⚠ Please note that you need to submit the JSON file with the following format:
|
| 59 |
|
| 60 |
-
### ***Important Notes: We have released MMEB-V2 and will deprecate MMEB-V1 soon
|
|
|
|
| 61 |
### ***In V2, the detailed scores of each dataset will be included, and our code will automatically generate the results and calculate the overall scores. See the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for more information.***
|
| 62 |
-
### **A V2 Submission would look like this:**
|
| 63 |
```json
|
| 64 |
{
|
| 65 |
"metadata": {
|
|
@@ -103,7 +103,7 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
|
|
| 103 |
}
|
| 104 |
}
|
| 105 |
```
|
| 106 |
-
Note: We still accept the old format until 2025-06-30
|
| 107 |
Please refer to the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for detailed instructions about evaluating your model. \n
|
| 108 |
To submit, create a pull request and upload the generated JSON file to the ***scores*** folder, then send us an email at [email protected], including your model's information. \n We will review your submission and update the leaderboard accordingly. \n
|
| 109 |
Please also share any feedback or suggestions you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
|
@@ -179,7 +179,7 @@ def process_model_size(size):
|
|
| 179 |
return 'unknown'
|
| 180 |
try:
|
| 181 |
val = float(size)
|
| 182 |
-
return val
|
| 183 |
except (ValueError, TypeError):
|
| 184 |
return 'unknown'
|
| 185 |
|
|
|
|
| 57 |
|
| 58 |
## ⚠ Please note that you need to submit the JSON file with the following format:
|
| 59 |
|
| 60 |
+
### ***Important Notes: We have released MMEB-V2 and will deprecate MMEB-V1 soon.*** \n
|
| 61 |
+
### ***All further submissions should be made using the V2 format (see following).*** \n
|
| 62 |
### ***In V2, the detailed scores of each dataset will be included, and our code will automatically generate the results and calculate the overall scores. See the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for more information.***
|
|
|
|
| 63 |
```json
|
| 64 |
{
|
| 65 |
"metadata": {
|
|
|
|
| 103 |
}
|
| 104 |
}
|
| 105 |
```
|
| 106 |
+
Note: We still accept the old format until 2025-06-30.
|
| 107 |
Please refer to the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for detailed instructions about evaluating your model. \n
|
| 108 |
To submit, create a pull request and upload the generated JSON file to the ***scores*** folder, then send us an email at [email protected], including your model's information. \n We will review your submission and update the leaderboard accordingly. \n
|
| 109 |
Please also share any feedback or suggestions you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
|
|
|
| 179 |
return 'unknown'
|
| 180 |
try:
|
| 181 |
val = float(size)
|
| 182 |
+
return round(val, 3)
|
| 183 |
except (ValueError, TypeError):
|
| 184 |
return 'unknown'
|
| 185 |
|