Delete eval_milebench

eval_milebench/ALFRED.log

@@ -1,107 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ALFRED, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ALFRED, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ALFRED, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ALFRED, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Initialization Finished
-Predicting ALFRED Using internvl
-Proceeding 5-length images samples | Num: 36
-Initialization Finished
-Predicting ALFRED Using internvl
-Proceeding 5-length images samples | Num: 36
-Initialization Finished
-Predicting ALFRED Using internvl
-Proceeding 5-length images samples | Num: 36
-Initialization Finished
-Predicting ALFRED Using internvl
-Proceeding 5-length images samples | Num: 36
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:02<00:17,  2.25s/it]
 22%|██▏       | 2/9 [00:03<00:10,  1.48s/it]
 33%|███▎      | 3/9 [00:04<00:07,  1.23s/it]
 44%|████▍     | 4/9 [00:05<00:05,  1.12s/it]
 56%|█████▌    | 5/9 [00:06<00:04,  1.06s/it]
 67%|██████▋   | 6/9 [00:06<00:02,  1.16it/s]
 78%|███████▊  | 7/9 [00:07<00:01,  1.12it/s]
 89%|████████▉ | 8/9 [00:08<00:00,  1.09it/s]
-Proceeding 4-length images samples | Num: 62
-Proceeding 4-length images samples | Num: 62
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:02<00:22,  2.78s/it]
 22%|██▏       | 2/9 [00:03<00:12,  1.73s/it]
 33%|███▎      | 3/9 [00:04<00:08,  1.37s/it]
 44%|████▍     | 4/9 [00:05<00:06,  1.23s/it]
 56%|█████▌    | 5/9 [00:07<00:05,  1.25s/it]
 67%|██████▋   | 6/9 [00:08<00:03,  1.16s/it]
 78%|███████▊  | 7/9 [00:08<00:02,  1.10s/it]
 89%|████████▉ | 8/9 [00:10<00:01,  1.20s/it]
-Proceeding 4-length images samples | Num: 62
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:02<00:21,  2.73s/it]
 22%|██▏       | 2/9 [00:03<00:12,  1.75s/it]
 33%|███▎      | 3/9 [00:04<00:08,  1.41s/it]
 44%|████▍     | 4/9 [00:05<00:06,  1.24s/it]
 56%|█████▌    | 5/9 [00:06<00:04,  1.13s/it]
 67%|██████▋   | 6/9 [00:07<00:03,  1.08s/it]
 78%|███████▊  | 7/9 [00:08<00:02,  1.04s/it]
 89%|████████▉ | 8/9 [00:10<00:01,  1.33s/it]
-Proceeding 4-length images samples | Num: 62
-
  0%|          | 0/15 [00:00<?, ?it/s]
  7%|▋         | 1/15 [00:01<00:17,  1.25s/it]
 13%|█▎        | 2/15 [00:02<00:13,  1.05s/it]
 20%|██        | 3/15 [00:03<00:11,  1.01it/s]
 27%|██▋       | 4/15 [00:04<00:11,  1.06s/it]
 33%|███▎      | 5/15 [00:05<00:10,  1.01s/it]
 40%|████      | 6/15 [00:06<00:08,  1.01it/s]
 47%|████▋     | 7/15 [00:07<00:08,  1.07s/it]
 53%|█████▎    | 8/15 [00:08<00:07,  1.03s/it]
 60%|██████    | 9/15 [00:08<00:05,  1.17it/s]
 67%|██████▋   | 10/15 [00:09<00:04,  1.06it/s]
 73%|███████▎  | 11/15 [00:10<00:03,  1.06it/s]
 80%|████████  | 12/15 [00:11<00:02,  1.05it/s]
 87%|████████▋ | 13/15 [00:12<00:01,  1.06it/s]
 93%|█████████▎| 14/15 [00:13<00:00,  1.06it/s]
-Proceeding 3-length images samples | Num: 51
-Proceeding 3-length images samples | Num: 51
-
  0%|          | 0/15 [00:00<?, ?it/s]
  7%|▋         | 1/15 [00:01<00:17,  1.26s/it]
 13%|█▎        | 2/15 [00:02<00:16,  1.25s/it]
 20%|██        | 3/15 [00:03<00:12,  1.06s/it]
 27%|██▋       | 4/15 [00:04<00:11,  1.02s/it]
 33%|███▎      | 5/15 [00:05<00:10,  1.04s/it]
 40%|████      | 6/15 [00:06<00:09,  1.00s/it]
 47%|████▋     | 7/15 [00:07<00:07,  1.02it/s]
 53%|█████▎    | 8/15 [00:08<00:06,  1.03it/s]
 60%|██████    | 9/15 [00:09<00:05,  1.04it/s]
 67%|██████▋   | 10/15 [00:10<00:04,  1.05it/s]
 73%|███████▎  | 11/15 [00:11<00:03,  1.05it/s]
 80%|████████  | 12/15 [00:11<00:02,  1.05it/s]
 87%|████████▋ | 13/15 [00:12<00:01,  1.05it/s]
 93%|█████████▎| 14/15 [00:13<00:00,  1.10it/s]
-Proceeding 3-length images samples | Num: 51
-
  0%|          | 0/16 [00:00<?, ?it/s]
  6%|▋         | 1/16 [00:01<00:19,  1.32s/it]
 12%|█▎        | 2/16 [00:02<00:15,  1.10s/it]
 19%|█▉        | 3/16 [00:03<00:12,  1.05it/s]
 25%|██▌       | 4/16 [00:04<00:12,  1.03s/it]
 31%|███▏      | 5/16 [00:05<00:11,  1.02s/it]
 38%|███▊      | 6/16 [00:06<00:09,  1.00it/s]
 44%|████▍     | 7/16 [00:07<00:08,  1.02it/s]
 50%|█████     | 8/16 [00:07<00:07,  1.10it/s]
 56%|█████▋    | 9/16 [00:08<00:06,  1.08it/s]
 62%|██████▎   | 10/16 [00:09<00:05,  1.07it/s]
 69%|██████▉   | 11/16 [00:10<00:04,  1.07it/s]
 75%|███████▌  | 12/16 [00:11<00:03,  1.06it/s]
 81%|████████▏ | 13/16 [00:12<00:02,  1.05it/s]
 88%|████████▊ | 14/16 [00:13<00:02,  1.03s/it]
 94%|█████████▍| 15/16 [00:14<00:00,  1.10it/s]
-Proceeding 3-length images samples | Num: 51
-
  0%|          | 0/13 [00:00<?, ?it/s]
  8%|▊         | 1/13 [00:01<00:13,  1.12s/it]
 15%|█▌        | 2/13 [00:01<00:08,  1.28it/s]
 23%|██▎       | 3/13 [00:02<00:08,  1.13it/s]
 31%|███       | 4/13 [00:03<00:08,  1.09it/s]
 38%|███▊      | 5/13 [00:04<00:07,  1.09it/s]
 46%|████▌     | 6/13 [00:05<00:06,  1.08it/s]
 54%|█████▍    | 7/13 [00:06<00:05,  1.19it/s]
 62%|██████▏   | 8/13 [00:07<00:04,  1.16it/s]
 69%|██████▉   | 9/13 [00:07<00:03,  1.20it/s]
 77%|███████▋  | 10/13 [00:08<00:02,  1.17it/s]
 85%|████████▍ | 11/13 [00:09<00:01,  1.14it/s]
 92%|█████████▏| 12/13 [00:10<00:00,  1.13it/s]
-Proceeding 2-length images samples | Num: 51
-Proceeding 2-length images samples | Num: 51
-
  0%|          | 0/12 [00:00<?, ?it/s]
  8%|▊         | 1/12 [00:00<00:09,  1.15it/s]
 17%|█▋        | 2/12 [00:01<00:09,  1.10it/s]
 25%|██▌       | 3/12 [00:02<00:07,  1.19it/s]
 33%|███▎      | 4/12 [00:03<00:07,  1.14it/s]
 42%|████▏     | 5/12 [00:04<00:06,  1.12it/s]
 50%|█████     | 6/12 [00:05<00:05,  1.14it/s]
 58%|█████▊    | 7/12 [00:05<00:04,  1.22it/s]
 67%|██████▋   | 8/12 [00:06<00:02,  1.35it/s]
 75%|███████▌  | 9/12 [00:07<00:02,  1.24it/s]
 83%|████████▎ | 10/12 [00:08<00:01,  1.19it/s]
 92%|█████████▏| 11/12 [00:09<00:00,  1.15it/s]
-Proceeding 2-length images samples | Num: 51
-
  0%|          | 0/13 [00:00<?, ?it/s]
  8%|▊         | 1/13 [00:01<00:14,  1.22s/it]
 15%|█▌        | 2/13 [00:01<00:10,  1.05it/s]
 23%|██▎       | 3/13 [00:02<00:09,  1.07it/s]
 31%|███       | 4/13 [00:03<00:08,  1.07it/s]
 38%|███▊      | 5/13 [00:04<00:07,  1.07it/s]
 46%|████▌     | 6/13 [00:05<00:06,  1.16it/s]
 54%|█████▍    | 7/13 [00:05<00:04,  1.37it/s]
 62%|██████▏   | 8/13 [00:06<00:03,  1.26it/s]
 69%|██████▉   | 9/13 [00:07<00:03,  1.19it/s]
 77%|███████▋  | 10/13 [00:08<00:02,  1.25it/s]
 85%|████████▍ | 11/13 [00:09<00:01,  1.21it/s]
 92%|█████████▏| 12/13 [00:10<00:00,  1.17it/s]
-Proceeding 2-length images samples | Num: 51
-
  0%|          | 0/13 [00:00<?, ?it/s]
  8%|▊         | 1/13 [00:01<00:14,  1.19s/it]
 15%|█▌        | 2/13 [00:01<00:08,  1.33it/s]
 23%|██▎       | 3/13 [00:02<00:08,  1.23it/s]
 31%|███       | 4/13 [00:02<00:05,  1.56it/s]
 38%|███▊      | 5/13 [00:03<00:05,  1.43it/s]
 46%|████▌     | 6/13 [00:04<00:04,  1.64it/s]
 54%|█████▍    | 7/13 [00:05<00:04,  1.42it/s]
 62%|██████▏   | 8/13 [00:05<00:03,  1.30it/s]
 69%|██████▉   | 9/13 [00:06<00:03,  1.23it/s]
 77%|███████▋  | 10/13 [00:07<00:02,  1.36it/s]
 85%|████████▍ | 11/13 [00:07<00:01,  1.57it/s]
 92%|█████████▏| 12/13 [00:08<00:00,  1.38it/s]
-
  0%|          | 0/12 [00:00<?, ?it/s]
  8%|▊         | 1/12 [00:00<00:07,  1.45it/s]
 17%|█▋        | 2/12 [00:01<00:08,  1.23it/s]
 25%|██▌       | 3/12 [00:02<00:07,  1.17it/s]
 33%|███▎      | 4/12 [00:03<00:06,  1.15it/s]
 42%|████▏     | 5/12 [00:03<00:05,  1.36it/s]
 50%|█████     | 6/12 [00:04<00:04,  1.36it/s]
 58%|█████▊    | 7/12 [00:05<00:03,  1.27it/s]
 67%|██████▋   | 8/12 [00:05<00:02,  1.49it/s]
 75%|███████▌  | 9/12 [00:06<00:02,  1.35it/s]
 83%|████████▎ | 10/12 [00:07<00:01,  1.41it/s]
 92%|█████████▏| 11/12 [00:08<00:00,  1.52it/s]
-
  0%|          | 0/13 [00:00<?, ?it/s]
  8%|▊         | 1/13 [00:01<00:14,  1.20s/it]
 15%|█▌        | 2/13 [00:01<00:09,  1.21it/s]
 23%|██▎       | 3/13 [00:02<00:08,  1.16it/s]
 31%|███       | 4/13 [00:03<00:07,  1.14it/s]
 38%|███▊      | 5/13 [00:04<00:05,  1.39it/s]
 46%|████▌     | 6/13 [00:04<00:04,  1.49it/s]
 54%|█████▍    | 7/13 [00:05<00:04,  1.34it/s]
 62%|██████▏   | 8/13 [00:06<00:03,  1.25it/s]
 69%|██████▉   | 9/13 [00:07<00:03,  1.33it/s]
 77%|███████▋  | 10/13 [00:07<00:02,  1.43it/s]
 85%|████████▍ | 11/13 [00:08<00:01,  1.31it/s]
 92%|█████████▏| 12/13 [00:08<00:00,  1.57it/s]
-evaluating ALFRED ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/ALFRED/ALFRED_240803234459.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset ALFRED                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/ALFRED 
-internvl:  ALFRED:  {'Rouge-L f': 0.2537993395427049, 'image_quantity_level-Accuracy': {'Few': 0.2537993395427049, 'Medium': 0, 'Many': 0}, 'image_quantity_level-Result': {'Few': [50.759867908540976, 200], 'Medium': [0, 0], 'Many': [0, 0]}}

eval_milebench/ALFRED/eval.json

@@ -1 +0,0 @@
-{"Rouge-L f": 0.2537993395427049, "image_quantity_level-Accuracy": {"Few": 0.2537993395427049, "Medium": 0, "Many": 0}, "image_quantity_level-Result": {"Few": [50.759867908540976, 200], "Medium": [0, 0], "Many": [0, 0]}}

eval_milebench/ALFRED/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "0.129"
-    },
-    {
-        "id": "6",
-        "score": "0.364"
-    },
-    {
-        "id": "11",
-        "score": "0.286"
-    },
-    {
-        "id": "26",
-        "score": "0.2"
-    },
-    {
-        "id": "29",
-        "score": "0.273"
-    },
-    {
-        "id": "46",
-        "score": "0.194"
-    },
-    {
-        "id": "51",
-        "score": "0.062"
-    },
-    {
-        "id": "54",
-        "score": "0.25"
-    },
-    {
-        "id": "56",
-        "score": "0.333"
-    },
-    {
-        "id": "1",
-        "score": "0.19"
-    },
-    {
-        "id": "4",
-        "score": "0.261"
-    },
-    {
-        "id": "13",
-        "score": "0.091"
-    },
-    {
-        "id": "14",
-        "score": "0.2"
-    },
-    {
-        "id": "18",
-        "score": "0.069"
-    },
-    {
-        "id": "19",
-        "score": "0.296"
-    },
-    {
-        "id": "23",
-        "score": "0.207"
-    },
-    {
-        "id": "24",
-        "score": "0.16"
-    },
-    {
-        "id": "25",
-        "score": "0.296"
-    },
-    {
-        "id": "27",
-        "score": "0.148"
-    },
-    {
-        "id": "28",
-        "score": "0.205"
-    },
-    {
-        "id": "31",
-        "score": "0.08"
-    },
-    {
-        "id": "33",
-        "score": "0.174"
-    },
-    {
-        "id": "34",
-        "score": "0.345"
-    },
-    {
-        "id": "39",
-        "score": "0.375"
-    },
-    {
-        "id": "43",
-        "score": "0.286"
-    },
-    {
-        "id": "2",
-        "score": "0.286"
-    },
-    {
-        "id": "7",
-        "score": "0.333"
-    },
-    {
-        "id": "8",
-        "score": "0.625"
-    },
-    {
-        "id": "9",
-        "score": "0.5"
-    },
-    {
-        "id": "10",
-        "score": "0.25"
-    },
-    {
-        "id": "12",
-        "score": "0.387"
-    },
-    {
-        "id": "15",
-        "score": "0.385"
-    },
-    {
-        "id": "21",
-        "score": "0.593"
-    },
-    {
-        "id": "22",
-        "score": "0.187"
-    },
-    {
-        "id": "35",
-        "score": "0.357"
-    },
-    {
-        "id": "40",
-        "score": "0.286"
-    },
-    {
-        "id": "41",
-        "score": "0.444"
-    },
-    {
-        "id": "42",
-        "score": "0.276"
-    },
-    {
-        "id": "3",
-        "score": "0.231"
-    },
-    {
-        "id": "5",
-        "score": "0.24"
-    },
-    {
-        "id": "16",
-        "score": "0.2"
-    },
-    {
-        "id": "17",
-        "score": "0.385"
-    },
-    {
-        "id": "20",
-        "score": "0.174"
-    },
-    {
-        "id": "30",
-        "score": "0.308"
-    },
-    {
-        "id": "32",
-        "score": "0.148"
-    },
-    {
-        "id": "36",
-        "score": "0.0"
-    },
-    {
-        "id": "37",
-        "score": "0.267"
-    },
-    {
-        "id": "38",
-        "score": "0.222"
-    },
-    {
-        "id": "44",
-        "score": "0.095"
-    },
-    {
-        "id": "47",
-        "score": "0.133"
-    },
-    {
-        "id": "57",
-        "score": "0.182"
-    },
-    {
-        "id": "58",
-        "score": "0.429"
-    },
-    {
-        "id": "62",
-        "score": "0.133"
-    },
-    {
-        "id": "63",
-        "score": "0.214"
-    },
-    {
-        "id": "66",
-        "score": "0.2"
-    },
-    {
-        "id": "71",
-        "score": "0.5"
-    },
-    {
-        "id": "79",
-        "score": "0.133"
-    },
-    {
-        "id": "80",
-        "score": "0.143"
-    },
-    {
-        "id": "83",
-        "score": "0.424"
-    },
-    {
-        "id": "85",
-        "score": "0.412"
-    },
-    {
-        "id": "45",
-        "score": "0.194"
-    },
-    {
-        "id": "48",
-        "score": "0.083"
-    },
-    {
-        "id": "52",
-        "score": "0.214"
-    },
-    {
-        "id": "53",
-        "score": "0.211"
-    },
-    {
-        "id": "55",
-        "score": "0.235"
-    },
-    {
-        "id": "61",
-        "score": "0.074"
-    },
-    {
-        "id": "74",
-        "score": "0.067"
-    },
-    {
-        "id": "75",
-        "score": "0.154"
-    },
-    {
-        "id": "84",
-        "score": "0.16"
-    },
-    {
-        "id": "86",
-        "score": "0.357"
-    },
-    {
-        "id": "87",
-        "score": "0.176"
-    },
-    {
-        "id": "91",
-        "score": "0.19"
-    },
-    {
-        "id": "92",
-        "score": "0.235"
-    },
-    {
-        "id": "94",
-        "score": "0.08"
-    },
-    {
-        "id": "96",
-        "score": "0.25"
-    },
-    {
-        "id": "100",
-        "score": "0.133"
-    },
-    {
-        "id": "49",
-        "score": "0.348"
-    },
-    {
-        "id": "50",
-        "score": "0.375"
-    },
-    {
-        "id": "67",
-        "score": "0.6"
-    },
-    {
-        "id": "68",
-        "score": "0.258"
-    },
-    {
-        "id": "69",
-        "score": "0.455"
-    },
-    {
-        "id": "81",
-        "score": "0.467"
-    },
-    {
-        "id": "88",
-        "score": "0.211"
-    },
-    {
-        "id": "89",
-        "score": "0.414"
-    },
-    {
-        "id": "93",
-        "score": "0.214"
-    },
-    {
-        "id": "97",
-        "score": "0.083"
-    },
-    {
-        "id": "101",
-        "score": "0.4"
-    },
-    {
-        "id": "106",
-        "score": "0.182"
-    },
-    {
-        "id": "108",
-        "score": "0.385"
-    },
-    {
-        "id": "59",
-        "score": "0.148"
-    },
-    {
-        "id": "60",
-        "score": "0.476"
-    },
-    {
-        "id": "64",
-        "score": "0.08"
-    },
-    {
-        "id": "65",
-        "score": "0.125"
-    },
-    {
-        "id": "70",
-        "score": "0.118"
-    },
-    {
-        "id": "72",
-        "score": "0.148"
-    },
-    {
-        "id": "73",
-        "score": "0.167"
-    },
-    {
-        "id": "76",
-        "score": "0.133"
-    },
-    {
-        "id": "77",
-        "score": "0.2"
-    },
-    {
-        "id": "78",
-        "score": "0.471"
-    },
-    {
-        "id": "82",
-        "score": "0.171"
-    },
-    {
-        "id": "90",
-        "score": "0.308"
-    },
-    {
-        "id": "95",
-        "score": "0.0"
-    },
-    {
-        "id": "98",
-        "score": "0.133"
-    },
-    {
-        "id": "105",
-        "score": "0.333"
-    },
-    {
-        "id": "119",
-        "score": "0.167"
-    },
-    {
-        "id": "127",
-        "score": "0.385"
-    },
-    {
-        "id": "130",
-        "score": "0.111"
-    },
-    {
-        "id": "131",
-        "score": "0.333"
-    },
-    {
-        "id": "135",
-        "score": "0.519"
-    },
-    {
-        "id": "154",
-        "score": "0.2"
-    },
-    {
-        "id": "155",
-        "score": "0.387"
-    },
-    {
-        "id": "103",
-        "score": "0.057"
-    },
-    {
-        "id": "110",
-        "score": "0.148"
-    },
-    {
-        "id": "112",
-        "score": "0.08"
-    },
-    {
-        "id": "115",
-        "score": "0.16"
-    },
-    {
-        "id": "117",
-        "score": "0.091"
-    },
-    {
-        "id": "120",
-        "score": "0.138"
-    },
-    {
-        "id": "122",
-        "score": "0.091"
-    },
-    {
-        "id": "123",
-        "score": "0.074"
-    },
-    {
-        "id": "126",
-        "score": "0.222"
-    },
-    {
-        "id": "137",
-        "score": "0.194"
-    },
-    {
-        "id": "139",
-        "score": "0.25"
-    },
-    {
-        "id": "143",
-        "score": "0.167"
-    },
-    {
-        "id": "147",
-        "score": "0.148"
-    },
-    {
-        "id": "148",
-        "score": "0.069"
-    },
-    {
-        "id": "149",
-        "score": "0.174"
-    },
-    {
-        "id": "114",
-        "score": "0.32"
-    },
-    {
-        "id": "118",
-        "score": "0.4"
-    },
-    {
-        "id": "125",
-        "score": "0.286"
-    },
-    {
-        "id": "128",
-        "score": "0.385"
-    },
-    {
-        "id": "132",
-        "score": "0.4"
-    },
-    {
-        "id": "136",
-        "score": "0.258"
-    },
-    {
-        "id": "141",
-        "score": "0.696"
-    },
-    {
-        "id": "145",
-        "score": "0.333"
-    },
-    {
-        "id": "146",
-        "score": "0.348"
-    },
-    {
-        "id": "150",
-        "score": "0.286"
-    },
-    {
-        "id": "152",
-        "score": "0.364"
-    },
-    {
-        "id": "153",
-        "score": "0.308"
-    },
-    {
-        "id": "158",
-        "score": "0.533"
-    },
-    {
-        "id": "99",
-        "score": "0.303"
-    },
-    {
-        "id": "102",
-        "score": "0.333"
-    },
-    {
-        "id": "104",
-        "score": "0.16"
-    },
-    {
-        "id": "107",
-        "score": "0.286"
-    },
-    {
-        "id": "109",
-        "score": "0.19"
-    },
-    {
-        "id": "111",
-        "score": "0.353"
-    },
-    {
-        "id": "113",
-        "score": "0.182"
-    },
-    {
-        "id": "116",
-        "score": "0.08"
-    },
-    {
-        "id": "121",
-        "score": "0.182"
-    },
-    {
-        "id": "124",
-        "score": "0.19"
-    },
-    {
-        "id": "129",
-        "score": "0.0"
-    },
-    {
-        "id": "133",
-        "score": "0.4"
-    },
-    {
-        "id": "134",
-        "score": "0.0"
-    },
-    {
-        "id": "156",
-        "score": "0.24"
-    },
-    {
-        "id": "159",
-        "score": "0.545"
-    },
-    {
-        "id": "163",
-        "score": "0.545"
-    },
-    {
-        "id": "170",
-        "score": "0.48"
-    },
-    {
-        "id": "175",
-        "score": "0.556"
-    },
-    {
-        "id": "176",
-        "score": "0.455"
-    },
-    {
-        "id": "184",
-        "score": "0.353"
-    },
-    {
-        "id": "189",
-        "score": "0.195"
-    },
-    {
-        "id": "193",
-        "score": "0.348"
-    },
-    {
-        "id": "151",
-        "score": "0.231"
-    },
-    {
-        "id": "157",
-        "score": "0.222"
-    },
-    {
-        "id": "164",
-        "score": "0.143"
-    },
-    {
-        "id": "165",
-        "score": "0.071"
-    },
-    {
-        "id": "172",
-        "score": "0.1"
-    },
-    {
-        "id": "177",
-        "score": "0.129"
-    },
-    {
-        "id": "179",
-        "score": "0.24"
-    },
-    {
-        "id": "182",
-        "score": "0.167"
-    },
-    {
-        "id": "183",
-        "score": "0.095"
-    },
-    {
-        "id": "188",
-        "score": "0.182"
-    },
-    {
-        "id": "190",
-        "score": "0.077"
-    },
-    {
-        "id": "194",
-        "score": "0.324"
-    },
-    {
-        "id": "195",
-        "score": "0.16"
-    },
-    {
-        "id": "196",
-        "score": "0.111"
-    },
-    {
-        "id": "198",
-        "score": "0.267"
-    },
-    {
-        "id": "160",
-        "score": "0.476"
-    },
-    {
-        "id": "166",
-        "score": "0.348"
-    },
-    {
-        "id": "169",
-        "score": "0.444"
-    },
-    {
-        "id": "171",
-        "score": "0.48"
-    },
-    {
-        "id": "173",
-        "score": "0.267"
-    },
-    {
-        "id": "174",
-        "score": "0.37"
-    },
-    {
-        "id": "178",
-        "score": "0.455"
-    },
-    {
-        "id": "181",
-        "score": "0.333"
-    },
-    {
-        "id": "185",
-        "score": "0.222"
-    },
-    {
-        "id": "187",
-        "score": "0.16"
-    },
-    {
-        "id": "192",
-        "score": "0.462"
-    },
-    {
-        "id": "199",
-        "score": "0.375"
-    },
-    {
-        "id": "138",
-        "score": "0.125"
-    },
-    {
-        "id": "140",
-        "score": "0.231"
-    },
-    {
-        "id": "142",
-        "score": "0.167"
-    },
-    {
-        "id": "144",
-        "score": "0.414"
-    },
-    {
-        "id": "161",
-        "score": "0.111"
-    },
-    {
-        "id": "162",
-        "score": "0.16"
-    },
-    {
-        "id": "167",
-        "score": "0.133"
-    },
-    {
-        "id": "168",
-        "score": "0.235"
-    },
-    {
-        "id": "180",
-        "score": "0.296"
-    },
-    {
-        "id": "186",
-        "score": "0.4"
-    },
-    {
-        "id": "191",
-        "score": "0.19"
-    },
-    {
-        "id": "197",
-        "score": "0.158"
-    }
-]

eval_milebench/ActionLocalization.log

@@ -1,296 +0,0 @@
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionLocalization, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting ActionLocalization Using internvl
-Proceeding 30-length images samples | Num: 40
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionLocalization, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Initialization Finished
-Predicting ActionLocalization Using internvl
-Proceeding 30-length images samples | Num: 40
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionLocalization, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Initialization Finished
-Predicting ActionLocalization Using internvl
-Proceeding 30-length images samples | Num: 40
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionLocalization, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Initialization Finished
-Predicting ActionLocalization Using internvl
-Proceeding 30-length images samples | Num: 40
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:04<00:43,  4.83s/it]
 20%|██        | 2/10 [00:06<00:25,  3.24s/it]
 30%|███       | 3/10 [00:07<00:15,  2.19s/it]
 40%|████      | 4/10 [00:10<00:13,  2.27s/it]
 50%|█████     | 5/10 [00:13<00:13,  2.73s/it]
 60%|██████    | 6/10 [00:17<00:11,  2.99s/it]
 70%|███████   | 7/10 [00:19<00:08,  2.73s/it]
 80%|████████  | 8/10 [00:21<00:04,  2.39s/it]
 90%|█████████ | 9/10 [00:22<00:01,  1.99s/it]
-Proceeding 31-length images samples | Num: 24
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:04<00:43,  4.85s/it]
 20%|██        | 2/10 [00:07<00:28,  3.56s/it]
 30%|███       | 3/10 [00:09<00:21,  3.06s/it]
 40%|████      | 4/10 [00:13<00:18,  3.10s/it]
 50%|█████     | 5/10 [00:14<00:11,  2.35s/it]
 60%|██████    | 6/10 [00:17<00:10,  2.59s/it]
 70%|███████   | 7/10 [00:19<00:07,  2.58s/it]
 80%|████████  | 8/10 [00:21<00:04,  2.31s/it]
 90%|█████████ | 9/10 [00:22<00:01,  1.98s/it]
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:04<00:44,  4.94s/it]
 20%|██        | 2/10 [00:07<00:28,  3.60s/it]
 30%|███       | 3/10 [00:10<00:21,  3.12s/it]
 40%|████      | 4/10 [00:13<00:19,  3.30s/it]
 50%|█████     | 5/10 [00:16<00:16,  3.25s/it]
 60%|██████    | 6/10 [00:17<00:09,  2.43s/it]
 70%|███████   | 7/10 [00:19<00:07,  2.35s/it]
 80%|████████  | 8/10 [00:21<00:04,  2.15s/it]
 90%|█████████ | 9/10 [00:23<00:01,  1.98s/it]
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:16,  3.27s/it]
 33%|███▎      | 2/6 [00:06<00:14,  3.51s/it]
 50%|█████     | 3/6 [00:08<00:08,  2.81s/it]
 67%|██████▋   | 4/6 [00:09<00:04,  2.00s/it]
 83%|████████▎ | 5/6 [00:10<00:01,  1.73s/it]
-Proceeding 42-length images samples | Num: 4
-Proceeding 42-length images samples | Num: 4
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:18,  3.65s/it]
 33%|███▎      | 2/6 [00:07<00:14,  3.50s/it]
 50%|█████     | 3/6 [00:09<00:08,  2.86s/it]
 67%|██████▋   | 4/6 [00:10<00:04,  2.26s/it]
 83%|████████▎ | 5/6 [00:13<00:02,  2.36s/it]
-Proceeding 42-length images samples | Num: 4
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:18,  3.74s/it]
 33%|███▎      | 2/6 [00:07<00:14,  3.53s/it]
 50%|█████     | 3/6 [00:09<00:08,  2.86s/it]
 67%|██████▋   | 4/6 [00:10<00:04,  2.27s/it]
 83%|████████▎ | 5/6 [00:13<00:02,  2.52s/it]
-Proceeding 42-length images samples | Num: 4
-Proceeding 33-length images samples | Num: 12
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 12
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 12
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 12
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.29s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.72s/it]
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.50s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.76s/it]
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.76s/it]
 67%|██████▋   | 2/3 [00:04<00:01,  1.94s/it]
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 10
-Proceeding 37-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 10
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 10
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.88s/it]
-Proceeding 58-length images samples | Num: 2
-Proceeding 35-length images samples | Num: 10
-
-Proceeding 39-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.64s/it]
-Proceeding 58-length images samples | Num: 2
-
-Proceeding 39-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:03<00:07,  3.82s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.26s/it]
-Proceeding 58-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 40-length images samples | Num: 3
-
-Proceeding 43-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 40-length images samples | Num: 3
-
-Proceeding 36-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 5
-Proceeding 58-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 40-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 43-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.22s/it]
-Proceeding 48-length images samples | Num: 1
-Proceeding 39-length images samples | Num: 5
-
-Proceeding 44-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 43-length images samples | Num: 3
-
-Proceeding 46-length images samples | Num: 1
-
-Proceeding 38-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 36-length images samples | Num: 9
-
-Proceeding 34-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 36-length images samples | Num: 9
-Proceeding 40-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.92s/it]
-Proceeding 48-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.95s/it]
-Proceeding 50-length images samples | Num: 1
-
-Proceeding 44-length images samples | Num: 2
-
-Proceeding 32-length images samples | Num: 17
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.50s/it]
-Proceeding 48-length images samples | Num: 1
-
-Proceeding 46-length images samples | Num: 1
-Proceeding 43-length images samples | Num: 3
-
-Proceeding 44-length images samples | Num: 2
-
-Proceeding 38-length images samples | Num: 3
-Proceeding 36-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 34-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 46-length images samples | Num: 1
-
-Proceeding 38-length images samples | Num: 3
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:01<00:03,  1.21s/it]
 50%|█████     | 2/4 [00:05<00:05,  2.91s/it]
 75%|███████▌  | 3/4 [00:06<00:01,  1.96s/it]
-Proceeding 26-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 34-length images samples | Num: 9
-
-Proceeding 27-length images samples | Num: 4
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.63s/it]
-Proceeding 50-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 2
-Proceeding 48-length images samples | Num: 1
-
-Proceeding 32-length images samples | Num: 17
-
-Proceeding 45-length images samples | Num: 2
-
-Proceeding 29-length images samples | Num: 10
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.64s/it]
-Proceeding 50-length images samples | Num: 1
-
-Proceeding 32-length images samples | Num: 17
-Proceeding 44-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.84s/it]
-Proceeding 28-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 8
-Proceeding 46-length images samples | Num: 1
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:03<00:09,  3.22s/it]
 50%|█████     | 2/4 [00:04<00:03,  1.82s/it]
 75%|███████▌  | 3/4 [00:06<00:02,  2.08s/it]
-Proceeding 26-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 27-length images samples | Num: 4
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:00<00:00,  1.04it/s]
-Proceeding 22-length images samples | Num: 1
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:01<00:04,  1.59s/it]
 50%|█████     | 2/4 [00:04<00:05,  2.55s/it]
 75%|███████▌  | 3/4 [00:05<00:01,  1.78s/it]
-Proceeding 26-length images samples | Num: 3
-
-Proceeding 21-length images samples | Num: 5
-Proceeding 38-length images samples | Num: 3
-Proceeding 20-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-
-Proceeding 45-length images samples | Num: 2
-
-Proceeding 29-length images samples | Num: 10
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 27-length images samples | Num: 4
-
-Proceeding 17-length images samples | Num: 1
-Proceeding 34-length images samples | Num: 9
-
-Proceeding 24-length images samples | Num: 1
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.80s/it]
-Proceeding 28-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 29-length images samples | Num: 10
-Proceeding 50-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.12s/it]
-Proceeding 22-length images samples | Num: 1
-
-Proceeding 21-length images samples | Num: 5
-Proceeding 32-length images samples | Num: 17
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 1
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:02,  1.17s/it]
 67%|██████▋   | 2/3 [00:02<00:01,  1.09s/it]
-Proceeding 28-length images samples | Num: 6
-
-Proceeding 17-length images samples | Num: 1
-
-Proceeding 24-length images samples | Num: 1
-
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.09s/it]
-Proceeding 23-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.25s/it]
-Proceeding 22-length images samples | Num: 1
-
-Proceeding 21-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 1
-
-Proceeding 17-length images samples | Num: 1
-Proceeding 26-length images samples | Num: 3
-
-Proceeding 24-length images samples | Num: 1
-
-Proceeding 27-length images samples | Num: 4
-Proceeding 20-length images samples | Num: 2
-Proceeding 45-length images samples | Num: 2
-Proceeding 29-length images samples | Num: 10
-Proceeding 28-length images samples | Num: 6
-Proceeding 23-length images samples | Num: 8
-Proceeding 22-length images samples | Num: 1
-Proceeding 21-length images samples | Num: 5
-Proceeding 19-length images samples | Num: 1
-Proceeding 17-length images samples | Num: 1
-Proceeding 24-length images samples | Num: 1
-evaluating ActionLocalization ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionLocalization/ActionLocalization_240803234615.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset ActionLocalization                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionLocalization 
-internvl:  ActionLocalization:  {'Accuracy': 0.255, 'image_quantity_level-Accuracy': {'Few': 0, 'Medium': 0.330188679245283, 'Many': 0.1702127659574468}, 'image_quantity_level-Result': {'Few': [0, 0], 'Medium': [35, 106], 'Many': [16, 94]}}

eval_milebench/ActionLocalization/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.255, "image_quantity_level-Accuracy": {"Few": 0, "Medium": 0.330188679245283, "Many": 0.1702127659574468}, "image_quantity_level-Result": {"Few": [0, 0], "Medium": [35, 106], "Many": [16, 94]}}

eval_milebench/ActionLocalization/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "0"
-    },
-    {
-        "id": "19",
-        "score": "0"
-    },
-    {
-        "id": "37",
-        "score": "0"
-    },
-    {
-        "id": "39",
-        "score": "0"
-    },
-    {
-        "id": "55",
-        "score": "0"
-    },
-    {
-        "id": "61",
-        "score": "0"
-    },
-    {
-        "id": "62",
-        "score": "0"
-    },
-    {
-        "id": "67",
-        "score": "0"
-    },
-    {
-        "id": "68",
-        "score": "0"
-    },
-    {
-        "id": "69",
-        "score": "0"
-    },
-    {
-        "id": "1",
-        "score": "0"
-    },
-    {
-        "id": "2",
-        "score": "0"
-    },
-    {
-        "id": "21",
-        "score": "0"
-    },
-    {
-        "id": "35",
-        "score": "0"
-    },
-    {
-        "id": "38",
-        "score": "0"
-    },
-    {
-        "id": "60",
-        "score": "0"
-    },
-    {
-        "id": "3",
-        "score": "0"
-    },
-    {
-        "id": "4",
-        "score": "0"
-    },
-    {
-        "id": "29",
-        "score": "0"
-    },
-    {
-        "id": "31",
-        "score": "0"
-    },
-    {
-        "id": "5",
-        "score": "0"
-    },
-    {
-        "id": "6",
-        "score": "0"
-    },
-    {
-        "id": "7",
-        "score": "0"
-    },
-    {
-        "id": "8",
-        "score": "0"
-    },
-    {
-        "id": "10",
-        "score": "0"
-    },
-    {
-        "id": "18",
-        "score": "0"
-    },
-    {
-        "id": "27",
-        "score": "0"
-    },
-    {
-        "id": "11",
-        "score": "0"
-    },
-    {
-        "id": "12",
-        "score": "0"
-    },
-    {
-        "id": "22",
-        "score": "0"
-    },
-    {
-        "id": "13",
-        "score": "0"
-    },
-    {
-        "id": "14",
-        "score": "0"
-    },
-    {
-        "id": "15",
-        "score": "0"
-    },
-    {
-        "id": "26",
-        "score": "0"
-    },
-    {
-        "id": "36",
-        "score": "0"
-    },
-    {
-        "id": "16",
-        "score": "0"
-    },
-    {
-        "id": "20",
-        "score": "0"
-    },
-    {
-        "id": "23",
-        "score": "0"
-    },
-    {
-        "id": "30",
-        "score": "0"
-    },
-    {
-        "id": "33",
-        "score": "0"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "63",
-        "score": "1"
-    },
-    {
-        "id": "34",
-        "score": "0"
-    },
-    {
-        "id": "42",
-        "score": "0"
-    },
-    {
-        "id": "43",
-        "score": "0"
-    },
-    {
-        "id": "59",
-        "score": "0"
-    },
-    {
-        "id": "65",
-        "score": "0"
-    },
-    {
-        "id": "72",
-        "score": "0"
-    },
-    {
-        "id": "50",
-        "score": "0"
-    },
-    {
-        "id": "64",
-        "score": "0"
-    },
-    {
-        "id": "66",
-        "score": "0"
-    },
-    {
-        "id": "71",
-        "score": "0"
-    },
-    {
-        "id": "73",
-        "score": "0"
-    },
-    {
-        "id": "89",
-        "score": "0"
-    },
-    {
-        "id": "108",
-        "score": "0"
-    },
-    {
-        "id": "80",
-        "score": "0"
-    },
-    {
-        "id": "85",
-        "score": "0"
-    },
-    {
-        "id": "88",
-        "score": "0"
-    },
-    {
-        "id": "99",
-        "score": "0"
-    },
-    {
-        "id": "94",
-        "score": "0"
-    },
-    {
-        "id": "111",
-        "score": "0"
-    },
-    {
-        "id": "123",
-        "score": "1"
-    },
-    {
-        "id": "153",
-        "score": "0"
-    },
-    {
-        "id": "165",
-        "score": "0"
-    },
-    {
-        "id": "181",
-        "score": "0"
-    },
-    {
-        "id": "76",
-        "score": "0"
-    },
-    {
-        "id": "77",
-        "score": "0"
-    },
-    {
-        "id": "87",
-        "score": "0"
-    },
-    {
-        "id": "90",
-        "score": "0"
-    },
-    {
-        "id": "97",
-        "score": "0"
-    },
-    {
-        "id": "98",
-        "score": "0"
-    },
-    {
-        "id": "104",
-        "score": "1"
-    },
-    {
-        "id": "110",
-        "score": "1"
-    },
-    {
-        "id": "112",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "1"
-    },
-    {
-        "id": "84",
-        "score": "0"
-    },
-    {
-        "id": "92",
-        "score": "0"
-    },
-    {
-        "id": "100",
-        "score": "0"
-    },
-    {
-        "id": "129",
-        "score": "1"
-    },
-    {
-        "id": "130",
-        "score": "0"
-    },
-    {
-        "id": "131",
-        "score": "1"
-    },
-    {
-        "id": "44",
-        "score": "0"
-    },
-    {
-        "id": "41",
-        "score": "0"
-    },
-    {
-        "id": "53",
-        "score": "0"
-    },
-    {
-        "id": "86",
-        "score": "0"
-    },
-    {
-        "id": "25",
-        "score": "0"
-    },
-    {
-        "id": "9",
-        "score": "0"
-    },
-    {
-        "id": "49",
-        "score": "0"
-    },
-    {
-        "id": "51",
-        "score": "0"
-    },
-    {
-        "id": "54",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "1"
-    },
-    {
-        "id": "28",
-        "score": "0"
-    },
-    {
-        "id": "32",
-        "score": "0"
-    },
-    {
-        "id": "17",
-        "score": "0"
-    },
-    {
-        "id": "47",
-        "score": "0"
-    },
-    {
-        "id": "58",
-        "score": "0"
-    },
-    {
-        "id": "40",
-        "score": "0"
-    },
-    {
-        "id": "56",
-        "score": "0"
-    },
-    {
-        "id": "75",
-        "score": "0"
-    },
-    {
-        "id": "83",
-        "score": "0"
-    },
-    {
-        "id": "82",
-        "score": "0"
-    },
-    {
-        "id": "102",
-        "score": "0"
-    },
-    {
-        "id": "113",
-        "score": "0"
-    },
-    {
-        "id": "121",
-        "score": "0"
-    },
-    {
-        "id": "126",
-        "score": "1"
-    },
-    {
-        "id": "78",
-        "score": "0"
-    },
-    {
-        "id": "154",
-        "score": "0"
-    },
-    {
-        "id": "81",
-        "score": "0"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "145",
-        "score": "0"
-    },
-    {
-        "id": "149",
-        "score": "0"
-    },
-    {
-        "id": "95",
-        "score": "0"
-    },
-    {
-        "id": "128",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "0"
-    },
-    {
-        "id": "105",
-        "score": "0"
-    },
-    {
-        "id": "148",
-        "score": "1"
-    },
-    {
-        "id": "116",
-        "score": "0"
-    },
-    {
-        "id": "118",
-        "score": "1"
-    },
-    {
-        "id": "120",
-        "score": "1"
-    },
-    {
-        "id": "122",
-        "score": "1"
-    },
-    {
-        "id": "125",
-        "score": "1"
-    },
-    {
-        "id": "127",
-        "score": "0"
-    },
-    {
-        "id": "140",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "0"
-    },
-    {
-        "id": "146",
-        "score": "0"
-    },
-    {
-        "id": "147",
-        "score": "1"
-    },
-    {
-        "id": "132",
-        "score": "1"
-    },
-    {
-        "id": "135",
-        "score": "0"
-    },
-    {
-        "id": "137",
-        "score": "0"
-    },
-    {
-        "id": "151",
-        "score": "1"
-    },
-    {
-        "id": "160",
-        "score": "1"
-    },
-    {
-        "id": "161",
-        "score": "0"
-    },
-    {
-        "id": "52",
-        "score": "0"
-    },
-    {
-        "id": "93",
-        "score": "0"
-    },
-    {
-        "id": "114",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "0"
-    },
-    {
-        "id": "46",
-        "score": "0"
-    },
-    {
-        "id": "45",
-        "score": "0"
-    },
-    {
-        "id": "57",
-        "score": "0"
-    },
-    {
-        "id": "106",
-        "score": "0"
-    },
-    {
-        "id": "142",
-        "score": "0"
-    },
-    {
-        "id": "124",
-        "score": "1"
-    },
-    {
-        "id": "24",
-        "score": "0"
-    },
-    {
-        "id": "74",
-        "score": "0"
-    },
-    {
-        "id": "133",
-        "score": "0"
-    },
-    {
-        "id": "70",
-        "score": "0"
-    },
-    {
-        "id": "91",
-        "score": "0"
-    },
-    {
-        "id": "107",
-        "score": "0"
-    },
-    {
-        "id": "138",
-        "score": "1"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "152",
-        "score": "1"
-    },
-    {
-        "id": "168",
-        "score": "0"
-    },
-    {
-        "id": "169",
-        "score": "0"
-    },
-    {
-        "id": "134",
-        "score": "1"
-    },
-    {
-        "id": "159",
-        "score": "0"
-    },
-    {
-        "id": "174",
-        "score": "1"
-    },
-    {
-        "id": "173",
-        "score": "1"
-    },
-    {
-        "id": "136",
-        "score": "0"
-    },
-    {
-        "id": "143",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "0"
-    },
-    {
-        "id": "156",
-        "score": "1"
-    },
-    {
-        "id": "163",
-        "score": "1"
-    },
-    {
-        "id": "167",
-        "score": "0"
-    },
-    {
-        "id": "175",
-        "score": "0"
-    },
-    {
-        "id": "176",
-        "score": "1"
-    },
-    {
-        "id": "179",
-        "score": "0"
-    },
-    {
-        "id": "183",
-        "score": "1"
-    },
-    {
-        "id": "189",
-        "score": "0"
-    },
-    {
-        "id": "191",
-        "score": "0"
-    },
-    {
-        "id": "193",
-        "score": "0"
-    },
-    {
-        "id": "170",
-        "score": "1"
-    },
-    {
-        "id": "177",
-        "score": "1"
-    },
-    {
-        "id": "184",
-        "score": "0"
-    },
-    {
-        "id": "187",
-        "score": "0"
-    },
-    {
-        "id": "197",
-        "score": "1"
-    },
-    {
-        "id": "198",
-        "score": "0"
-    },
-    {
-        "id": "155",
-        "score": "0"
-    },
-    {
-        "id": "164",
-        "score": "0"
-    },
-    {
-        "id": "166",
-        "score": "0"
-    },
-    {
-        "id": "192",
-        "score": "1"
-    },
-    {
-        "id": "101",
-        "score": "0"
-    },
-    {
-        "id": "96",
-        "score": "0"
-    },
-    {
-        "id": "117",
-        "score": "1"
-    },
-    {
-        "id": "180",
-        "score": "1"
-    },
-    {
-        "id": "190",
-        "score": "1"
-    },
-    {
-        "id": "144",
-        "score": "0"
-    },
-    {
-        "id": "157",
-        "score": "0"
-    },
-    {
-        "id": "109",
-        "score": "0"
-    },
-    {
-        "id": "119",
-        "score": "1"
-    },
-    {
-        "id": "178",
-        "score": "1"
-    },
-    {
-        "id": "186",
-        "score": "1"
-    },
-    {
-        "id": "194",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "1"
-    },
-    {
-        "id": "162",
-        "score": "0"
-    },
-    {
-        "id": "182",
-        "score": "1"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "188",
-        "score": "0"
-    },
-    {
-        "id": "195",
-        "score": "0"
-    },
-    {
-        "id": "196",
-        "score": "1"
-    },
-    {
-        "id": "172",
-        "score": "1"
-    }
-]

eval_milebench/ActionPrediction.log

@@ -1,366 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionPrediction, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionPrediction, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionPrediction, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Initialization Finished
-Predicting ActionPrediction Using internvl
-Proceeding 36-length images samples | Num: 5
-Initialization Finished
-Predicting ActionPrediction Using internvl
-Proceeding 36-length images samples | Num: 5
-Initialization Finished
-Predicting ActionPrediction Using internvl
-Proceeding 36-length images samples | Num: 5
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionPrediction, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting ActionPrediction Using internvl
-Proceeding 36-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 28-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 28-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 28-length images samples | Num: 9
-Proceeding 28-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.09s/it]
-Proceeding 29-length images samples | Num: 13
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.25s/it]
-Proceeding 29-length images samples | Num: 13
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.30s/it]
-Proceeding 29-length images samples | Num: 13
-Proceeding 29-length images samples | Num: 13
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:02,  1.02s/it]
 67%|██████▋   | 2/3 [00:02<00:01,  1.46s/it]
-Proceeding 32-length images samples | Num: 15
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.05s/it]
 67%|██████▋   | 2/3 [00:02<00:01,  1.32s/it]
-Proceeding 32-length images samples | Num: 15
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.42s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.50s/it]
-Proceeding 32-length images samples | Num: 15
-Proceeding 32-length images samples | Num: 15
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:03<00:06,  3.17s/it]
 67%|██████▋   | 2/3 [00:05<00:02,  2.79s/it]
-Proceeding 31-length images samples | Num: 26
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:03<00:10,  3.63s/it]
 50%|█████     | 2/4 [00:06<00:05,  2.99s/it]
 75%|███████▌  | 3/4 [00:06<00:01,  1.99s/it]
-Proceeding 31-length images samples | Num: 26
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:02<00:08,  2.83s/it]
 50%|█████     | 2/4 [00:05<00:05,  2.80s/it]
 75%|███████▌  | 3/4 [00:06<00:01,  1.96s/it]
-Proceeding 31-length images samples | Num: 26
-Proceeding 31-length images samples | Num: 26
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:01<00:06,  1.20s/it]
 33%|███▎      | 2/6 [00:02<00:05,  1.39s/it]
 50%|█████     | 3/6 [00:03<00:03,  1.12s/it]
 67%|██████▋   | 4/6 [00:04<00:02,  1.19s/it]
 83%|████████▎ | 5/6 [00:06<00:01,  1.39s/it]
-Proceeding 34-length images samples | Num: 9
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:02<00:10,  2.03s/it]
 33%|███▎      | 2/6 [00:02<00:05,  1.34s/it]
 50%|█████     | 3/6 [00:05<00:05,  1.92s/it]
 67%|██████▋   | 4/6 [00:06<00:03,  1.58s/it]
 83%|████████▎ | 5/6 [00:09<00:01,  1.90s/it]
-Proceeding 34-length images samples | Num: 9
-Proceeding 37-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.00s/it]
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 44-length images samples | Num: 2
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:02<00:17,  2.92s/it]
 29%|██▊       | 2/7 [00:03<00:08,  1.63s/it]
 43%|████▎     | 3/7 [00:06<00:08,  2.21s/it]
 57%|█████▋    | 4/7 [00:09<00:07,  2.45s/it]
 71%|███████▏  | 5/7 [00:10<00:03,  1.84s/it]
 86%|████████▌ | 6/7 [00:13<00:02,  2.25s/it]
-Proceeding 34-length images samples | Num: 9
-
-Proceeding 23-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.73s/it]
-Proceeding 37-length images samples | Num: 6
-Proceeding 34-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 56-length images samples | Num: 1
-
-Proceeding 17-length images samples | Num: 3
-Proceeding 33-length images samples | Num: 8
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 44-length images samples | Num: 2
-
-Proceeding 23-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:04<00:04,  4.15s/it]
-Proceeding 37-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 56-length images samples | Num: 1
-
-Proceeding 17-length images samples | Num: 3
-Proceeding 37-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.37s/it]
-Proceeding 24-length images samples | Num: 3
-
-Proceeding 35-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.49s/it]
-Proceeding 44-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 46
-Proceeding 44-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.06s/it]
-Proceeding 24-length images samples | Num: 3
-Proceeding 23-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.72s/it]
-Proceeding 56-length images samples | Num: 1
-
-Proceeding 17-length images samples | Num: 3
-Proceeding 56-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 46
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 8
-Proceeding 17-length images samples | Num: 3
-Proceeding 33-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.55s/it]
-Proceeding 24-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 5
-Proceeding 24-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 46
-
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:01<00:11,  1.16s/it]
 18%|█▊        | 2/11 [00:02<00:12,  1.36s/it]
 27%|██▋       | 3/11 [00:03<00:08,  1.09s/it]
 36%|███▋      | 4/11 [00:04<00:06,  1.01it/s]
 45%|████▌     | 5/11 [00:06<00:08,  1.46s/it]
 55%|█████▍    | 6/11 [00:08<00:07,  1.46s/it]
 64%|██████▎   | 7/11 [00:10<00:06,  1.71s/it]
 73%|███████▎  | 8/11 [00:10<00:04,  1.40s/it]
 82%|████████▏ | 9/11 [00:13<00:03,  1.84s/it]
 91%|█████████ | 10/11 [00:15<00:01,  1.75s/it]
-Proceeding 21-length images samples | Num: 2
-
-Proceeding 41-length images samples | Num: 1
-Proceeding 59-length images samples | Num: 1
-
-
-Proceeding 22-length images samples | Num: 2
-Proceeding 15-length images samples | Num: 3
-
-
-Proceeding 38-length images samples | Num: 1
-
-Proceeding 25-length images samples | Num: 2
-
-Proceeding 20-length images samples | Num: 1
-Proceeding 35-length images samples | Num: 5
-
-Proceeding 39-length images samples | Num: 1
-Proceeding 16-length images samples | Num: 4
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 43-length images samples | Num: 1
-
-Proceeding 40-length images samples | Num: 2
-Proceeding 27-length images samples | Num: 7
-
-Proceeding 30-length images samples | Num: 46
-
  0%|          | 0/11 [00:00<?, ?it/s]
  9%|▉         | 1/11 [00:03<00:36,  3.65s/it]
 18%|█▊        | 2/11 [00:05<00:23,  2.60s/it]
 27%|██▋       | 3/11 [00:06<00:14,  1.78s/it]
 36%|███▋      | 4/11 [00:08<00:13,  1.99s/it]
 45%|████▌     | 5/11 [00:09<00:10,  1.70s/it]
 55%|█████▍    | 6/11 [00:13<00:11,  2.28s/it]
 64%|██████▎   | 7/11 [00:13<00:07,  1.78s/it]
 73%|███████▎  | 8/11 [00:16<00:05,  1.94s/it]
 82%|████████▏ | 9/11 [00:17<00:03,  1.71s/it]
 91%|█████████ | 10/11 [00:18<00:01,  1.46s/it]
-Proceeding 21-length images samples | Num: 2
-
-Proceeding 41-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 26-length images samples | Num: 1
-
-Proceeding 59-length images samples | Num: 1
-
-Proceeding 81-length images samples | Num: 1
-
-Proceeding 22-length images samples | Num: 2
-
-Proceeding 12-length images samples | Num: 2
-
-Proceeding 46-length images samples | Num: 1
-
-Proceeding 15-length images samples | Num: 3
-
-Proceeding 18-length images samples | Num: 1
-
-Proceeding 60-length images samples | Num: 1
-
-Proceeding 45-length images samples | Num: 2
-
-Proceeding 13-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 38-length images samples | Num: 1
-
-Proceeding 19-length images samples | Num: 1
-
-Proceeding 25-length images samples | Num: 2
-
-Proceeding 9-length images samples | Num: 1
-
-Proceeding 20-length images samples | Num: 1
-
-Proceeding 49-length images samples | Num: 1
-
-Proceeding 39-length images samples | Num: 1
-
-Proceeding 16-length images samples | Num: 4
-
-Proceeding 54-length images samples | Num: 1
-
-Proceeding 7-length images samples | Num: 1
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 43-length images samples | Num: 1
-Proceeding 40-length images samples | Num: 2
-
-
-Proceeding 27-length images samples | Num: 7
-
  0%|          | 0/12 [00:00<?, ?it/s]
  8%|▊         | 1/12 [00:02<00:27,  2.54s/it]
 17%|█▋        | 2/12 [00:03<00:17,  1.76s/it]
 25%|██▌       | 3/12 [00:04<00:12,  1.35s/it]
 33%|███▎      | 4/12 [00:05<00:09,  1.17s/it]
 42%|████▏     | 5/12 [00:06<00:07,  1.06s/it]
 50%|█████     | 6/12 [00:07<00:05,  1.03it/s]
 58%|█████▊    | 7/12 [00:08<00:05,  1.08s/it]
 67%|██████▋   | 8/12 [00:09<00:03,  1.03it/s]
 75%|███████▌  | 9/12 [00:09<00:02,  1.14it/s]
 83%|████████▎ | 10/12 [00:11<00:02,  1.15s/it]
 92%|█████████▏| 11/12 [00:12<00:01,  1.06s/it]
-Proceeding 21-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.94s/it]
-Proceeding 26-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 41-length images samples | Num: 1
-
-Proceeding 81-length images samples | Num: 1
-
-Proceeding 59-length images samples | Num: 1
-
-Proceeding 12-length images samples | Num: 2
-
-Proceeding 22-length images samples | Num: 2
-
-Proceeding 46-length images samples | Num: 1
-
-Proceeding 18-length images samples | Num: 1
-
-Proceeding 60-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 15-length images samples | Num: 3
-
-Proceeding 45-length images samples | Num: 2
-Proceeding 21-length images samples | Num: 2
-
-Proceeding 13-length images samples | Num: 1
-
-Proceeding 19-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 38-length images samples | Num: 1
-
-Proceeding 9-length images samples | Num: 1
-
-Proceeding 25-length images samples | Num: 2
-Proceeding 41-length images samples | Num: 1
-
-Proceeding 49-length images samples | Num: 1
-
-Proceeding 54-length images samples | Num: 1
-
-Proceeding 7-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 1
-
-
-Proceeding 39-length images samples | Num: 1
-Proceeding 59-length images samples | Num: 1
-
-Proceeding 16-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 43-length images samples | Num: 1
-
-Proceeding 40-length images samples | Num: 2
-Proceeding 22-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 27-length images samples | Num: 7
-Proceeding 15-length images samples | Num: 3
-Proceeding 38-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.07s/it]
-Proceeding 26-length images samples | Num: 1
-
-Proceeding 81-length images samples | Num: 1
-
-Proceeding 12-length images samples | Num: 2
-Proceeding 25-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 46-length images samples | Num: 1
-
-Proceeding 18-length images samples | Num: 1
-Proceeding 20-length images samples | Num: 1
-
-Proceeding 60-length images samples | Num: 1
-
-Proceeding 45-length images samples | Num: 2
-Proceeding 39-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 13-length images samples | Num: 1
-
-Proceeding 19-length images samples | Num: 1
-Proceeding 16-length images samples | Num: 4
-
-Proceeding 9-length images samples | Num: 1
-
-Proceeding 49-length images samples | Num: 1
-
-Proceeding 54-length images samples | Num: 1
-Proceeding 43-length images samples | Num: 1
-
-Proceeding 7-length images samples | Num: 1
-
-Proceeding 40-length images samples | Num: 2
-Proceeding 27-length images samples | Num: 7
-Proceeding 26-length images samples | Num: 1
-Proceeding 81-length images samples | Num: 1
-Proceeding 12-length images samples | Num: 2
-Proceeding 46-length images samples | Num: 1
-Proceeding 18-length images samples | Num: 1
-Proceeding 60-length images samples | Num: 1
-Proceeding 45-length images samples | Num: 2
-Proceeding 13-length images samples | Num: 1
-Proceeding 19-length images samples | Num: 1
-Proceeding 9-length images samples | Num: 1
-Proceeding 49-length images samples | Num: 1
-Proceeding 54-length images samples | Num: 1
-Proceeding 7-length images samples | Num: 1
-evaluating ActionPrediction ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionPrediction/ActionPrediction_240803234615.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset ActionPrediction                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionPrediction 
-internvl:  ActionPrediction:  {'Accuracy': 0.755, 'image_quantity_level-Accuracy': {'Few': 0, 'Medium': 0.7555555555555555, 'Many': 0.7538461538461538}, 'image_quantity_level-Result': {'Few': [0, 0], 'Medium': [102, 135], 'Many': [49, 65]}}

eval_milebench/ActionPrediction/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.755, "image_quantity_level-Accuracy": {"Few": 0, "Medium": 0.7555555555555555, "Many": 0.7538461538461538}, "image_quantity_level-Result": {"Few": [0, 0], "Medium": [102, 135], "Many": [49, 65]}}

eval_milebench/ActionPrediction/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "1"
-    },
-    {
-        "id": "56",
-        "score": "1"
-    },
-    {
-        "id": "1",
-        "score": "1"
-    },
-    {
-        "id": "16",
-        "score": "0"
-    },
-    {
-        "id": "33",
-        "score": "1"
-    },
-    {
-        "id": "2",
-        "score": "1"
-    },
-    {
-        "id": "23",
-        "score": "1"
-    },
-    {
-        "id": "25",
-        "score": "1"
-    },
-    {
-        "id": "28",
-        "score": "1"
-    },
-    {
-        "id": "3",
-        "score": "1"
-    },
-    {
-        "id": "18",
-        "score": "0"
-    },
-    {
-        "id": "37",
-        "score": "1"
-    },
-    {
-        "id": "42",
-        "score": "1"
-    },
-    {
-        "id": "4",
-        "score": "1"
-    },
-    {
-        "id": "14",
-        "score": "1"
-    },
-    {
-        "id": "21",
-        "score": "1"
-    },
-    {
-        "id": "38",
-        "score": "1"
-    },
-    {
-        "id": "50",
-        "score": "1"
-    },
-    {
-        "id": "57",
-        "score": "1"
-    },
-    {
-        "id": "75",
-        "score": "0"
-    },
-    {
-        "id": "5",
-        "score": "1"
-    },
-    {
-        "id": "43",
-        "score": "1"
-    },
-    {
-        "id": "60",
-        "score": "1"
-    },
-    {
-        "id": "6",
-        "score": "0"
-    },
-    {
-        "id": "29",
-        "score": "1"
-    },
-    {
-        "id": "7",
-        "score": "1"
-    },
-    {
-        "id": "8",
-        "score": "1"
-    },
-    {
-        "id": "47",
-        "score": "1"
-    },
-    {
-        "id": "9",
-        "score": "1"
-    },
-    {
-        "id": "10",
-        "score": "1"
-    },
-    {
-        "id": "11",
-        "score": "1"
-    },
-    {
-        "id": "12",
-        "score": "1"
-    },
-    {
-        "id": "13",
-        "score": "0"
-    },
-    {
-        "id": "15",
-        "score": "0"
-    },
-    {
-        "id": "85",
-        "score": "0"
-    },
-    {
-        "id": "17",
-        "score": "0"
-    },
-    {
-        "id": "22",
-        "score": "1"
-    },
-    {
-        "id": "24",
-        "score": "1"
-    },
-    {
-        "id": "26",
-        "score": "0"
-    },
-    {
-        "id": "31",
-        "score": "1"
-    },
-    {
-        "id": "34",
-        "score": "1"
-    },
-    {
-        "id": "35",
-        "score": "1"
-    },
-    {
-        "id": "51",
-        "score": "0"
-    },
-    {
-        "id": "58",
-        "score": "1"
-    },
-    {
-        "id": "65",
-        "score": "1"
-    },
-    {
-        "id": "67",
-        "score": "1"
-    },
-    {
-        "id": "68",
-        "score": "1"
-    },
-    {
-        "id": "20",
-        "score": "0"
-    },
-    {
-        "id": "27",
-        "score": "1"
-    },
-    {
-        "id": "32",
-        "score": "1"
-    },
-    {
-        "id": "36",
-        "score": "1"
-    },
-    {
-        "id": "41",
-        "score": "1"
-    },
-    {
-        "id": "44",
-        "score": "0"
-    },
-    {
-        "id": "46",
-        "score": "1"
-    },
-    {
-        "id": "49",
-        "score": "1"
-    },
-    {
-        "id": "52",
-        "score": "1"
-    },
-    {
-        "id": "53",
-        "score": "1"
-    },
-    {
-        "id": "59",
-        "score": "0"
-    },
-    {
-        "id": "61",
-        "score": "1"
-    },
-    {
-        "id": "70",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "1"
-    },
-    {
-        "id": "82",
-        "score": "1"
-    },
-    {
-        "id": "89",
-        "score": "1"
-    },
-    {
-        "id": "98",
-        "score": "1"
-    },
-    {
-        "id": "111",
-        "score": "1"
-    },
-    {
-        "id": "133",
-        "score": "1"
-    },
-    {
-        "id": "134",
-        "score": "1"
-    },
-    {
-        "id": "138",
-        "score": "0"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "1"
-    },
-    {
-        "id": "164",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "0"
-    },
-    {
-        "id": "174",
-        "score": "1"
-    },
-    {
-        "id": "194",
-        "score": "0"
-    },
-    {
-        "id": "62",
-        "score": "0"
-    },
-    {
-        "id": "40",
-        "score": "1"
-    },
-    {
-        "id": "64",
-        "score": "1"
-    },
-    {
-        "id": "30",
-        "score": "1"
-    },
-    {
-        "id": "63",
-        "score": "1"
-    },
-    {
-        "id": "99",
-        "score": "1"
-    },
-    {
-        "id": "45",
-        "score": "0"
-    },
-    {
-        "id": "55",
-        "score": "1"
-    },
-    {
-        "id": "81",
-        "score": "0"
-    },
-    {
-        "id": "94",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "1"
-    },
-    {
-        "id": "95",
-        "score": "0"
-    },
-    {
-        "id": "104",
-        "score": "1"
-    },
-    {
-        "id": "106",
-        "score": "0"
-    },
-    {
-        "id": "108",
-        "score": "1"
-    },
-    {
-        "id": "116",
-        "score": "1"
-    },
-    {
-        "id": "122",
-        "score": "1"
-    },
-    {
-        "id": "76",
-        "score": "1"
-    },
-    {
-        "id": "84",
-        "score": "1"
-    },
-    {
-        "id": "113",
-        "score": "1"
-    },
-    {
-        "id": "127",
-        "score": "1"
-    },
-    {
-        "id": "72",
-        "score": "0"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "73",
-        "score": "0"
-    },
-    {
-        "id": "120",
-        "score": "1"
-    },
-    {
-        "id": "54",
-        "score": "1"
-    },
-    {
-        "id": "135",
-        "score": "1"
-    },
-    {
-        "id": "19",
-        "score": "1"
-    },
-    {
-        "id": "121",
-        "score": "0"
-    },
-    {
-        "id": "74",
-        "score": "1"
-    },
-    {
-        "id": "77",
-        "score": "1"
-    },
-    {
-        "id": "80",
-        "score": "1"
-    },
-    {
-        "id": "83",
-        "score": "1"
-    },
-    {
-        "id": "88",
-        "score": "0"
-    },
-    {
-        "id": "90",
-        "score": "0"
-    },
-    {
-        "id": "91",
-        "score": "0"
-    },
-    {
-        "id": "93",
-        "score": "0"
-    },
-    {
-        "id": "97",
-        "score": "0"
-    },
-    {
-        "id": "100",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "1"
-    },
-    {
-        "id": "107",
-        "score": "1"
-    },
-    {
-        "id": "39",
-        "score": "1"
-    },
-    {
-        "id": "180",
-        "score": "0"
-    },
-    {
-        "id": "71",
-        "score": "1"
-    },
-    {
-        "id": "145",
-        "score": "1"
-    },
-    {
-        "id": "117",
-        "score": "1"
-    },
-    {
-        "id": "195",
-        "score": "1"
-    },
-    {
-        "id": "149",
-        "score": "0"
-    },
-    {
-        "id": "163",
-        "score": "1"
-    },
-    {
-        "id": "137",
-        "score": "0"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "129",
-        "score": "0"
-    },
-    {
-        "id": "69",
-        "score": "1"
-    },
-    {
-        "id": "87",
-        "score": "1"
-    },
-    {
-        "id": "131",
-        "score": "0"
-    },
-    {
-        "id": "132",
-        "score": "1"
-    },
-    {
-        "id": "155",
-        "score": "1"
-    },
-    {
-        "id": "101",
-        "score": "1"
-    },
-    {
-        "id": "102",
-        "score": "1"
-    },
-    {
-        "id": "112",
-        "score": "1"
-    },
-    {
-        "id": "118",
-        "score": "1"
-    },
-    {
-        "id": "126",
-        "score": "0"
-    },
-    {
-        "id": "128",
-        "score": "0"
-    },
-    {
-        "id": "130",
-        "score": "1"
-    },
-    {
-        "id": "148",
-        "score": "0"
-    },
-    {
-        "id": "152",
-        "score": "1"
-    },
-    {
-        "id": "153",
-        "score": "1"
-    },
-    {
-        "id": "86",
-        "score": "1"
-    },
-    {
-        "id": "96",
-        "score": "1"
-    },
-    {
-        "id": "146",
-        "score": "1"
-    },
-    {
-        "id": "78",
-        "score": "1"
-    },
-    {
-        "id": "144",
-        "score": "1"
-    },
-    {
-        "id": "151",
-        "score": "1"
-    },
-    {
-        "id": "161",
-        "score": "1"
-    },
-    {
-        "id": "66",
-        "score": "1"
-    },
-    {
-        "id": "140",
-        "score": "1"
-    },
-    {
-        "id": "109",
-        "score": "1"
-    },
-    {
-        "id": "110",
-        "score": "0"
-    },
-    {
-        "id": "114",
-        "score": "1"
-    },
-    {
-        "id": "119",
-        "score": "1"
-    },
-    {
-        "id": "123",
-        "score": "0"
-    },
-    {
-        "id": "124",
-        "score": "0"
-    },
-    {
-        "id": "125",
-        "score": "0"
-    },
-    {
-        "id": "136",
-        "score": "0"
-    },
-    {
-        "id": "142",
-        "score": "1"
-    },
-    {
-        "id": "143",
-        "score": "1"
-    },
-    {
-        "id": "154",
-        "score": "0"
-    },
-    {
-        "id": "182",
-        "score": "1"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "179",
-        "score": "1"
-    },
-    {
-        "id": "186",
-        "score": "1"
-    },
-    {
-        "id": "160",
-        "score": "1"
-    },
-    {
-        "id": "92",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "0"
-    },
-    {
-        "id": "157",
-        "score": "1"
-    },
-    {
-        "id": "183",
-        "score": "0"
-    },
-    {
-        "id": "191",
-        "score": "1"
-    },
-    {
-        "id": "147",
-        "score": "1"
-    },
-    {
-        "id": "167",
-        "score": "1"
-    },
-    {
-        "id": "187",
-        "score": "1"
-    },
-    {
-        "id": "168",
-        "score": "1"
-    },
-    {
-        "id": "170",
-        "score": "1"
-    },
-    {
-        "id": "173",
-        "score": "1"
-    },
-    {
-        "id": "176",
-        "score": "1"
-    },
-    {
-        "id": "177",
-        "score": "1"
-    },
-    {
-        "id": "190",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "0"
-    },
-    {
-        "id": "169",
-        "score": "1"
-    },
-    {
-        "id": "192",
-        "score": "1"
-    },
-    {
-        "id": "105",
-        "score": "1"
-    },
-    {
-        "id": "165",
-        "score": "1"
-    },
-    {
-        "id": "198",
-        "score": "1"
-    },
-    {
-        "id": "197",
-        "score": "1"
-    },
-    {
-        "id": "156",
-        "score": "1"
-    },
-    {
-        "id": "159",
-        "score": "1"
-    },
-    {
-        "id": "162",
-        "score": "1"
-    },
-    {
-        "id": "166",
-        "score": "1"
-    },
-    {
-        "id": "172",
-        "score": "1"
-    },
-    {
-        "id": "175",
-        "score": "1"
-    },
-    {
-        "id": "178",
-        "score": "0"
-    },
-    {
-        "id": "181",
-        "score": "1"
-    },
-    {
-        "id": "184",
-        "score": "1"
-    },
-    {
-        "id": "188",
-        "score": "1"
-    },
-    {
-        "id": "193",
-        "score": "1"
-    },
-    {
-        "id": "189",
-        "score": "1"
-    },
-    {
-        "id": "196",
-        "score": "1"
-    }
-]

eval_milebench/ActionSequence.log

@@ -1,338 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionSequence, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionSequence, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionSequence, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ActionSequence, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting ActionSequence Using internvl
-Proceeding 44-length images samples | Num: 5
-Initialization Finished
-Predicting ActionSequence Using internvl
-Proceeding 44-length images samples | Num: 5
-Initialization Finished
-Predicting ActionSequence Using internvl
-Proceeding 44-length images samples | Num: 5
-Initialization Finished
-Predicting ActionSequence Using internvl
-Proceeding 44-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 31-length images samples | Num: 24
-Proceeding 31-length images samples | Num: 24
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:18,  3.68s/it]
 33%|███▎      | 2/6 [00:04<00:07,  2.00s/it]
 50%|█████     | 3/6 [00:06<00:06,  2.15s/it]
 67%|██████▋   | 4/6 [00:08<00:03,  1.79s/it]
 83%|████████▎ | 5/6 [00:09<00:01,  1.58s/it]
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:19,  3.97s/it]
 33%|���██▎      | 2/6 [00:06<00:12,  3.08s/it]
 50%|█████     | 3/6 [00:07<00:06,  2.06s/it]
 67%|██████▋   | 4/6 [00:08<00:03,  1.67s/it]
 83%|████████▎ | 5/6 [00:10<00:01,  1.95s/it]
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:04<00:20,  4.02s/it]
 33%|███▎      | 2/6 [00:06<00:12,  3.12s/it]
 50%|█████     | 3/6 [00:07<00:06,  2.22s/it]
 67%|██████▋   | 4/6 [00:08<00:03,  1.67s/it]
 83%|████████▎ | 5/6 [00:10<00:01,  1.93s/it]
-Proceeding 32-length images samples | Num: 13
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:02,  1.19s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.73s/it]
-Proceeding 16-length images samples | Num: 4
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.72s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.21s/it]
-Proceeding 16-length images samples | Num: 4
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:04<00:08,  4.09s/it]
 67%|██████▋   | 2/3 [00:06<00:03,  3.01s/it]
-Proceeding 16-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 4
-Proceeding 16-length images samples | Num: 4
-Proceeding 45-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 3
-Proceeding 39-length images samples | Num: 3
-
-Proceeding 69-length images samples | Num: 1
-
-Proceeding 27-length images samples | Num: 14
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 69-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 69-length images samples | Num: 1
-Proceeding 69-length images samples | Num: 1
-
-Proceeding 27-length images samples | Num: 14
-
-Proceeding 27-length images samples | Num: 14
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:03<00:06,  3.44s/it]
 67%|██████▋   | 2/3 [00:04<00:01,  1.81s/it]
-Proceeding 38-length images samples | Num: 4
-Proceeding 27-length images samples | Num: 14
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 21-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:04<00:08,  4.03s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.13s/it]
-Proceeding 38-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 8-length images samples | Num: 1
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:04<00:12,  4.02s/it]
 50%|█████     | 2/4 [00:04<00:04,  2.22s/it]
 75%|███████▌  | 3/4 [00:05<00:01,  1.54s/it]
-Proceeding 38-length images samples | Num: 4
-
-Proceeding 13-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 21-length images samples | Num: 5
-
-Proceeding 30-length images samples | Num: 37
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 21-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 8-length images samples | Num: 1
-Proceeding 38-length images samples | Num: 4
-
-Proceeding 13-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 8-length images samples | Num: 1
-
-Proceeding 30-length images samples | Num: 37
-Proceeding 13-length images samples | Num: 2
-
-Proceeding 21-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 37
-Proceeding 8-length images samples | Num: 1
-Proceeding 13-length images samples | Num: 2
-Proceeding 30-length images samples | Num: 37
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:02<00:22,  2.81s/it]
 22%|██▏       | 2/9 [00:03<00:11,  1.63s/it]
 33%|███▎      | 3/9 [00:04<00:08,  1.46s/it]
 44%|████▍     | 4/9 [00:05<00:05,  1.19s/it]
 56%|█████▌    | 5/9 [00:07<00:06,  1.57s/it]
 67%|██████▋   | 6/9 [00:10<00:06,  2.02s/it]
 78%|███████▊  | 7/9 [00:13<00:04,  2.29s/it]
 89%|████████▉ | 8/9 [00:14<00:01,  1.80s/it]
-Proceeding 36-length images samples | Num: 2
-
-Proceeding 24-length images samples | Num: 4
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:01<00:13,  1.67s/it]
 22%|██▏       | 2/9 [00:04<00:14,  2.08s/it]
 33%|███▎      | 3/9 [00:04<00:08,  1.48s/it]
 44%|████▍     | 4/9 [00:07<00:09,  1.91s/it]
 56%|█████▌    | 5/9 [00:10<00:09,  2.29s/it]
 67%|██████▋   | 6/9 [00:12<00:06,  2.09s/it]
 78%|███████▊  | 7/9 [00:13<00:03,  1.76s/it]
 89%|████████▉ | 8/9 [00:13<00:01,  1.46s/it]
-Proceeding 36-length images samples | Num: 2
-Proceeding 24-length images samples | Num: 4
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.46s/it]
-Proceeding 11-length images samples | Num: 2
-
-Proceeding 15-length images samples | Num: 2
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:02<00:23,  2.89s/it]
 22%|██▏       | 2/9 [00:05<00:19,  2.84s/it]
 33%|███▎      | 3/9 [00:07<00:13,  2.28s/it]
 44%|████▍     | 4/9 [00:09<00:10,  2.09s/it]
 56%|█████▌    | 5/9 [00:11<00:08,  2.17s/it]
 67%|██████▋   | 6/9 [00:12<00:05,  1.72s/it]
 78%|███████▊  | 7/9 [00:14<00:03,  1.81s/it]
 89%|████████▉ | 8/9 [00:15<00:01,  1.61s/it]
-Proceeding 36-length images samples | Num: 2
-
-Proceeding 28-length images samples | Num: 7
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.41s/it]
-Proceeding 11-length images samples | Num: 2
-Proceeding 15-length images samples | Num: 2
-
-
-Proceeding 28-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 29-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 24-length images samples | Num: 4
-Proceeding 36-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.74s/it]
-Proceeding 29-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 25-length images samples | Num: 3
-Proceeding 19-length images samples | Num: 4
-
-Proceeding 24-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.29s/it]
-Proceeding 25-length images samples | Num: 3
-
-Proceeding 35-length images samples | Num: 7
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.57s/it]
-Proceeding 11-length images samples | Num: 2
-Proceeding 19-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 15-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 40-length images samples | Num: 2
-
-Proceeding 35-length images samples | Num: 7
-
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 28-length images samples | Num: 7
-Proceeding 11-length images samples | Num: 2
-Proceeding 15-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 60-length images samples | Num: 1
-Proceeding 28-length images samples | Num: 7
-
-Proceeding 49-length images samples | Num: 2
-
-Proceeding 23-length images samples | Num: 7
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.21s/it]
-Proceeding 40-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.92s/it]
-Proceeding 29-length images samples | Num: 7
-
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 70-length images samples | Num: 1
-Proceeding 37-length images samples | Num: 3
-
-
-Proceeding 22-length images samples | Num: 2
-
-Proceeding 20-length images samples | Num: 2
-Proceeding 29-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 60-length images samples | Num: 1
-
-Proceeding 17-length images samples | Num: 5
-
-Proceeding 49-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.74s/it]
-Proceeding 25-length images samples | Num: 3
-
-Proceeding 23-length images samples | Num: 7
-Proceeding 18-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 1
-Proceeding 25-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 1
-
-Proceeding 26-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.32s/it]
-Proceeding 70-length images samples | Num: 1
-
-Proceeding 35-length images samples | Num: 7
-
-
-Proceeding 37-length images samples | Num: 3
-Proceeding 19-length images samples | Num: 4
-Proceeding 51-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 22-length images samples | Num: 2
-
-Proceeding 20-length images samples | Num: 2
-
-Proceeding 17-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.38s/it]
-Proceeding 40-length images samples | Num: 2
-Proceeding 35-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 18-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 1
-
-Proceeding 26-length images samples | Num: 1
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 60-length images samples | Num: 1
-Proceeding 40-length images samples | Num: 2
-
-Proceeding 49-length images samples | Num: 2
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 7
-Proceeding 60-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.03s/it]
-Proceeding 70-length images samples | Num: 1
-
-Proceeding 37-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 22-length images samples | Num: 2
-Proceeding 49-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 2
-Proceeding 23-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 17-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 18-length images samples | Num: 4
-Proceeding 70-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 1
-
-Proceeding 26-length images samples | Num: 1
-
-Proceeding 37-length images samples | Num: 3
-Proceeding 22-length images samples | Num: 2
-Proceeding 20-length images samples | Num: 2
-Proceeding 17-length images samples | Num: 5
-Proceeding 18-length images samples | Num: 4
-Proceeding 50-length images samples | Num: 1
-Proceeding 26-length images samples | Num: 1
-evaluating ActionSequence ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionSequence/ActionSequence_240803234618.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset ActionSequence                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/ActionSequence 
-internvl:  ActionSequence:  {'Accuracy': 0.715, 'image_quantity_level-Accuracy': {'Few': 0, 'Medium': 0.7153284671532847, 'Many': 0.7142857142857143}, 'image_quantity_level-Result': {'Few': [0, 0], 'Medium': [98, 137], 'Many': [45, 63]}}

eval_milebench/ActionSequence/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.715, "image_quantity_level-Accuracy": {"Few": 0, "Medium": 0.7153284671532847, "Many": 0.7142857142857143}, "image_quantity_level-Result": {"Few": [0, 0], "Medium": [98, 137], "Many": [45, 63]}}

eval_milebench/ActionSequence/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "1"
-    },
-    {
-        "id": "36",
-        "score": "1"
-    },
-    {
-        "id": "1",
-        "score": "1"
-    },
-    {
-        "id": "4",
-        "score": "1"
-    },
-    {
-        "id": "13",
-        "score": "1"
-    },
-    {
-        "id": "22",
-        "score": "1"
-    },
-    {
-        "id": "24",
-        "score": "1"
-    },
-    {
-        "id": "33",
-        "score": "0"
-    },
-    {
-        "id": "2",
-        "score": "1"
-    },
-    {
-        "id": "14",
-        "score": "1"
-    },
-    {
-        "id": "44",
-        "score": "0"
-    },
-    {
-        "id": "82",
-        "score": "1"
-    },
-    {
-        "id": "3",
-        "score": "0"
-    },
-    {
-        "id": "5",
-        "score": "1"
-    },
-    {
-        "id": "6",
-        "score": "1"
-    },
-    {
-        "id": "7",
-        "score": "1"
-    },
-    {
-        "id": "8",
-        "score": "1"
-    },
-    {
-        "id": "23",
-        "score": "0"
-    },
-    {
-        "id": "26",
-        "score": "0"
-    },
-    {
-        "id": "47",
-        "score": "1"
-    },
-    {
-        "id": "9",
-        "score": "1"
-    },
-    {
-        "id": "10",
-        "score": "1"
-    },
-    {
-        "id": "41",
-        "score": "1"
-    },
-    {
-        "id": "11",
-        "score": "0"
-    },
-    {
-        "id": "12",
-        "score": "1"
-    },
-    {
-        "id": "15",
-        "score": "0"
-    },
-    {
-        "id": "18",
-        "score": "1"
-    },
-    {
-        "id": "19",
-        "score": "0"
-    },
-    {
-        "id": "28",
-        "score": "1"
-    },
-    {
-        "id": "29",
-        "score": "1"
-    },
-    {
-        "id": "34",
-        "score": "1"
-    },
-    {
-        "id": "38",
-        "score": "1"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "51",
-        "score": "0"
-    },
-    {
-        "id": "58",
-        "score": "0"
-    },
-    {
-        "id": "16",
-        "score": "1"
-    },
-    {
-        "id": "17",
-        "score": "1"
-    },
-    {
-        "id": "20",
-        "score": "1"
-    },
-    {
-        "id": "31",
-        "score": "1"
-    },
-    {
-        "id": "21",
-        "score": "1"
-    },
-    {
-        "id": "25",
-        "score": "1"
-    },
-    {
-        "id": "30",
-        "score": "1"
-    },
-    {
-        "id": "35",
-        "score": "1"
-    },
-    {
-        "id": "32",
-        "score": "1"
-    },
-    {
-        "id": "66",
-        "score": "1"
-    },
-    {
-        "id": "37",
-        "score": "1"
-    },
-    {
-        "id": "40",
-        "score": "0"
-    },
-    {
-        "id": "42",
-        "score": "0"
-    },
-    {
-        "id": "49",
-        "score": "1"
-    },
-    {
-        "id": "62",
-        "score": "1"
-    },
-    {
-        "id": "50",
-        "score": "1"
-    },
-    {
-        "id": "52",
-        "score": "1"
-    },
-    {
-        "id": "64",
-        "score": "1"
-    },
-    {
-        "id": "53",
-        "score": "1"
-    },
-    {
-        "id": "56",
-        "score": "1"
-    },
-    {
-        "id": "59",
-        "score": "1"
-    },
-    {
-        "id": "105",
-        "score": "1"
-    },
-    {
-        "id": "75",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "0"
-    },
-    {
-        "id": "106",
-        "score": "0"
-    },
-    {
-        "id": "117",
-        "score": "1"
-    },
-    {
-        "id": "134",
-        "score": "1"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "135",
-        "score": "1"
-    },
-    {
-        "id": "163",
-        "score": "1"
-    },
-    {
-        "id": "196",
-        "score": "1"
-    },
-    {
-        "id": "131",
-        "score": "1"
-    },
-    {
-        "id": "45",
-        "score": "1"
-    },
-    {
-        "id": "46",
-        "score": "1"
-    },
-    {
-        "id": "55",
-        "score": "0"
-    },
-    {
-        "id": "68",
-        "score": "1"
-    },
-    {
-        "id": "73",
-        "score": "1"
-    },
-    {
-        "id": "74",
-        "score": "0"
-    },
-    {
-        "id": "87",
-        "score": "1"
-    },
-    {
-        "id": "107",
-        "score": "1"
-    },
-    {
-        "id": "108",
-        "score": "1"
-    },
-    {
-        "id": "78",
-        "score": "0"
-    },
-    {
-        "id": "109",
-        "score": "1"
-    },
-    {
-        "id": "63",
-        "score": "0"
-    },
-    {
-        "id": "57",
-        "score": "1"
-    },
-    {
-        "id": "60",
-        "score": "1"
-    },
-    {
-        "id": "90",
-        "score": "1"
-    },
-    {
-        "id": "104",
-        "score": "1"
-    },
-    {
-        "id": "39",
-        "score": "1"
-    },
-    {
-        "id": "67",
-        "score": "1"
-    },
-    {
-        "id": "188",
-        "score": "1"
-    },
-    {
-        "id": "61",
-        "score": "1"
-    },
-    {
-        "id": "69",
-        "score": "1"
-    },
-    {
-        "id": "72",
-        "score": "1"
-    },
-    {
-        "id": "89",
-        "score": "0"
-    },
-    {
-        "id": "92",
-        "score": "1"
-    },
-    {
-        "id": "93",
-        "score": "0"
-    },
-    {
-        "id": "96",
-        "score": "1"
-    },
-    {
-        "id": "102",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "0"
-    },
-    {
-        "id": "76",
-        "score": "1"
-    },
-    {
-        "id": "27",
-        "score": "1"
-    },
-    {
-        "id": "43",
-        "score": "0"
-    },
-    {
-        "id": "77",
-        "score": "0"
-    },
-    {
-        "id": "65",
-        "score": "1"
-    },
-    {
-        "id": "157",
-        "score": "1"
-    },
-    {
-        "id": "86",
-        "score": "0"
-    },
-    {
-        "id": "148",
-        "score": "0"
-    },
-    {
-        "id": "80",
-        "score": "1"
-    },
-    {
-        "id": "81",
-        "score": "1"
-    },
-    {
-        "id": "112",
-        "score": "1"
-    },
-    {
-        "id": "88",
-        "score": "1"
-    },
-    {
-        "id": "97",
-        "score": "1"
-    },
-    {
-        "id": "125",
-        "score": "1"
-    },
-    {
-        "id": "180",
-        "score": "1"
-    },
-    {
-        "id": "94",
-        "score": "1"
-    },
-    {
-        "id": "156",
-        "score": "1"
-    },
-    {
-        "id": "146",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "1"
-    },
-    {
-        "id": "121",
-        "score": "1"
-    },
-    {
-        "id": "193",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "0"
-    },
-    {
-        "id": "145",
-        "score": "1"
-    },
-    {
-        "id": "151",
-        "score": "0"
-    },
-    {
-        "id": "143",
-        "score": "1"
-    },
-    {
-        "id": "100",
-        "score": "1"
-    },
-    {
-        "id": "113",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "0"
-    },
-    {
-        "id": "127",
-        "score": "1"
-    },
-    {
-        "id": "132",
-        "score": "1"
-    },
-    {
-        "id": "138",
-        "score": "1"
-    },
-    {
-        "id": "118",
-        "score": "0"
-    },
-    {
-        "id": "124",
-        "score": "1"
-    },
-    {
-        "id": "130",
-        "score": "1"
-    },
-    {
-        "id": "126",
-        "score": "1"
-    },
-    {
-        "id": "152",
-        "score": "1"
-    },
-    {
-        "id": "116",
-        "score": "0"
-    },
-    {
-        "id": "120",
-        "score": "0"
-    },
-    {
-        "id": "123",
-        "score": "1"
-    },
-    {
-        "id": "133",
-        "score": "1"
-    },
-    {
-        "id": "54",
-        "score": "1"
-    },
-    {
-        "id": "70",
-        "score": "0"
-    },
-    {
-        "id": "110",
-        "score": "1"
-    },
-    {
-        "id": "114",
-        "score": "0"
-    },
-    {
-        "id": "119",
-        "score": "0"
-    },
-    {
-        "id": "122",
-        "score": "1"
-    },
-    {
-        "id": "128",
-        "score": "1"
-    },
-    {
-        "id": "137",
-        "score": "0"
-    },
-    {
-        "id": "144",
-        "score": "1"
-    },
-    {
-        "id": "147",
-        "score": "1"
-    },
-    {
-        "id": "159",
-        "score": "1"
-    },
-    {
-        "id": "71",
-        "score": "1"
-    },
-    {
-        "id": "83",
-        "score": "1"
-    },
-    {
-        "id": "84",
-        "score": "0"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "174",
-        "score": "0"
-    },
-    {
-        "id": "85",
-        "score": "1"
-    },
-    {
-        "id": "101",
-        "score": "1"
-    },
-    {
-        "id": "198",
-        "score": "0"
-    },
-    {
-        "id": "99",
-        "score": "0"
-    },
-    {
-        "id": "142",
-        "score": "1"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "111",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "1"
-    },
-    {
-        "id": "189",
-        "score": "0"
-    },
-    {
-        "id": "155",
-        "score": "0"
-    },
-    {
-        "id": "164",
-        "score": "1"
-    },
-    {
-        "id": "168",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "1"
-    },
-    {
-        "id": "153",
-        "score": "1"
-    },
-    {
-        "id": "161",
-        "score": "1"
-    },
-    {
-        "id": "175",
-        "score": "1"
-    },
-    {
-        "id": "182",
-        "score": "0"
-    },
-    {
-        "id": "184",
-        "score": "1"
-    },
-    {
-        "id": "191",
-        "score": "0"
-    },
-    {
-        "id": "136",
-        "score": "1"
-    },
-    {
-        "id": "166",
-        "score": "0"
-    },
-    {
-        "id": "172",
-        "score": "0"
-    },
-    {
-        "id": "129",
-        "score": "0"
-    },
-    {
-        "id": "165",
-        "score": "0"
-    },
-    {
-        "id": "140",
-        "score": "1"
-    },
-    {
-        "id": "154",
-        "score": "0"
-    },
-    {
-        "id": "173",
-        "score": "1"
-    },
-    {
-        "id": "167",
-        "score": "0"
-    },
-    {
-        "id": "91",
-        "score": "1"
-    },
-    {
-        "id": "160",
-        "score": "0"
-    },
-    {
-        "id": "162",
-        "score": "0"
-    },
-    {
-        "id": "169",
-        "score": "1"
-    },
-    {
-        "id": "170",
-        "score": "1"
-    },
-    {
-        "id": "176",
-        "score": "1"
-    },
-    {
-        "id": "179",
-        "score": "1"
-    },
-    {
-        "id": "187",
-        "score": "1"
-    },
-    {
-        "id": "190",
-        "score": "1"
-    },
-    {
-        "id": "192",
-        "score": "1"
-    },
-    {
-        "id": "95",
-        "score": "1"
-    },
-    {
-        "id": "98",
-        "score": "0"
-    },
-    {
-        "id": "194",
-        "score": "0"
-    },
-    {
-        "id": "183",
-        "score": "1"
-    },
-    {
-        "id": "178",
-        "score": "1"
-    },
-    {
-        "id": "177",
-        "score": "0"
-    },
-    {
-        "id": "197",
-        "score": "1"
-    },
-    {
-        "id": "149",
-        "score": "0"
-    },
-    {
-        "id": "195",
-        "score": "1"
-    },
-    {
-        "id": "181",
-        "score": "1"
-    },
-    {
-        "id": "186",
-        "score": "1"
-    }
-]

eval_milebench/CLEVR-Change.log

@@ -1,86 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CLEVR-Change, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Initialization Finished
-Predicting CLEVR-Change Using internvl
-Proceeding 2-length images samples | Num: 200
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CLEVR-Change, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Initialization Finished
-Predicting CLEVR-Change Using internvl
-Proceeding 2-length images samples | Num: 200
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CLEVR-Change, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Initialization Finished
-Predicting CLEVR-Change Using internvl
-Proceeding 2-length images samples | Num: 200
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CLEVR-Change, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Initialization Finished
-Predicting CLEVR-Change Using internvl
-Proceeding 2-length images samples | Num: 200
-
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:02<02:03,  2.53s/it]
  4%|▍         | 2/50 [00:03<01:16,  1.58s/it]
  6%|▌         | 3/50 [00:04<00:59,  1.26s/it]
  8%|▊         | 4/50 [00:05<00:51,  1.11s/it]
 10%|█         | 5/50 [00:06<00:46,  1.03s/it]
 12%|█▏        | 6/50 [00:06<00:43,  1.02it/s]
 14%|█▍        | 7/50 [00:07<00:40,  1.05it/s]
 16%|█▌        | 8/50 [00:08<00:39,  1.05it/s]
 18%|█▊        | 9/50 [00:09<00:37,  1.08it/s]
 20%|██        | 10/50 [00:10<00:36,  1.10it/s]
 22%|██▏       | 11/50 [00:11<00:34,  1.12it/s]
 24%|██▍       | 12/50 [00:12<00:33,  1.13it/s]
 26%|██▌       | 13/50 [00:13<00:32,  1.13it/s]
 28%|██▊       | 14/50 [00:14<00:31,  1.14it/s]
 30%|███       | 15/50 [00:14<00:30,  1.14it/s]
 32%|███▏      | 16/50 [00:15<00:29,  1.14it/s]
 34%|███▍      | 17/50 [00:16<00:28,  1.14it/s]
 36%|███▌      | 18/50 [00:17<00:28,  1.14it/s]
 38%|███▊      | 19/50 [00:18<00:27, 
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:02<02:09,  2.64s/it]
  4%|▍         | 2/50 [00:03<01:18,  1.63s/it]
  6%|▌         | 3/50 [00:04<01:00,  1.28s/it]
  8%|▊         | 4/50 [00:05<00:51,  1.12s/it]
 10%|█         | 5/50 [00:06<00:46,  1.04s/it]
 12%|���▏        | 6/50 [00:07<00:43,  1.02it/s]
 14%|█▍        | 7/50 [00:07<00:40,  1.06it/s]
 16%|█▌        | 8/50 [00:08<00:39,  1.05it/s]
 18%|█▊        | 9/50 [00:09<00:38,  1.08it/s]
 20%|██        | 10/50 [00:10<00:36,  1.10it/s]
 22%|██▏       | 11/50 [00:11<00:35,  1.11it/s]
 24%|██▍       | 12/50 [00:12<00:34,  1.12it/s]
 26%|██▌       | 13/50 [00:13<00:32,  1.12it/s]
 28%|██▊       | 14/50 [00:14<00:31,  1.13it/s]
 30%|███       | 15/50 [00:15<00:31,  1.13it/s]
 32%|███▏      | 16/50 [00:15<00:30,  1.13it/s]
 34%|███▍      | 17/50 [00:16<00:29,  1.13it/s]
 36%|███▌      | 18/50 [00:17<00:28,  1.13it/s]
 38%|███▊      | 19/50 [00:18<00:27, 
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:02<02:12,  2.70s/it]
  4%|▍         | 2/50 [00:03<01:19,  1.65s/it]
  6%|▌         | 3/50 [00:04<01:01,  1.31s/it]
  8%|▊         | 4/50 [00:05<00:52,  1.15s/it]
 10%|█         | 5/50 [00:06<00:47,  1.06s/it]
 12%|█▏        | 6/50 [00:07<00:44,  1.01s/it]
 14%|█▍        | 7/50 [00:08<00:41,  1.03it/s]
 16%|█▌        | 8/50 [00:09<00:39,  1.06it/s]
 18%|█▊        | 9/50 [00:09<00:38,  1.07it/s]
 20%|██        | 10/50 [00:10<00:36,  1.08it/s]
 22%|██▏       | 11/50 [00:11<00:35,  1.09it/s]
 24%|██▍       | 12/50 [00:12<00:34,  1.10it/s]
 26%|██▌       | 13/50 [00:13<00:33,  1.11it/s]
 28%|██▊       | 14/50 [00:14<00:32,  1.11it/s]
 30%|███       | 15/50 [00:15<00:31,  1.11it/s]
 32%|███▏      | 16/50 [00:16<00:30,  1.11it/s]
 34%|███▍      | 17/50 [00:17<00:29,  1.12it/s]
 36%|███▌      | 18/50 [00:17<00:28,  1.12it/s]
 38%|███▊      | 19/50 [00:18<00:27,  1.13it/s]
 40%|████      | 20/50 [00:19<00:26,  1.13it/s]
 42%|████▏     | 21/50 [00:20<00:25,  1.14it/s]
 44%|████▍     | 22/50 [00:21<00:24,  1.14it/s]
 46%|████▌     | 23/50 [00:21<00:23,  1.14it/s]
 48%|████▊     | 24/50 [00:22<00:22,  1.14it/s]
 50%|█████     | 25/50 [00:23<00:21,  1.14it/s]
 52%|█████▏    | 26/50 [00:24<00:21,  1.14it/s]
 54%|█████▍    | 27/50 [00:25<00:20,  1.14it/s]
 56%|█████▌    | 28/50 [00:26<00:19,  1.13it/s]
 58%|█████▊    | 29/50 [00:27<00:18,  1.13it/s]
 60%|██████    | 30/50 [00:28<00:17,  1.13it/s]
 62%|██████▏   | 31/50 [00:28<00:16,  1.13it/s]
 64%|██████▍   | 32/50 [00:29<00:15,  1.13it/s]
 66%|██████▌   | 33/50 [00:30<00:14,  1.14it/s]
 68%|██████▊   | 34/50 [00:31<00:13,  1.14it/s]
 70%|███████   | 35/50 [00:32<00:13,  1.15it/s]
 72%|███████▏  | 36/50 [00:33<00:12,  1.1 1.13it/s]
 40%|████      | 20/50 [00:19<00:26,  1.13it/s]
 42%|████▏     | 21/50 [00:20<00:25,  1.14it/s]
 44%|████▍     | 22/50 [00:21<00:24,  1.14it/s]
 46%|████▌     | 23/50 [00:22<00:23,  1.14it/s]
 48%|████▊     | 24/50 [00:22<00:22,  1.14it/s]
 50%|█████     | 25/50 [00:23<00:21,  1.14it/s]
 52%|█████▏    | 26/50 [00:24<00:21,  1.14it/s]
 54%|█████▍    | 27/50 [00:25<00:20,  1.14it/s]
 56%|█████▌    | 28/50 [00:26<00:19,  1.14it/s]
 58%|█████▊    | 29/50 [00:27<00:18,  1.14it/s]
 60%|██████    | 30/50 [00:28<00:17,  1.14it/s]
 62%|██████▏   | 31/50 [00:29<00:16,  1.14it/s]
 64%|██████▍   | 32/50 [00:29<00:15,  1.15it/s]
 66%|██████▌   | 33/50 [00:30<00:14,  1.15it/s]
 68%|██████▊   | 34/50 [00:31<00:13,  1.15it/s]
 70%|███████   | 35/50 [00:32<00:13,  1.15it/s]
 72%|███████▏  | 36/50 [00:33<00:12,  1.1 1.12it/s]
 40%|████      | 20/50 [00:19<00:26,  1.12it/s]
 42%|████▏     | 21/50 [00:20<00:25,  1.13it/s]
 44%|████▍     | 22/50 [00:21<00:24,  1.13it/s]
 46%|████▌     | 23/50 [00:22<00:23,  1.13it/s]
 48%|████▊     | 24/50 [00:23<00:22,  1.13it/s]
 50%|█████     | 25/50 [00:24<00:22,  1.14it/s]
 52%|█████▏    | 26/50 [00:25<00:21,  1.14it/s]
 54%|█████▍    | 27/50 [00:25<00:20,  1.13it/s]
 56%|█████▌    | 28/50 [00:26<00:19,  1.13it/s]
 58%|█████▊    | 29/50 [00:27<00:18,  1.13it/s]
 60%|██████    | 30/50 [00:28<00:17,  1.12it/s]
 62%|██████▏   | 31/50 [00:29<00:16,  1.12it/s]
 64%|██████▍   | 32/50 [00:30<00:16,  1.12it/s]
 66%|██████▌   | 33/50 [00:31<00:15,  1.12it/s]
 68%|██████▊   | 34/50 [00:32<00:14,  1.12it/s]
 70%|███████   | 35/50 [00:33<00:13,  1.13it/s]
 72%|███████▏  | 36/50 [00:33<00:12,  1.15it/s]
 74%|███████▍  | 37/50 [00:34<00:11,  1.15it/s]
 76%|███████▌  | 38/50 [00:35<00:10,  1.15it/s]
 78%|███████▊  | 39/50 [00:35<00:09,  1.16it/s]
 80%|████████  | 40/50 [00:36<00:08,  1.16it/s]
 82%|███████��▏ | 41/50 [00:37<00:07,  1.16it/s]
 84%|████████▍ | 42/50 [00:38<00:06,  1.16it/s]
 86%|████████▌ | 43/50 [00:39<00:06,  1.16it/s]
 88%|████████▊ | 44/50 [00:40<00:05,  1.16it/s]
 90%|█████████ | 45/50 [00:41<00:04,  1.16it/s]
 92%|█████████▏| 46/50 [00:41<00:03,  1.16it/s]
 94%|█████████▍| 47/50 [00:42<00:02,  1.16it/s]
 96%|█████████▌| 48/50 [00:43<00:01,  1.15it/s]
 98%|█████████▊| 49/50 [00:44<00:00,  1.15it/s]
-5it/s]
 74%|███████▍  | 37/50 [00:34<00:11,  1.15it/s]
 76%|███████▌  | 38/50 [00:35<00:10,  1.15it/s]
 78%|███████▊  | 39/50 [00:36<00:09,  1.15it/s]
 80%|████████  | 40/50 [00:36<00:08,  1.15it/s]
 82%|████████▏ | 41/50 [00:37<00:07,  1.15it/s]
 84%|████████▍ | 42/50 [00:38<00:06,  1.15it/s]
 86%|████████▌ | 43/50 [00:39<00:06,  1.15it/s]
 88%|████████▊ | 44/50 [00:40<00:05,  1.15it/s]
 90%|█████████ | 45/50 [00:41<00:04,  1.15it/s]
 92%|█████████▏| 46/50 [00:42<00:03,  1.15it/s]
 94%|█████████▍| 47/50 [00:42<00:02,  1.15it/s]
 96%|█████████▌| 48/50 [00:43<00:01,  1.15it/s]
 98%|█████████▊| 49/50 [00:44<00:00,  1.15it/s]
-2it/s]
 74%|███████▍  | 37/50 [00:34<00:11,  1.12it/s]
 76%|███████▌  | 38/50 [00:35<00:10,  1.11it/s]
 78%|███████▊  | 39/50 [00:36<00:09,  1.11it/s]
 80%|████████  | 40/50 [00:37<00:08,  1.12it/s]
 82%|████████▏ | 41/50 [00:38<00:08,  1.11it/s]
 84%|████████▍ | 42/50 [00:39<00:07,  1.10it/s]
 86%|████████▌ | 43/50 [00:40<00:06,  1.11it/s]
 88%|████████▊ | 44/50 [00:41<00:05,  1.12it/s]
 90%|█████████ | 45/50 [00:42<00:04,  1.12it/s]
 92%|█████████▏| 46/50 [00:42<00:03,  1.12it/s]
 94%|█████████▍| 47/50 [00:43<00:02,  1.12it/s]
 96%|█████████▌| 48/50 [00:44<00:01,  1.12it/s]
 98%|█████████▊| 49/50 [00:45<00:00,  1.12it/s]
-evaluating CLEVR-Change ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/CLEVR-Change/CLEVR-Change_240803234510.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset CLEVR-Change                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/CLEVR-Change 
-internvl:  CLEVR-Change:  {'Rouge-L f': 0.19528275526918393, 'image_quantity_level-Accuracy': {'Few': 0.19528275526918393, 'Medium': 0, 'Many': 0}, 'image_quantity_level-Result': {'Few': [39.05655105383679, 200], 'Medium': [0, 0], 'Many': [0, 0]}}

eval_milebench/CLEVR-Change/eval.json

@@ -1 +0,0 @@
-{"Rouge-L f": 0.19528275526918393, "image_quantity_level-Accuracy": {"Few": 0.19528275526918393, "Medium": 0, "Many": 0}, "image_quantity_level-Result": {"Few": [39.05655105383679, 200], "Medium": [0, 0], "Many": [0, 0]}}

eval_milebench/CLEVR-Change/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "0.35"
-    },
-    {
-        "id": "1",
-        "score": "0.222"
-    },
-    {
-        "id": "2",
-        "score": "0.08"
-    },
-    {
-        "id": "3",
-        "score": "0.118"
-    },
-    {
-        "id": "4",
-        "score": "0.238"
-    },
-    {
-        "id": "5",
-        "score": "0.304"
-    },
-    {
-        "id": "6",
-        "score": "0.182"
-    },
-    {
-        "id": "7",
-        "score": "0.311"
-    },
-    {
-        "id": "8",
-        "score": "0.244"
-    },
-    {
-        "id": "9",
-        "score": "0.222"
-    },
-    {
-        "id": "10",
-        "score": "0.163"
-    },
-    {
-        "id": "11",
-        "score": "0.244"
-    },
-    {
-        "id": "12",
-        "score": "0.085"
-    },
-    {
-        "id": "13",
-        "score": "0.341"
-    },
-    {
-        "id": "14",
-        "score": "0.24"
-    },
-    {
-        "id": "15",
-        "score": "0.25"
-    },
-    {
-        "id": "16",
-        "score": "0.14"
-    },
-    {
-        "id": "17",
-        "score": "0.19"
-    },
-    {
-        "id": "18",
-        "score": "0.08"
-    },
-    {
-        "id": "19",
-        "score": "0.115"
-    },
-    {
-        "id": "20",
-        "score": "0.227"
-    },
-    {
-        "id": "21",
-        "score": "0.208"
-    },
-    {
-        "id": "22",
-        "score": "0.222"
-    },
-    {
-        "id": "23",
-        "score": "0.091"
-    },
-    {
-        "id": "24",
-        "score": "0.12"
-    },
-    {
-        "id": "25",
-        "score": "0.208"
-    },
-    {
-        "id": "26",
-        "score": "0.2"
-    },
-    {
-        "id": "27",
-        "score": "0.186"
-    },
-    {
-        "id": "28",
-        "score": "0.182"
-    },
-    {
-        "id": "29",
-        "score": "0.19"
-    },
-    {
-        "id": "30",
-        "score": "0.195"
-    },
-    {
-        "id": "31",
-        "score": "0.25"
-    },
-    {
-        "id": "32",
-        "score": "0.222"
-    },
-    {
-        "id": "33",
-        "score": "0.093"
-    },
-    {
-        "id": "34",
-        "score": "0.154"
-    },
-    {
-        "id": "35",
-        "score": "0.205"
-    },
-    {
-        "id": "36",
-        "score": "0.083"
-    },
-    {
-        "id": "37",
-        "score": "0.125"
-    },
-    {
-        "id": "38",
-        "score": "0.154"
-    },
-    {
-        "id": "39",
-        "score": "0.286"
-    },
-    {
-        "id": "40",
-        "score": "0.205"
-    },
-    {
-        "id": "41",
-        "score": "0.233"
-    },
-    {
-        "id": "42",
-        "score": "0.217"
-    },
-    {
-        "id": "43",
-        "score": "0.061"
-    },
-    {
-        "id": "44",
-        "score": "0.244"
-    },
-    {
-        "id": "45",
-        "score": "0.244"
-    },
-    {
-        "id": "46",
-        "score": "0.227"
-    },
-    {
-        "id": "47",
-        "score": "0.208"
-    },
-    {
-        "id": "48",
-        "score": "0.128"
-    },
-    {
-        "id": "49",
-        "score": "0.125"
-    },
-    {
-        "id": "50",
-        "score": "0.17"
-    },
-    {
-        "id": "51",
-        "score": "0.222"
-    },
-    {
-        "id": "52",
-        "score": "0.286"
-    },
-    {
-        "id": "53",
-        "score": "0.286"
-    },
-    {
-        "id": "54",
-        "score": "0.286"
-    },
-    {
-        "id": "55",
-        "score": "0.273"
-    },
-    {
-        "id": "56",
-        "score": "0.178"
-    },
-    {
-        "id": "57",
-        "score": "0.1"
-    },
-    {
-        "id": "58",
-        "score": "0.17"
-    },
-    {
-        "id": "59",
-        "score": "0.227"
-    },
-    {
-        "id": "60",
-        "score": "0.208"
-    },
-    {
-        "id": "61",
-        "score": "0.174"
-    },
-    {
-        "id": "62",
-        "score": "0.273"
-    },
-    {
-        "id": "63",
-        "score": "0.292"
-    },
-    {
-        "id": "64",
-        "score": "0.157"
-    },
-    {
-        "id": "65",
-        "score": "0.255"
-    },
-    {
-        "id": "66",
-        "score": "0.182"
-    },
-    {
-        "id": "67",
-        "score": "0.167"
-    },
-    {
-        "id": "68",
-        "score": "0.25"
-    },
-    {
-        "id": "69",
-        "score": "0.1"
-    },
-    {
-        "id": "70",
-        "score": "0.105"
-    },
-    {
-        "id": "71",
-        "score": "0.082"
-    },
-    {
-        "id": "72",
-        "score": "0.111"
-    },
-    {
-        "id": "73",
-        "score": "0.14"
-    },
-    {
-        "id": "74",
-        "score": "0.195"
-    },
-    {
-        "id": "75",
-        "score": "0.178"
-    },
-    {
-        "id": "76",
-        "score": "0.178"
-    },
-    {
-        "id": "77",
-        "score": "0.216"
-    },
-    {
-        "id": "78",
-        "score": "0.095"
-    },
-    {
-        "id": "79",
-        "score": "0.216"
-    },
-    {
-        "id": "80",
-        "score": "0.089"
-    },
-    {
-        "id": "81",
-        "score": "0.136"
-    },
-    {
-        "id": "82",
-        "score": "0.192"
-    },
-    {
-        "id": "83",
-        "score": "0.122"
-    },
-    {
-        "id": "84",
-        "score": "0.098"
-    },
-    {
-        "id": "85",
-        "score": "0.082"
-    },
-    {
-        "id": "86",
-        "score": "0.267"
-    },
-    {
-        "id": "87",
-        "score": "0.133"
-    },
-    {
-        "id": "88",
-        "score": "0.136"
-    },
-    {
-        "id": "89",
-        "score": "0.14"
-    },
-    {
-        "id": "90",
-        "score": "0.158"
-    },
-    {
-        "id": "91",
-        "score": "0.186"
-    },
-    {
-        "id": "92",
-        "score": "0.213"
-    },
-    {
-        "id": "93",
-        "score": "0.1"
-    },
-    {
-        "id": "94",
-        "score": "0.298"
-    },
-    {
-        "id": "95",
-        "score": "0.133"
-    },
-    {
-        "id": "96",
-        "score": "0.227"
-    },
-    {
-        "id": "97",
-        "score": "0.217"
-    },
-    {
-        "id": "98",
-        "score": "0.089"
-    },
-    {
-        "id": "99",
-        "score": "0.174"
-    },
-    {
-        "id": "100",
-        "score": "0.211"
-    },
-    {
-        "id": "101",
-        "score": "0.17"
-    },
-    {
-        "id": "102",
-        "score": "0.204"
-    },
-    {
-        "id": "103",
-        "score": "0.136"
-    },
-    {
-        "id": "104",
-        "score": "0.205"
-    },
-    {
-        "id": "105",
-        "score": "0.286"
-    },
-    {
-        "id": "106",
-        "score": "0.146"
-    },
-    {
-        "id": "107",
-        "score": "0.24"
-    },
-    {
-        "id": "108",
-        "score": "0.245"
-    },
-    {
-        "id": "109",
-        "score": "0.15"
-    },
-    {
-        "id": "110",
-        "score": "0.273"
-    },
-    {
-        "id": "111",
-        "score": "0.083"
-    },
-    {
-        "id": "112",
-        "score": "0.255"
-    },
-    {
-        "id": "113",
-        "score": "0.17"
-    },
-    {
-        "id": "114",
-        "score": "0.109"
-    },
-    {
-        "id": "115",
-        "score": "0.261"
-    },
-    {
-        "id": "116",
-        "score": "0.293"
-    },
-    {
-        "id": "117",
-        "score": "0.098"
-    },
-    {
-        "id": "118",
-        "score": "0.273"
-    },
-    {
-        "id": "119",
-        "score": "0.217"
-    },
-    {
-        "id": "120",
-        "score": "0.17"
-    },
-    {
-        "id": "121",
-        "score": "0.255"
-    },
-    {
-        "id": "122",
-        "score": "0.196"
-    },
-    {
-        "id": "123",
-        "score": "0.12"
-    },
-    {
-        "id": "124",
-        "score": "0.278"
-    },
-    {
-        "id": "125",
-        "score": "0.298"
-    },
-    {
-        "id": "126",
-        "score": "0.151"
-    },
-    {
-        "id": "127",
-        "score": "0.218"
-    },
-    {
-        "id": "128",
-        "score": "0.186"
-    },
-    {
-        "id": "129",
-        "score": "0.053"
-    },
-    {
-        "id": "130",
-        "score": "0.143"
-    },
-    {
-        "id": "131",
-        "score": "0.085"
-    },
-    {
-        "id": "132",
-        "score": "0.2"
-    },
-    {
-        "id": "133",
-        "score": "0.24"
-    },
-    {
-        "id": "134",
-        "score": "0.095"
-    },
-    {
-        "id": "135",
-        "score": "0.292"
-    },
-    {
-        "id": "136",
-        "score": "0.227"
-    },
-    {
-        "id": "137",
-        "score": "0.245"
-    },
-    {
-        "id": "138",
-        "score": "0.167"
-    },
-    {
-        "id": "139",
-        "score": "0.133"
-    },
-    {
-        "id": "140",
-        "score": "0.14"
-    },
-    {
-        "id": "141",
-        "score": "0.133"
-    },
-    {
-        "id": "142",
-        "score": "0.233"
-    },
-    {
-        "id": "143",
-        "score": "0.255"
-    },
-    {
-        "id": "144",
-        "score": "0.128"
-    },
-    {
-        "id": "145",
-        "score": "0.298"
-    },
-    {
-        "id": "146",
-        "score": "0.143"
-    },
-    {
-        "id": "147",
-        "score": "0.083"
-    },
-    {
-        "id": "148",
-        "score": "0.077"
-    },
-    {
-        "id": "149",
-        "score": "0.17"
-    },
-    {
-        "id": "150",
-        "score": "0.261"
-    },
-    {
-        "id": "151",
-        "score": "0.14"
-    },
-    {
-        "id": "152",
-        "score": "0.113"
-    },
-    {
-        "id": "153",
-        "score": "0.17"
-    },
-    {
-        "id": "154",
-        "score": "0.261"
-    },
-    {
-        "id": "155",
-        "score": "0.227"
-    },
-    {
-        "id": "156",
-        "score": "0.208"
-    },
-    {
-        "id": "157",
-        "score": "0.178"
-    },
-    {
-        "id": "158",
-        "score": "0.213"
-    },
-    {
-        "id": "159",
-        "score": "0.186"
-    },
-    {
-        "id": "160",
-        "score": "0.244"
-    },
-    {
-        "id": "161",
-        "score": "0.255"
-    },
-    {
-        "id": "162",
-        "score": "0.13"
-    },
-    {
-        "id": "163",
-        "score": "0.14"
-    },
-    {
-        "id": "164",
-        "score": "0.12"
-    },
-    {
-        "id": "165",
-        "score": "0.308"
-    },
-    {
-        "id": "166",
-        "score": "0.244"
-    },
-    {
-        "id": "167",
-        "score": "0.326"
-    },
-    {
-        "id": "168",
-        "score": "0.267"
-    },
-    {
-        "id": "169",
-        "score": "0.195"
-    },
-    {
-        "id": "170",
-        "score": "0.192"
-    },
-    {
-        "id": "171",
-        "score": "0.286"
-    },
-    {
-        "id": "172",
-        "score": "0.213"
-    },
-    {
-        "id": "173",
-        "score": "0.233"
-    },
-    {
-        "id": "174",
-        "score": "0.211"
-    },
-    {
-        "id": "175",
-        "score": "0.267"
-    },
-    {
-        "id": "176",
-        "score": "0.267"
-    },
-    {
-        "id": "177",
-        "score": "0.25"
-    },
-    {
-        "id": "178",
-        "score": "0.326"
-    },
-    {
-        "id": "179",
-        "score": "0.227"
-    },
-    {
-        "id": "180",
-        "score": "0.383"
-    },
-    {
-        "id": "181",
-        "score": "0.208"
-    },
-    {
-        "id": "182",
-        "score": "0.222"
-    },
-    {
-        "id": "183",
-        "score": "0.17"
-    },
-    {
-        "id": "184",
-        "score": "0.174"
-    },
-    {
-        "id": "185",
-        "score": "0.118"
-    },
-    {
-        "id": "186",
-        "score": "0.213"
-    },
-    {
-        "id": "187",
-        "score": "0.19"
-    },
-    {
-        "id": "188",
-        "score": "0.227"
-    },
-    {
-        "id": "189",
-        "score": "0.213"
-    },
-    {
-        "id": "190",
-        "score": "0.196"
-    },
-    {
-        "id": "191",
-        "score": "0.238"
-    },
-    {
-        "id": "192",
-        "score": "0.178"
-    },
-    {
-        "id": "193",
-        "score": "0.318"
-    },
-    {
-        "id": "194",
-        "score": "0.318"
-    },
-    {
-        "id": "195",
-        "score": "0.25"
-    },
-    {
-        "id": "196",
-        "score": "0.128"
-    },
-    {
-        "id": "197",
-        "score": "0.17"
-    },
-    {
-        "id": "198",
-        "score": "0.261"
-    },
-    {
-        "id": "199",
-        "score": "0.213"
-    }
-]

eval_milebench/CharacterOrder.log

@@ -1,261 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CharacterOrder, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Initialization Finished
-Predicting CharacterOrder Using internvl
-Proceeding 29-length images samples | Num: 8
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CharacterOrder, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting CharacterOrder Using internvl
-Proceeding 29-length images samples | Num: 8
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CharacterOrder, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Initialization Finished
-Predicting CharacterOrder Using internvl
-Proceeding 29-length images samples | Num: 8
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CharacterOrder, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Initialization Finished
-Predicting CharacterOrder Using internvl
-Proceeding 29-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.04s/it]
-Proceeding 27-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.03s/it]
-Proceeding 27-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.54s/it]
-Proceeding 27-length images samples | Num: 5
-Proceeding 27-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 23-length images samples | Num: 9
-Proceeding 23-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.64s/it]
-Proceeding 25-length images samples | Num: 12
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.67s/it]
-Proceeding 25-length images samples | Num: 12
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.00s/it]
-Proceeding 25-length images samples | Num: 12
-Proceeding 25-length images samples | Num: 12
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.15s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.68s/it]
-Proceeding 28-length images samples | Num: 12
-Proceeding 28-length images samples | Num: 12
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.62s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.64s/it]
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.63s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.32s/it]
-Proceeding 28-length images samples | Num: 12
-Proceeding 28-length images samples | Num: 12
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.17s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.13s/it]
-Proceeding 16-length images samples | Num: 3
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.42s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.29s/it]
-Proceeding 16-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 20-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 7
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.94s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.43s/it]
-Proceeding 16-length images samples | Num: 3
-
-Proceeding 20-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.61s/it]
-Proceeding 19-length images samples | Num: 7
-Proceeding 16-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 19-length images samples | Num: 7
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.14s/it]
-Proceeding 35-length images samples | Num: 20
-Proceeding 20-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 20
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.40s/it]
-Proceeding 35-length images samples | Num: 20
-Proceeding 19-length images samples | Num: 7
-Proceeding 35-length images samples | Num: 20
-
  0%|          | 0/5 [00:00<?, ?it/s]
 20%|██        | 1/5 [00:03<00:12,  3.01s/it]
 40%|████      | 2/5 [00:05<00:07,  2.61s/it]
 60%|██████    | 3/5 [00:07<00:04,  2.44s/it]
 80%|████████  | 4/5 [00:09<00:02,  2.41s/it]
-Proceeding 17-length images samples | Num: 4
-
  0%|          | 0/5 [00:00<?, ?it/s]
 20%|██        | 1/5 [00:03<00:13,  3.26s/it]
 40%|████      | 2/5 [00:05<00:07,  2.57s/it]
 60%|██████    | 3/5 [00:07<00:04,  2.37s/it]
 80%|████████  | 4/5 [00:09<00:02,  2.09s/it]
-Proceeding 17-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 26-length images samples | Num: 11
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 26-length images samples | Num: 11
-
  0%|          | 0/5 [00:00<?, ?it/s]
 20%|██        | 1/5 [00:02<00:10,  2.68s/it]
 40%|████      | 2/5 [00:04<00:07,  2.43s/it]
 60%|██████    | 3/5 [00:07<00:04,  2.40s/it]
 80%|████████  | 4/5 [00:09<00:02,  2.35s/it]
-Proceeding 17-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 26-length images samples | Num: 11
-Proceeding 17-length images samples | Num: 4
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.96s/it]
-Proceeding 22-length images samples | Num: 9
-Proceeding 26-length images samples | Num: 11
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.80s/it]
 67%|██████▋   | 2/3 [00:03<00:02,  2.03s/it]
-Proceeding 22-length images samples | Num: 9
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.85s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.74s/it]
-Proceeding 22-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.91s/it]
-Proceeding 30-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.30s/it]
-Proceeding 30-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.69s/it]
-Proceeding 30-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 24-length images samples | Num: 12
-Proceeding 22-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 24-length images samples | Num: 12
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 24-length images samples | Num: 12
-Proceeding 30-length images samples | Num: 5
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.83s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.95s/it]
-Proceeding 34-length images samples | Num: 28
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.91s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.84s/it]
-Proceeding 34-length images samples | Num: 28
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.94s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.64s/it]
-Proceeding 34-length images samples | Num: 28
-Proceeding 24-length images samples | Num: 12
-Proceeding 34-length images samples | Num: 28
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:02<00:17,  2.97s/it]
 29%|██▊       | 2/7 [00:05<00:14,  2.80s/it]
 43%|████▎     | 3/7 [00:08<00:10,  2.69s/it]
 57%|█████▋    | 4/7 [00:10<00:07,  2.47s/it]
 71%|███████▏  | 5/7 [00:12<00:04,  2.34s/it]
 86%|████████▌ | 6/7 [00:14<00:02,  2.19s/it]
-Proceeding 10-length images samples | Num: 3
-
-Proceeding 21-length images samples | Num: 8
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:03<00:20,  3.43s/it]
 29%|██▊       | 2/7 [00:05<00:13,  2.72s/it]
 43%|████▎     | 3/7 [00:07<00:09,  2.49s/it]
 57%|█████▋    | 4/7 [00:10<00:08,  2.68s/it]
 71%|███████▏  | 5/7 [00:12<00:04,  2.44s/it]
 86%|████████▌ | 6/7 [00:15<00:02,  2.49s/it]
-Proceeding 10-length images samples | Num: 3
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:02<00:12,  2.15s/it]
 29%|██▊       | 2/7 [00:04<00:12,  2.47s/it]
 43%|████▎     | 3/7 [00:07<00:10,  2.57s/it]
 57%|█████▋    | 4/7 [00:09<00:07,  2.43s/it]
 71%|███████▏  | 5/7 [00:11<00:04,  2.33s/it]
 86%|████████▌ | 6/7 [00:14<00:02,  2.35s/it]
-Proceeding 10-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 21-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 21-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.87s/it]
-Proceeding 12-length images samples | Num: 3
-
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.75s/it]
-Proceeding 12-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.52s/it]
-Proceeding 12-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 32-length images samples | Num: 13
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 10-length images samples | Num: 3
-Proceeding 21-length images samples | Num: 8
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.83s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.08s/it]
-Proceeding 33-length images samples | Num: 3
-
-Proceeding 15-length images samples | Num: 3
-
-Proceeding 14-length images samples | Num: 3
-
-Proceeding 13-length images samples | Num: 2
-
-Proceeding 31-length images samples | Num: 8
-Proceeding 12-length images samples | Num: 3
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.50s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.80s/it]
-Proceeding 33-length images samples | Num: 3
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:01<00:03,  1.85s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.94s/it]
-Proceeding 33-length images samples | Num: 3
-Proceeding 32-length images samples | Num: 13
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 15-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.28s/it]
-Proceeding 18-length images samples | Num: 1
-
-Proceeding 9-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 15-length images samples | Num: 3
-
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 14-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 14-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 13-length images samples | Num: 2
-
-Proceeding 31-length images samples | Num: 8
-Proceeding 13-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 31-length images samples | Num: 8
-Proceeding 33-length images samples | Num: 3
-Proceeding 18-length images samples | Num: 1
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.75s/it]
-
-Proceeding 9-length images samples | Num: 2
-
-Proceeding 15-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.19s/it]
-Proceeding 18-length images samples | Num: 1
-
-Proceeding 9-length images samples | Num: 2
-Proceeding 14-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 13-length images samples | Num: 2
-Proceeding 31-length images samples | Num: 8
-Proceeding 18-length images samples | Num: 1
-Proceeding 9-length images samples | Num: 2
-evaluating CharacterOrder ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/CharacterOrder/CharacterOrder_240803234555.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset CharacterOrder                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/CharacterOrder 
-internvl:  CharacterOrder:  {'Accuracy': 0.53, 'image_quantity_level-Accuracy': {'Few': 0, 'Medium': 0.5514705882352942, 'Many': 0.484375}, 'image_quantity_level-Result': {'Few': [0, 0], 'Medium': [75, 136], 'Many': [31, 64]}}

eval_milebench/CharacterOrder/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.53, "image_quantity_level-Accuracy": {"Few": 0, "Medium": 0.5514705882352942, "Many": 0.484375}, "image_quantity_level-Result": {"Few": [0, 0], "Medium": [75, 136], "Many": [31, 64]}}

eval_milebench/CharacterOrder/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "1"
-    },
-    {
-        "id": "4",
-        "score": "1"
-    },
-    {
-        "id": "1",
-        "score": "0"
-    },
-    {
-        "id": "16",
-        "score": "1"
-    },
-    {
-        "id": "2",
-        "score": "1"
-    },
-    {
-        "id": "18",
-        "score": "0"
-    },
-    {
-        "id": "30",
-        "score": "1"
-    },
-    {
-        "id": "3",
-        "score": "0"
-    },
-    {
-        "id": "7",
-        "score": "1"
-    },
-    {
-        "id": "8",
-        "score": "0"
-    },
-    {
-        "id": "5",
-        "score": "0"
-    },
-    {
-        "id": "15",
-        "score": "0"
-    },
-    {
-        "id": "43",
-        "score": "0"
-    },
-    {
-        "id": "6",
-        "score": "0"
-    },
-    {
-        "id": "10",
-        "score": "1"
-    },
-    {
-        "id": "17",
-        "score": "1"
-    },
-    {
-        "id": "12",
-        "score": "0"
-    },
-    {
-        "id": "14",
-        "score": "0"
-    },
-    {
-        "id": "13",
-        "score": "0"
-    },
-    {
-        "id": "22",
-        "score": "0"
-    },
-    {
-        "id": "31",
-        "score": "1"
-    },
-    {
-        "id": "33",
-        "score": "0"
-    },
-    {
-        "id": "46",
-        "score": "1"
-    },
-    {
-        "id": "19",
-        "score": "1"
-    },
-    {
-        "id": "21",
-        "score": "0"
-    },
-    {
-        "id": "51",
-        "score": "1"
-    },
-    {
-        "id": "54",
-        "score": "1"
-    },
-    {
-        "id": "23",
-        "score": "1"
-    },
-    {
-        "id": "26",
-        "score": "1"
-    },
-    {
-        "id": "37",
-        "score": "0"
-    },
-    {
-        "id": "24",
-        "score": "1"
-    },
-    {
-        "id": "61",
-        "score": "1"
-    },
-    {
-        "id": "25",
-        "score": "1"
-    },
-    {
-        "id": "35",
-        "score": "0"
-    },
-    {
-        "id": "39",
-        "score": "1"
-    },
-    {
-        "id": "27",
-        "score": "0"
-    },
-    {
-        "id": "29",
-        "score": "0"
-    },
-    {
-        "id": "32",
-        "score": "1"
-    },
-    {
-        "id": "49",
-        "score": "0"
-    },
-    {
-        "id": "65",
-        "score": "0"
-    },
-    {
-        "id": "74",
-        "score": "1"
-    },
-    {
-        "id": "75",
-        "score": "1"
-    },
-    {
-        "id": "28",
-        "score": "1"
-    },
-    {
-        "id": "36",
-        "score": "0"
-    },
-    {
-        "id": "50",
-        "score": "1"
-    },
-    {
-        "id": "40",
-        "score": "1"
-    },
-    {
-        "id": "44",
-        "score": "0"
-    },
-    {
-        "id": "58",
-        "score": "0"
-    },
-    {
-        "id": "69",
-        "score": "1"
-    },
-    {
-        "id": "77",
-        "score": "1"
-    },
-    {
-        "id": "52",
-        "score": "1"
-    },
-    {
-        "id": "53",
-        "score": "1"
-    },
-    {
-        "id": "66",
-        "score": "1"
-    },
-    {
-        "id": "72",
-        "score": "1"
-    },
-    {
-        "id": "84",
-        "score": "1"
-    },
-    {
-        "id": "106",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "1"
-    },
-    {
-        "id": "107",
-        "score": "1"
-    },
-    {
-        "id": "20",
-        "score": "1"
-    },
-    {
-        "id": "45",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "0"
-    },
-    {
-        "id": "42",
-        "score": "1"
-    },
-    {
-        "id": "138",
-        "score": "0"
-    },
-    {
-        "id": "9",
-        "score": "0"
-    },
-    {
-        "id": "11",
-        "score": "0"
-    },
-    {
-        "id": "123",
-        "score": "0"
-    },
-    {
-        "id": "64",
-        "score": "0"
-    },
-    {
-        "id": "82",
-        "score": "1"
-    },
-    {
-        "id": "105",
-        "score": "0"
-    },
-    {
-        "id": "94",
-        "score": "0"
-    },
-    {
-        "id": "34",
-        "score": "0"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "38",
-        "score": "0"
-    },
-    {
-        "id": "70",
-        "score": "1"
-    },
-    {
-        "id": "47",
-        "score": "1"
-    },
-    {
-        "id": "60",
-        "score": "0"
-    },
-    {
-        "id": "63",
-        "score": "1"
-    },
-    {
-        "id": "67",
-        "score": "0"
-    },
-    {
-        "id": "76",
-        "score": "0"
-    },
-    {
-        "id": "68",
-        "score": "1"
-    },
-    {
-        "id": "71",
-        "score": "0"
-    },
-    {
-        "id": "73",
-        "score": "0"
-    },
-    {
-        "id": "80",
-        "score": "1"
-    },
-    {
-        "id": "41",
-        "score": "1"
-    },
-    {
-        "id": "59",
-        "score": "1"
-    },
-    {
-        "id": "124",
-        "score": "1"
-    },
-    {
-        "id": "55",
-        "score": "1"
-    },
-    {
-        "id": "57",
-        "score": "0"
-    },
-    {
-        "id": "83",
-        "score": "1"
-    },
-    {
-        "id": "85",
-        "score": "0"
-    },
-    {
-        "id": "86",
-        "score": "0"
-    },
-    {
-        "id": "92",
-        "score": "0"
-    },
-    {
-        "id": "101",
-        "score": "0"
-    },
-    {
-        "id": "114",
-        "score": "0"
-    },
-    {
-        "id": "120",
-        "score": "1"
-    },
-    {
-        "id": "125",
-        "score": "1"
-    },
-    {
-        "id": "144",
-        "score": "0"
-    },
-    {
-        "id": "90",
-        "score": "1"
-    },
-    {
-        "id": "126",
-        "score": "1"
-    },
-    {
-        "id": "111",
-        "score": "0"
-    },
-    {
-        "id": "78",
-        "score": "0"
-    },
-    {
-        "id": "88",
-        "score": "0"
-    },
-    {
-        "id": "98",
-        "score": "0"
-    },
-    {
-        "id": "116",
-        "score": "1"
-    },
-    {
-        "id": "95",
-        "score": "0"
-    },
-    {
-        "id": "113",
-        "score": "1"
-    },
-    {
-        "id": "152",
-        "score": "1"
-    },
-    {
-        "id": "108",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "1"
-    },
-    {
-        "id": "154",
-        "score": "1"
-    },
-    {
-        "id": "56",
-        "score": "1"
-    },
-    {
-        "id": "110",
-        "score": "0"
-    },
-    {
-        "id": "81",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "1"
-    },
-    {
-        "id": "146",
-        "score": "1"
-    },
-    {
-        "id": "142",
-        "score": "1"
-    },
-    {
-        "id": "161",
-        "score": "1"
-    },
-    {
-        "id": "177",
-        "score": "0"
-    },
-    {
-        "id": "132",
-        "score": "1"
-    },
-    {
-        "id": "134",
-        "score": "0"
-    },
-    {
-        "id": "145",
-        "score": "1"
-    },
-    {
-        "id": "182",
-        "score": "0"
-    },
-    {
-        "id": "62",
-        "score": "1"
-    },
-    {
-        "id": "117",
-        "score": "0"
-    },
-    {
-        "id": "151",
-        "score": "0"
-    },
-    {
-        "id": "87",
-        "score": "0"
-    },
-    {
-        "id": "97",
-        "score": "0"
-    },
-    {
-        "id": "100",
-        "score": "1"
-    },
-    {
-        "id": "127",
-        "score": "1"
-    },
-    {
-        "id": "131",
-        "score": "1"
-    },
-    {
-        "id": "91",
-        "score": "1"
-    },
-    {
-        "id": "96",
-        "score": "0"
-    },
-    {
-        "id": "102",
-        "score": "0"
-    },
-    {
-        "id": "121",
-        "score": "1"
-    },
-    {
-        "id": "104",
-        "score": "1"
-    },
-    {
-        "id": "156",
-        "score": "1"
-    },
-    {
-        "id": "140",
-        "score": "1"
-    },
-    {
-        "id": "93",
-        "score": "0"
-    },
-    {
-        "id": "109",
-        "score": "0"
-    },
-    {
-        "id": "162",
-        "score": "1"
-    },
-    {
-        "id": "128",
-        "score": "1"
-    },
-    {
-        "id": "130",
-        "score": "1"
-    },
-    {
-        "id": "137",
-        "score": "0"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "148",
-        "score": "0"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "159",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "0"
-    },
-    {
-        "id": "143",
-        "score": "0"
-    },
-    {
-        "id": "153",
-        "score": "0"
-    },
-    {
-        "id": "160",
-        "score": "1"
-    },
-    {
-        "id": "99",
-        "score": "1"
-    },
-    {
-        "id": "118",
-        "score": "0"
-    },
-    {
-        "id": "133",
-        "score": "1"
-    },
-    {
-        "id": "122",
-        "score": "0"
-    },
-    {
-        "id": "164",
-        "score": "0"
-    },
-    {
-        "id": "170",
-        "score": "0"
-    },
-    {
-        "id": "136",
-        "score": "0"
-    },
-    {
-        "id": "157",
-        "score": "1"
-    },
-    {
-        "id": "112",
-        "score": "1"
-    },
-    {
-        "id": "119",
-        "score": "1"
-    },
-    {
-        "id": "89",
-        "score": "0"
-    },
-    {
-        "id": "163",
-        "score": "1"
-    },
-    {
-        "id": "198",
-        "score": "1"
-    },
-    {
-        "id": "188",
-        "score": "1"
-    },
-    {
-        "id": "194",
-        "score": "0"
-    },
-    {
-        "id": "196",
-        "score": "0"
-    },
-    {
-        "id": "147",
-        "score": "0"
-    },
-    {
-        "id": "189",
-        "score": "0"
-    },
-    {
-        "id": "193",
-        "score": "0"
-    },
-    {
-        "id": "181",
-        "score": "1"
-    },
-    {
-        "id": "191",
-        "score": "1"
-    },
-    {
-        "id": "135",
-        "score": "0"
-    },
-    {
-        "id": "168",
-        "score": "1"
-    },
-    {
-        "id": "178",
-        "score": "1"
-    },
-    {
-        "id": "184",
-        "score": "1"
-    },
-    {
-        "id": "192",
-        "score": "1"
-    },
-    {
-        "id": "167",
-        "score": "0"
-    },
-    {
-        "id": "129",
-        "score": "0"
-    },
-    {
-        "id": "169",
-        "score": "1"
-    },
-    {
-        "id": "165",
-        "score": "0"
-    },
-    {
-        "id": "187",
-        "score": "0"
-    },
-    {
-        "id": "197",
-        "score": "1"
-    },
-    {
-        "id": "172",
-        "score": "1"
-    },
-    {
-        "id": "174",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "1"
-    },
-    {
-        "id": "166",
-        "score": "0"
-    },
-    {
-        "id": "173",
-        "score": "0"
-    },
-    {
-        "id": "175",
-        "score": "1"
-    },
-    {
-        "id": "176",
-        "score": "0"
-    },
-    {
-        "id": "180",
-        "score": "0"
-    },
-    {
-        "id": "183",
-        "score": "0"
-    },
-    {
-        "id": "190",
-        "score": "0"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "186",
-        "score": "0"
-    },
-    {
-        "id": "149",
-        "score": "1"
-    },
-    {
-        "id": "155",
-        "score": "1"
-    },
-    {
-        "id": "195",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "0"
-    },
-    {
-        "id": "179",
-        "score": "0"
-    }
-]

eval_milebench/CounterfactualInference.log

@@ -1,86 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CounterfactualInference, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CounterfactualInference, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CounterfactualInference, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task CounterfactualInference, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting CounterfactualInference Using internvl
-Proceeding 5-length images samples | Num: 200
-Initialization Finished
-Predicting CounterfactualInference Using internvl
-Proceeding 5-length images samples | Num: 200
-Initialization Finished
-Predicting CounterfactualInference Using internvl
-Proceeding 5-length images samples | Num: 200
-Initialization Finished
-Predicting CounterfactualInference Using internvl
-Proceeding 5-length images samples | Num: 200
-
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:01<01:24,  1.72s/it]
  4%|▍         | 2/50 [00:02<00:56,  1.18s/it]
  6%|▌         | 3/50 [00:03<00:46,  1.02it/s]
  8%|▊         | 4/50 [00:03<00:36,  1.25it/s]
 10%|█         | 5/50 [00:04<00:28,  1.58it/s]
 12%|█▏        | 6/50 [00:04<00:22,  1.97it/s]
 14%|█▍        | 7/50 [00:04<00:19,  2.18it/s]
 16%|█▌        | 8/50 [00:05<00:17,  2.39it/s]
 18%|█▊        | 9/50 [00:05<00:16,  2.52it/s]
 20%|██        | 10/50 [00:05<00:13,  3.00it/s]
 22%|██▏       | 11/50 [00:05<00:11,  3.50it/s]
 24%|██▍       | 12/50 [00:06<00:11,  3.23it/s]
 26%|██▌       | 13/50 [00:06<00:09,  3.73it/s]
 28%|██▊       | 14/50 [00:06<00:10,  3.59it/s]
 30%|███       | 15/50 [00:06<00:08,  3.98it/s]
 32%|███▏      | 16/50 [00:06<00:07,  4.43it/s]
 34%|███▍      | 17/50 [00:07<00:06,  4.82it/s]
 36%|███▌      | 18/50 [00:07<00:06,  4.76it/s]
 38%|███▊      | 19/50 [00:07<00:06, 
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:01<01:24,  1.72s/it]
  4%|▍         | 2/50 [00:02<00:55,  1.17s/it]
  6%|▌         | 3/50 [00:03<00:45,  1.03it/s]
  8%|▊         | 4/50 [00:03<00:36,  1.27it/s]
 10%|█         | 5/50 [00:04<00:31,  1.42it/s]
 12%|█▏        | 6/50 [00:04<00:26,  1.66it/s]
 14%|█▍        | 7/50 [00:04<00:19,  2.17it/s]
 16%|█▌        | 8/50 [00:05<00:20,  2.09it/s]
 18%|█▊        | 9/50 [00:05<00:15,  2.60it/s]
 20%|██        | 10/50 [00:05<00:12,  3.10it/s]
 22%|██▏       | 11/50 [00:06<00:13,  2.79it/s]
 24%|██▍       | 12/50 [00:06<00:13,  2.77it/s]
 26%|██▌       | 13/50 [00:07<00:14,  2.51it/s]
 28%|██▊       | 14/50 [00:07<00:11,  3.05it/s]
 30%|███       | 15/50 [00:07<00:10,  3.38it/s]
 32%|███▏      | 16/50 [00:08<00:13,  2.52it/s]
 34%|███▍      | 17/50 [00:08<00:15,  2.10it/s]
 36%|███▌      | 18/50 [00:09<00:15,  2.07it/s]
 38%|███▊      | 19/50 [00:09<00:14, 
  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:01<01:24,  1.72s/it]
  4%|▍         | 2/50 [00:02<00:56,  1.17s/it]
  6%|▌         | 3/50 [00:03<00:46,  1.02it/s]
  8%|▊         | 4/50 [00:03<00:36,  1.25it/s]
 10%|█         | 5/50 [00:04<00:28,  1.60it/s]
 12%|█▏        | 6/50 [00:04<00:28,  1.57it/s]
 14%|█▍        | 7/50 [00:05<00:24,  1.77it/s]
 16%|█▌        | 8/50 [00:05<00:19,  2.11it/s]
 18%|█▊        | 9/50 [00:05<00:15,  2.61it/s]
 20%|██        | 10/50 [00:05<00:14,  2.84it/s]
 22%|██▏       | 11/50 [00:06<00:13,  3.00it/s]
 24%|██▍       | 12/50 [00:06<00:13,  2.87it/s]
 26%|██▌       | 13/50 [00:07<00:18,  2.04it/s]
 28%|██▊       | 14/50 [00:07<00:17,  2.02it/s]
 30%|███       | 15/50 [00:08<00:14,  2.48it/s]
 32%|███▏      | 16/50 [00:08<00:16,  2.10it/s]
 34%|███▍      | 17/50 [00:09<00:15,  2.07it/s]
 36%|███▌      | 18/50 [00:09<00:15,  2.13it/s]
 38%|███▊      | 19/50 [00:10<00:14,  5.09it/s]
 40%|████      | 20/50 [00:08<00:08,  3.37it/s]
 42%|████▏     | 21/50 [00:08<00:11,  2.45it/s]
 44%|████▍     | 22/50 [00:09<00:12,  2.28it/s]
 46%|████▌     | 23/50 [00:09<00:11,  2.30it/s]
 48%|████▊     | 24/50 [00:10<00:11,  2.18it/s]
 50%|█████     | 25/50 [00:10<00:10,  2.40it/s]
 52%|█████▏    | 26/50 [00:10<00:08,  2.88it/s]
 54%|█████▍    | 27/50 [00:10<00:06,  3.37it/s]
 56%|█████▌    | 28/50 [00:11<00:05,  3.83it/s]
 58%|█████▊    | 29/50 [00:11<00:04,  4.26it/s]
 60%|██████    | 30/50 [00:11<00:04,  4.66it/s]
 62%|██████▏   | 31/50 [00:11<00:03,  4.92it/s]
 64%|██████▍   | 32/50 [00:11<00:03,  5.24it/s]
 66%|██████▌   | 33/50 [00:11<00:03,  5.51it/s]
 68%|██████▊   | 34/50 [00:12<00:02,  5.71it/s]
 70%|███████   | 35/50 [00:12<00:02,  5.30it/s]
 72%|███████▏  | 36/50 [00:12<00:02,  5.5 2.07it/s]
 40%|████      | 20/50 [00:10<00:12,  2.40it/s]
 42%|████▏     | 21/50 [00:10<00:09,  2.93it/s]
 44%|████▍     | 22/50 [00:10<00:08,  3.47it/s]
 46%|████▌     | 23/50 [00:10<00:06,  3.97it/s]
 48%|████▊     | 24/50 [00:11<00:05,  4.43it/s]
 50%|█████     | 25/50 [00:11<00:05,  4.78it/s]
 52%|█████▏    | 26/50 [00:11<00:04,  5.00it/s]
 54%|█████▍    | 27/50 [00:11<00:04,  5.21it/s]
 56%|█████▌    | 28/50 [00:11<00:04,  5.29it/s]
 58%|█████▊    | 29/50 [00:12<00:03,  5.40it/s]
 60%|██████    | 30/50 [00:12<00:03,  5.09it/s]
 62%|██████▏   | 31/50 [00:12<00:03,  5.35it/s]
 64%|██████▍   | 32/50 [00:12<00:03,  5.59it/s]
 66%|██████▌   | 33/50 [00:12<00:02,  5.76it/s]
 68%|██████▊   | 34/50 [00:12<00:02,  5.94it/s]
 70%|███████   | 35/50 [00:13<00:02,  5.68it/s]
 72%|███████▏  | 36/50 [00:13<00:02,  5.8 2.21it/s]
 40%|████      | 20/50 [00:09<00:10,  2.73it/s]
 42%|████▏     | 21/50 [00:10<00:11,  2.57it/s]
 44%|████▍     | 22/50 [00:10<00:10,  2.75it/s]
 46%|████▌     | 23/50 [00:10<00:08,  3.20it/s]
 48%|████▊     | 24/50 [00:10<00:07,  3.68it/s]
 50%|█████     | 25/50 [00:11<00:06,  4.07it/s]
 52%|█████▏    | 26/50 [00:11<00:05,  4.35it/s]
 54%|█████▍    | 27/50 [00:11<00:05,  3.86it/s]
 56%|█████▌    | 28/50 [00:11<00:05,  4.26it/s]
 58%|█████▊    | 29/50 [00:11<00:04,  4.66it/s]
 60%|██████    | 30/50 [00:12<00:04,  4.99it/s]
 62%|██████▏   | 31/50 [00:12<00:03,  5.09it/s]
 64%|██████▍   | 32/50 [00:12<00:03,  5.26it/s]
 66%|██████▌   | 33/50 [00:12<00:03,  5.39it/s]
 68%|██████▊   | 34/50 [00:13<00:03,  4.12it/s]
 70%|███████   | 35/50 [00:13<00:03,  4.48it/s]
 72%|███████▏  | 36/50 [00:13<00:02,  4.86it/s]
 74%|███████▍  | 37/50 [00:12<00:02,  5.74it/s]
 76%|███████▌  | 38/50 [00:12<00:02,  5.85it/s]
 78%|███████▊  | 39/50 [00:12<00:01,  5.95it/s]
 80%|████████  | 40/50 [00:13<00:01,  5.88it/s]
 82%|████████▏ | 41/50 [00:13<00:01,  5.91it/s]
 84%|████████▍ | 42/50 [00:13<00:01,  5.81it/s]
 86%|████████▌ | 43/50 [00:13<00:01,  5.56it/s]
 88%|████████▊ | 44/50 [00:13<00:01,  5.56it/s]
 90%|█████████ | 45/50 [00:13<00:00,  5.67it/s]
 92%|█████████▏| 46/50 [00:14<00:00,  5.78it/s]
 94%|█████████▍| 47/50 [00:14<00:00,  5.84it/s]
 96%|█████████▌| 48/50 [00:14<00:00,  5.88it/s]
 98%|█████████▊| 49/50 [00:14<00:00,  5.77it/s]
-0it/s]
 74%|███████▍  | 37/50 [00:13<00:02,  5.72it/s]
 76%|███████▌  | 38/50 [00:13<00:02,  5.36it/s]
 78%|███████▊  | 39/50 [00:13<00:02,  5.49it/s]
 80%|████████  | 40/50 [00:13<00:01,  5.67it/s]
 82%|████████▏ | 41/50 [00:14<00:01,  5.80it/s]
 84%|████████▍ | 42/50 [00:14<00:01,  5.84it/s]
 86%|████████▌ | 43/50 [00:14<00:01,  5.92it/s]
 88%|████████▊ | 44/50 [00:14<00:01,  5.76it/s]
 90%|█████████ | 45/50 [00:14<00:00,  5.69it/s]
 92%|█████████▏| 46/50 [00:14<00:00,  5.81it/s]
 94%|█████████▍| 47/50 [00:15<00:00,  5.86it/s]
 96%|█████████▌| 48/50 [00:15<00:00,  5.91it/s]
 98%|█████████▊| 49/50 [00:15<00:00,  5.91it/s]
-5it/s]
 74%|███████▍  | 37/50 [00:13<00:02,  5.03it/s]
 76%|███████▌  | 38/50 [00:13<00:02,  5.31it/s]
 78%|███████▊  | 39/50 [00:13<00:02,  5.47it/s]
 80%|████████  | 40/50 [00:14<00:01,  5.62it/s]
 82%|████████▏ | 41/50 [00:14<00:01,  5.76it/s]
 84%|████████▍ | 42/50 [00:14<00:01,  4.13it/s]
 86%|████████▌ | 43/50 [00:14<00:01,  4.47it/s]
 88%|████████▊ | 44/50 [00:14<00:01,  4.81it/s]
 90%|█████████ | 45/50 [00:15<00:00,  5.08it/s]
 92%|█████████▏| 46/50 [00:15<00:00,  5.30it/s]
 94%|█████████▍| 47/50 [00:15<00:00,  5.49it/s]
 96%|█████████▌| 48/50 [00:15<00:00,  5.50it/s]
 98%|█████████▊| 49/50 [00:15<00:00,  5.73it/s]
-evaluating CounterfactualInference ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/CounterfactualInference/CounterfactualInference_240803234437.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset CounterfactualInference                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/CounterfactualInference 
-internvl:  CounterfactualInference:  {'Accuracy': 0.635, 'image_quantity_level-Accuracy': {'Few': 0.635, 'Medium': 0, 'Many': 0}, 'image_quantity_level-Result': {'Few': [127, 200], 'Medium': [0, 0], 'Many': [0, 0]}}

eval_milebench/CounterfactualInference/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.635, "image_quantity_level-Accuracy": {"Few": 0.635, "Medium": 0, "Many": 0}, "image_quantity_level-Result": {"Few": [127, 200], "Medium": [0, 0], "Many": [0, 0]}}

eval_milebench/CounterfactualInference/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "1"
-    },
-    {
-        "id": "1",
-        "score": "0"
-    },
-    {
-        "id": "2",
-        "score": "1"
-    },
-    {
-        "id": "3",
-        "score": "0"
-    },
-    {
-        "id": "4",
-        "score": "1"
-    },
-    {
-        "id": "5",
-        "score": "1"
-    },
-    {
-        "id": "6",
-        "score": "1"
-    },
-    {
-        "id": "7",
-        "score": "1"
-    },
-    {
-        "id": "8",
-        "score": "1"
-    },
-    {
-        "id": "9",
-        "score": "0"
-    },
-    {
-        "id": "10",
-        "score": "0"
-    },
-    {
-        "id": "11",
-        "score": "1"
-    },
-    {
-        "id": "12",
-        "score": "1"
-    },
-    {
-        "id": "13",
-        "score": "0"
-    },
-    {
-        "id": "14",
-        "score": "1"
-    },
-    {
-        "id": "15",
-        "score": "0"
-    },
-    {
-        "id": "16",
-        "score": "1"
-    },
-    {
-        "id": "17",
-        "score": "1"
-    },
-    {
-        "id": "18",
-        "score": "1"
-    },
-    {
-        "id": "19",
-        "score": "1"
-    },
-    {
-        "id": "20",
-        "score": "1"
-    },
-    {
-        "id": "21",
-        "score": "0"
-    },
-    {
-        "id": "22",
-        "score": "1"
-    },
-    {
-        "id": "23",
-        "score": "0"
-    },
-    {
-        "id": "24",
-        "score": "1"
-    },
-    {
-        "id": "25",
-        "score": "1"
-    },
-    {
-        "id": "26",
-        "score": "0"
-    },
-    {
-        "id": "27",
-        "score": "1"
-    },
-    {
-        "id": "28",
-        "score": "1"
-    },
-    {
-        "id": "29",
-        "score": "0"
-    },
-    {
-        "id": "30",
-        "score": "1"
-    },
-    {
-        "id": "31",
-        "score": "0"
-    },
-    {
-        "id": "32",
-        "score": "0"
-    },
-    {
-        "id": "33",
-        "score": "1"
-    },
-    {
-        "id": "34",
-        "score": "1"
-    },
-    {
-        "id": "35",
-        "score": "1"
-    },
-    {
-        "id": "36",
-        "score": "1"
-    },
-    {
-        "id": "37",
-        "score": "1"
-    },
-    {
-        "id": "38",
-        "score": "0"
-    },
-    {
-        "id": "39",
-        "score": "0"
-    },
-    {
-        "id": "40",
-        "score": "1"
-    },
-    {
-        "id": "41",
-        "score": "1"
-    },
-    {
-        "id": "42",
-        "score": "0"
-    },
-    {
-        "id": "43",
-        "score": "0"
-    },
-    {
-        "id": "44",
-        "score": "0"
-    },
-    {
-        "id": "45",
-        "score": "0"
-    },
-    {
-        "id": "46",
-        "score": "1"
-    },
-    {
-        "id": "47",
-        "score": "1"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "49",
-        "score": "1"
-    },
-    {
-        "id": "50",
-        "score": "1"
-    },
-    {
-        "id": "51",
-        "score": "1"
-    },
-    {
-        "id": "52",
-        "score": "1"
-    },
-    {
-        "id": "53",
-        "score": "1"
-    },
-    {
-        "id": "54",
-        "score": "0"
-    },
-    {
-        "id": "55",
-        "score": "1"
-    },
-    {
-        "id": "56",
-        "score": "0"
-    },
-    {
-        "id": "57",
-        "score": "1"
-    },
-    {
-        "id": "58",
-        "score": "0"
-    },
-    {
-        "id": "59",
-        "score": "1"
-    },
-    {
-        "id": "60",
-        "score": "1"
-    },
-    {
-        "id": "61",
-        "score": "1"
-    },
-    {
-        "id": "62",
-        "score": "1"
-    },
-    {
-        "id": "63",
-        "score": "0"
-    },
-    {
-        "id": "64",
-        "score": "1"
-    },
-    {
-        "id": "65",
-        "score": "1"
-    },
-    {
-        "id": "66",
-        "score": "1"
-    },
-    {
-        "id": "67",
-        "score": "1"
-    },
-    {
-        "id": "68",
-        "score": "0"
-    },
-    {
-        "id": "69",
-        "score": "0"
-    },
-    {
-        "id": "70",
-        "score": "1"
-    },
-    {
-        "id": "71",
-        "score": "1"
-    },
-    {
-        "id": "72",
-        "score": "1"
-    },
-    {
-        "id": "73",
-        "score": "1"
-    },
-    {
-        "id": "74",
-        "score": "0"
-    },
-    {
-        "id": "75",
-        "score": "1"
-    },
-    {
-        "id": "76",
-        "score": "1"
-    },
-    {
-        "id": "77",
-        "score": "0"
-    },
-    {
-        "id": "78",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "1"
-    },
-    {
-        "id": "80",
-        "score": "0"
-    },
-    {
-        "id": "81",
-        "score": "1"
-    },
-    {
-        "id": "82",
-        "score": "0"
-    },
-    {
-        "id": "83",
-        "score": "0"
-    },
-    {
-        "id": "84",
-        "score": "1"
-    },
-    {
-        "id": "85",
-        "score": "1"
-    },
-    {
-        "id": "86",
-        "score": "0"
-    },
-    {
-        "id": "87",
-        "score": "1"
-    },
-    {
-        "id": "88",
-        "score": "1"
-    },
-    {
-        "id": "89",
-        "score": "0"
-    },
-    {
-        "id": "90",
-        "score": "0"
-    },
-    {
-        "id": "91",
-        "score": "1"
-    },
-    {
-        "id": "92",
-        "score": "0"
-    },
-    {
-        "id": "93",
-        "score": "1"
-    },
-    {
-        "id": "94",
-        "score": "1"
-    },
-    {
-        "id": "95",
-        "score": "0"
-    },
-    {
-        "id": "96",
-        "score": "1"
-    },
-    {
-        "id": "97",
-        "score": "1"
-    },
-    {
-        "id": "98",
-        "score": "1"
-    },
-    {
-        "id": "99",
-        "score": "0"
-    },
-    {
-        "id": "100",
-        "score": "1"
-    },
-    {
-        "id": "101",
-        "score": "1"
-    },
-    {
-        "id": "102",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "0"
-    },
-    {
-        "id": "104",
-        "score": "0"
-    },
-    {
-        "id": "105",
-        "score": "1"
-    },
-    {
-        "id": "106",
-        "score": "0"
-    },
-    {
-        "id": "107",
-        "score": "0"
-    },
-    {
-        "id": "108",
-        "score": "0"
-    },
-    {
-        "id": "109",
-        "score": "0"
-    },
-    {
-        "id": "110",
-        "score": "1"
-    },
-    {
-        "id": "111",
-        "score": "1"
-    },
-    {
-        "id": "112",
-        "score": "1"
-    },
-    {
-        "id": "113",
-        "score": "1"
-    },
-    {
-        "id": "114",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "1"
-    },
-    {
-        "id": "116",
-        "score": "1"
-    },
-    {
-        "id": "117",
-        "score": "0"
-    },
-    {
-        "id": "118",
-        "score": "1"
-    },
-    {
-        "id": "119",
-        "score": "0"
-    },
-    {
-        "id": "120",
-        "score": "1"
-    },
-    {
-        "id": "121",
-        "score": "0"
-    },
-    {
-        "id": "122",
-        "score": "1"
-    },
-    {
-        "id": "123",
-        "score": "1"
-    },
-    {
-        "id": "124",
-        "score": "0"
-    },
-    {
-        "id": "125",
-        "score": "0"
-    },
-    {
-        "id": "126",
-        "score": "1"
-    },
-    {
-        "id": "127",
-        "score": "1"
-    },
-    {
-        "id": "128",
-        "score": "1"
-    },
-    {
-        "id": "129",
-        "score": "1"
-    },
-    {
-        "id": "130",
-        "score": "1"
-    },
-    {
-        "id": "131",
-        "score": "1"
-    },
-    {
-        "id": "132",
-        "score": "0"
-    },
-    {
-        "id": "133",
-        "score": "0"
-    },
-    {
-        "id": "134",
-        "score": "1"
-    },
-    {
-        "id": "135",
-        "score": "0"
-    },
-    {
-        "id": "136",
-        "score": "1"
-    },
-    {
-        "id": "137",
-        "score": "1"
-    },
-    {
-        "id": "138",
-        "score": "1"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "140",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "1"
-    },
-    {
-        "id": "142",
-        "score": "1"
-    },
-    {
-        "id": "143",
-        "score": "0"
-    },
-    {
-        "id": "144",
-        "score": "0"
-    },
-    {
-        "id": "145",
-        "score": "1"
-    },
-    {
-        "id": "146",
-        "score": "0"
-    },
-    {
-        "id": "147",
-        "score": "0"
-    },
-    {
-        "id": "148",
-        "score": "0"
-    },
-    {
-        "id": "149",
-        "score": "1"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "151",
-        "score": "0"
-    },
-    {
-        "id": "152",
-        "score": "0"
-    },
-    {
-        "id": "153",
-        "score": "1"
-    },
-    {
-        "id": "154",
-        "score": "1"
-    },
-    {
-        "id": "155",
-        "score": "1"
-    },
-    {
-        "id": "156",
-        "score": "1"
-    },
-    {
-        "id": "157",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "0"
-    },
-    {
-        "id": "159",
-        "score": "0"
-    },
-    {
-        "id": "160",
-        "score": "0"
-    },
-    {
-        "id": "161",
-        "score": "1"
-    },
-    {
-        "id": "162",
-        "score": "1"
-    },
-    {
-        "id": "163",
-        "score": "0"
-    },
-    {
-        "id": "164",
-        "score": "1"
-    },
-    {
-        "id": "165",
-        "score": "0"
-    },
-    {
-        "id": "166",
-        "score": "0"
-    },
-    {
-        "id": "167",
-        "score": "1"
-    },
-    {
-        "id": "168",
-        "score": "1"
-    },
-    {
-        "id": "169",
-        "score": "1"
-    },
-    {
-        "id": "170",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "1"
-    },
-    {
-        "id": "172",
-        "score": "1"
-    },
-    {
-        "id": "173",
-        "score": "1"
-    },
-    {
-        "id": "174",
-        "score": "1"
-    },
-    {
-        "id": "175",
-        "score": "0"
-    },
-    {
-        "id": "176",
-        "score": "0"
-    },
-    {
-        "id": "177",
-        "score": "0"
-    },
-    {
-        "id": "178",
-        "score": "1"
-    },
-    {
-        "id": "179",
-        "score": "1"
-    },
-    {
-        "id": "180",
-        "score": "1"
-    },
-    {
-        "id": "181",
-        "score": "1"
-    },
-    {
-        "id": "182",
-        "score": "1"
-    },
-    {
-        "id": "183",
-        "score": "0"
-    },
-    {
-        "id": "184",
-        "score": "1"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "186",
-        "score": "0"
-    },
-    {
-        "id": "187",
-        "score": "1"
-    },
-    {
-        "id": "188",
-        "score": "1"
-    },
-    {
-        "id": "189",
-        "score": "1"
-    },
-    {
-        "id": "190",
-        "score": "0"
-    },
-    {
-        "id": "191",
-        "score": "1"
-    },
-    {
-        "id": "192",
-        "score": "1"
-    },
-    {
-        "id": "193",
-        "score": "1"
-    },
-    {
-        "id": "194",
-        "score": "0"
-    },
-    {
-        "id": "195",
-        "score": "1"
-    },
-    {
-        "id": "196",
-        "score": "0"
-    },
-    {
-        "id": "197",
-        "score": "0"
-    },
-    {
-        "id": "198",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "1"
-    }
-]

eval_milebench/DocVQA.log

@@ -1,114 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task DocVQA, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task DocVQA, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task DocVQA, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task DocVQA, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Initialization Finished
-Predicting DocVQA Using internvl
-Proceeding 2-length images samples | Num: 71
-Initialization Finished
-Predicting DocVQA Using internvl
-Proceeding 2-length images samples | Num: 71
-Initialization Finished
-Predicting DocVQA Using internvl
-Proceeding 2-length images samples | Num: 71
-Initialization Finished
-Predicting DocVQA Using internvl
-Proceeding 2-length images samples | Num: 71
-
  0%|          | 0/17 [00:00<?, ?it/s]
  6%|▌         | 1/17 [00:01<00:22,  1.43s/it]
 12%|█▏        | 2/17 [00:01<00:11,  1.35it/s]
 18%|█▊        | 3/17 [00:02<00:08,  1.75it/s]
 24%|██▎       | 4/17 [00:02<00:05,  2.26it/s]
 29%|██▉       | 5/17 [00:02<00:04,  2.78it/s]
 35%|███▌      | 6/17 [00:02<00:03,  3.21it/s]
 41%|████      | 7/17 [00:02<00:02,  3.60it/s]
 47%|████▋     | 8/17 [00:03<00:02,  3.89it/s]
 53%|█████▎    | 9/17 [00:03<00:01,  4.07it/s]
 59%|█████▉    | 10/17 [00:03<00:01,  4.28it/s]
 65%|██████▍   | 11/17 [00:03<00:01,  4.35it/s]
 71%|███████   | 12/17 [00:03<00:01,  4.57it/s]
 76%|███████▋  | 13/17 [00:04<00:00,  4.50it/s]
 82%|████████▏ | 14/17 [00:04<00:00,  4.44it/s]
 88%|████████▊ | 15/17 [00:04<00:00,  4.48it/s]
 94%|█████████▍| 16/17 [00:04<00:00,  4.44it/s]
-Proceeding 5-length images samples | Num: 36
-
  0%|          | 0/18 [00:00<?, ?it/s]
  6%|▌         | 1/18 [00:01<00:24,  1.42s/it]
 11%|█         | 2/18 [00:01<00:11,  1.37it/s]
 17%|█▋        | 3/18 [00:02<00:08,  1.77it/s]
 22%|██▏       | 4/18 [00:02<00:05,  2.36it/s]
 28%|██▊       | 5/18 [00:02<00:04,  2.82it/s]
 33%|███▎      | 6/18 [00:02<00:03,  3.13it/s]
 39%|███▉      | 7/18 [00:02<00:03,  3.44it/s]
 44%|████▍     | 8/18 [00:03<00:02,  4.07it/s]
 50%|█████     | 9/18 [00:03<00:02,  4.06it/s]
 56%|█████▌    | 10/18 [00:03<00:01,  4.30it/s]
 61%|██████    | 11/18 [00:03<00:01,  4.36it/s]
 67%|██████▋   | 12/18 [00:03<00:01,  4.47it/s]
 72%|███████▏  | 13/18 [00:04<00:01,  4.38it/s]
 78%|███████▊  | 14/18 [00:04<00:00,  4.27it/s]
 83%|████████▎ | 15/18 [00:04<00:00,  4.40it/s]
 89%|████████▉ | 16/18 [00:04<00:00,  3.98it/s]
 94%|█████████▍| 17/18 [00:05<00:00,  4.14i
  0%|          | 0/18 [00:00<?, ?it/s]
  6%|▌         | 1/18 [00:01<00:24,  1.46s/it]
 11%|█         | 2/18 [00:01<00:11,  1.39it/s]
 17%|█▋        | 3/18 [00:02<00:08,  1.77it/s]
 22%|██▏       | 4/18 [00:02<00:06,  2.23it/s]
 28%|██▊       | 5/18 [00:02<00:04,  2.77it/s]
 33%|███▎      | 6/18 [00:02<00:03,  3.19it/s]
 39%|███▉      | 7/18 [00:02<00:03,  3.53it/s]
 44%|████▍     | 8/18 [00:03<00:02,  3.79it/s]
 50%|█████     | 9/18 [00:03<00:02,  3.92it/s]
 56%|█████▌    | 10/18 [00:03<00:02,  3.77it/s]
 61%|██████    | 11/18 [00:03<00:01,  4.03it/s]
 67%|██████▋   | 12/18 [00:04<00:01,  4.09it/s]
 72%|███████▏  | 13/18 [00:04<00:01,  3.84it/s]
 78%|███████▊  | 14/18 [00:04<00:01,  3.99it/s]
 83%|████████▎ | 15/18 [00:04<00:00,  4.23it/s]
 89%|████████▉ | 16/18 [00:05<00:00,  4.44it/s]
 94%|█████████▍| 17/18 [00:05<00:00,  4.49iProceeding 5-length images samples | Num: 36
-t/s]
-Proceeding 5-length images samples | Num: 36
-t/s]
-Proceeding 5-length images samples | Num: 36
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:00<00:07,  1.06it/s]
 22%|██▏       | 2/9 [00:01<00:04,  1.44it/s]
 33%|███▎      | 3/9 [00:01<00:03,  1.64it/s]
 44%|████▍     | 4/9 [00:02<00:02,  1.76it/s]
 56%|█████▌    | 5/9 [00:02<00:02,  1.84it/s]
 67%|██████▋   | 6/9 [00:03<00:01,  1.89it/s]
 78%|███████▊  | 7/9 [00:03<00:01,  1.96it/s]
 89%|████████▉ | 8/9 [00:04<00:00,  2.00it/s]
-Proceeding 3-length images samples | Num: 40
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:00<00:06,  1.28it/s]
 22%|██▏       | 2/9 [00:01<00:04,  1.63it/s]
 33%|███▎      | 3/9 [00:01<00:03,  1.79it/s]
 44%|████▍     | 4/9 [00:02<00:02,  1.93it/s]
 56%|█████▌    | 5/9 [00:02<00:02,  1.93it/s]
 67%|██████▋   | 6/9 [00:03<00:01,  1.98it/s]
 78%|███████▊  | 7/9 [00:03<00:00,  2.00it/s]
 89%|████████▉ | 8/9 [00:04<00:00,  2.12it/s]
-Proceeding 3-length images samples | Num: 40
-Proceeding 3-length images samples | Num: 40
-
  0%|          | 0/9 [00:00<?, ?it/s]
 11%|█         | 1/9 [00:00<00:06,  1.21it/s]
 22%|██▏       | 2/9 [00:01<00:04,  1.46it/s]
 33%|███▎      | 3/9 [00:02<00:04,  1.41it/s]
 44%|████▍     | 4/9 [00:02<00:03,  1.56it/s]
 56%|█████▌    | 5/9 [00:03<00:02,  1.69it/s]
 67%|██████▋   | 6/9 [00:03<00:01,  1.76it/s]
 78%|███████▊  | 7/9 [00:04<00:01,  1.83it/s]
 89%|████████▉ | 8/9 [00:04<00:00,  1.89it/s]
-Proceeding 3-length images samples | Num: 40
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:00<00:07,  1.23it/s]
 20%|██        | 2/10 [00:01<00:04,  1.63it/s]
 30%|███       | 3/10 [00:01<00:03,  2.13it/s]
 40%|████      | 4/10 [00:01<00:02,  2.49it/s]
 50%|█████     | 5/10 [00:02<00:02,  2.28it/s]
 60%|██████    | 6/10 [00:02<00:01,  2.45it/s]
 70%|███████   | 7/10 [00:03<00:01,  2.68it/s]
 80%|████████  | 8/10 [00:03<00:00,  2.85it/s]
 90%|█████████ | 9/10 [00:03<00:00,  2.88it/s]
-Proceeding 6-length images samples | Num: 24
-Proceeding 6-length images samples | Num: 24
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:00<00:08,  1.10it/s]
 20%|██        | 2/10 [00:01<00:04,  1.83it/s]
 30%|███       | 3/10 [00:01<00:02,  2.45it/s]
 40%|████      | 4/10 [00:01<00:02,  2.71it/s]
 50%|█████     | 5/10 [00:02<00:01,  2.84it/s]
 60%|██████    | 6/10 [00:02<00:01,  2.97it/s]
 70%|███████   | 7/10 [00:02<00:00,  3.03it/s]
 80%|████████  | 8/10 [00:02<00:00,  3.14it/s]
 90%|█████████ | 9/10 [00:03<00:00,  3.14it/s]
-Proceeding 6-length images samples | Num: 24
-
  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:01<00:09,  1.03s/it]
 20%|██        | 2/10 [00:02<00:09,  1.22s/it]
 30%|███       | 3/10 [00:02<00:05,  1.23it/s]
 40%|████      | 4/10 [00:03<00:03,  1.63it/s]
 50%|█████     | 5/10 [00:03<00:02,  2.00it/s]
 60%|██████    | 6/10 [00:03<00:01,  2.32it/s]
 70%|███████   | 7/10 [00:03<00:01,  2.58it/s]
 80%|████████  | 8/10 [00:04<00:00,  3.18it/s]
 90%|█████████ | 9/10 [00:04<00:00,  3.21it/s]
-Proceeding 6-length images samples | Num: 24
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:00<00:04,  1.19it/s]
 33%|███▎      | 2/6 [00:01<00:02,  1.53it/s]
 50%|█████     | 3/6 [00:01<00:01,  1.76it/s]
 67%|██████▋   | 4/6 [00:02<00:01,  1.76it/s]
 83%|████████▎ | 5/6 [00:03<00:00,  1.43it/s]
-Proceeding 4-length images samples | Num: 29
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:00<00:04,  1.13it/s]
 33%|███▎      | 2/6 [00:01<00:02,  1.42it/s]
 50%|█████     | 3/6 [00:02<00:01,  1.53it/s]
 67%|██████▋   | 4/6 [00:02<00:01,  1.31it/s]
 83%|████████▎ | 5/6 [00:03<00:00,  1.42it/s]
-Proceeding 4-length images samples | Num: 29
-Proceeding 4-length images samples | Num: 29
-
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:00<00:04,  1.25it/s]
 33%|███▎      | 2/6 [00:01<00:02,  1.80it/s]
 50%|█████     | 3/6 [00:01<00:01,  1.81it/s]
 67%|██████▋   | 4/6 [00:02<00:01,  1.42it/s]
 83%|████████▎ | 5/6 [00:03<00:00,  1.57it/s]
-Proceeding 4-length images samples | Num: 29
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:00<00:04,  1.48it/s]
 29%|██▊       | 2/7 [00:01<00:02,  1.97it/s]
 43%|████▎     | 3/7 [00:01<00:01,  2.18it/s]
 57%|█████▋    | 4/7 [00:02<00:02,  1.30it/s]
 71%|███████▏  | 5/7 [00:03<00:01,  1.54it/s]
 86%|████████▌ | 6/7 [00:03<00:00,  1.80it/s]
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:00<00:03,  1.63it/s]
 29%|██▊       | 2/7 [00:01<00:02,  2.03it/s]
 43%|████▎     | 3/7 [00:02<00:03,  1.24it/s]
 57%|█████▋    | 4/7 [00:02<00:01,  1.56it/s]
 71%|███████▏  | 5/7 [00:03<00:01,  1.72it/s]
 86%|████████▌ | 6/7 [00:03<00:00,  1.89it/s]
-
  0%|          | 0/7 [00:00<?, ?it/s]
 14%|█▍        | 1/7 [00:00<00:04,  1.47it/s]
 29%|██▊       | 2/7 [00:01<00:02,  1.73it/s]
 43%|████▎     | 3/7 [00:02<00:03,  1.09it/s]
 57%|█████▋    | 4/7 [00:02<00:02,  1.40it/s]
 71%|███████▏  | 5/7 [00:03<00:01,  1.85it/s]
 86%|████████▌ | 6/7 [00:03<00:00,  2.04it/s]
-evaluating DocVQA ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/DocVQA/DocVQA_240803234442.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset DocVQA                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/DocVQA 
-internvl:  DocVQA:  {'Accuracy': 0.61, 'image_quantity_level-Accuracy': {'Few': 0.5909090909090909, 'Medium': 0.75, 'Many': 0}, 'image_quantity_level-Result': {'Few': [104, 176], 'Medium': [18, 24], 'Many': [0, 0]}}

eval_milebench/DocVQA/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.61, "image_quantity_level-Accuracy": {"Few": 0.5909090909090909, "Medium": 0.75, "Many": 0}, "image_quantity_level-Result": {"Few": [104, 176], "Medium": [18, 24], "Many": [0, 0]}}

eval_milebench/DocVQA/eval_score.json

@@ -1,802 +0,0 @@
-[
-    {
-        "id": "0",
-        "score": "1"
-    },
-    {
-        "id": "5",
-        "score": "1"
-    },
-    {
-        "id": "7",
-        "score": "1"
-    },
-    {
-        "id": "12",
-        "score": "1"
-    },
-    {
-        "id": "14",
-        "score": "0"
-    },
-    {
-        "id": "15",
-        "score": "1"
-    },
-    {
-        "id": "20",
-        "score": "1"
-    },
-    {
-        "id": "23",
-        "score": "1"
-    },
-    {
-        "id": "34",
-        "score": "1"
-    },
-    {
-        "id": "35",
-        "score": "0"
-    },
-    {
-        "id": "36",
-        "score": "1"
-    },
-    {
-        "id": "40",
-        "score": "1"
-    },
-    {
-        "id": "42",
-        "score": "1"
-    },
-    {
-        "id": "43",
-        "score": "0"
-    },
-    {
-        "id": "45",
-        "score": "1"
-    },
-    {
-        "id": "46",
-        "score": "1"
-    },
-    {
-        "id": "49",
-        "score": "1"
-    },
-    {
-        "id": "50",
-        "score": "1"
-    },
-    {
-        "id": "1",
-        "score": "1"
-    },
-    {
-        "id": "8",
-        "score": "1"
-    },
-    {
-        "id": "11",
-        "score": "0"
-    },
-    {
-        "id": "16",
-        "score": "0"
-    },
-    {
-        "id": "19",
-        "score": "0"
-    },
-    {
-        "id": "21",
-        "score": "0"
-    },
-    {
-        "id": "22",
-        "score": "1"
-    },
-    {
-        "id": "24",
-        "score": "1"
-    },
-    {
-        "id": "29",
-        "score": "1"
-    },
-    {
-        "id": "2",
-        "score": "1"
-    },
-    {
-        "id": "3",
-        "score": "0"
-    },
-    {
-        "id": "10",
-        "score": "1"
-    },
-    {
-        "id": "13",
-        "score": "1"
-    },
-    {
-        "id": "17",
-        "score": "0"
-    },
-    {
-        "id": "26",
-        "score": "1"
-    },
-    {
-        "id": "31",
-        "score": "0"
-    },
-    {
-        "id": "38",
-        "score": "1"
-    },
-    {
-        "id": "52",
-        "score": "0"
-    },
-    {
-        "id": "54",
-        "score": "0"
-    },
-    {
-        "id": "4",
-        "score": "1"
-    },
-    {
-        "id": "6",
-        "score": "1"
-    },
-    {
-        "id": "18",
-        "score": "1"
-    },
-    {
-        "id": "32",
-        "score": "0"
-    },
-    {
-        "id": "39",
-        "score": "0"
-    },
-    {
-        "id": "55",
-        "score": "1"
-    },
-    {
-        "id": "9",
-        "score": "1"
-    },
-    {
-        "id": "25",
-        "score": "0"
-    },
-    {
-        "id": "27",
-        "score": "1"
-    },
-    {
-        "id": "28",
-        "score": "0"
-    },
-    {
-        "id": "30",
-        "score": "0"
-    },
-    {
-        "id": "33",
-        "score": "1"
-    },
-    {
-        "id": "41",
-        "score": "1"
-    },
-    {
-        "id": "48",
-        "score": "0"
-    },
-    {
-        "id": "53",
-        "score": "0"
-    },
-    {
-        "id": "56",
-        "score": "1"
-    },
-    {
-        "id": "67",
-        "score": "0"
-    },
-    {
-        "id": "68",
-        "score": "1"
-    },
-    {
-        "id": "73",
-        "score": "1"
-    },
-    {
-        "id": "78",
-        "score": "0"
-    },
-    {
-        "id": "82",
-        "score": "1"
-    },
-    {
-        "id": "88",
-        "score": "1"
-    },
-    {
-        "id": "92",
-        "score": "1"
-    },
-    {
-        "id": "94",
-        "score": "1"
-    },
-    {
-        "id": "99",
-        "score": "1"
-    },
-    {
-        "id": "100",
-        "score": "1"
-    },
-    {
-        "id": "101",
-        "score": "1"
-    },
-    {
-        "id": "102",
-        "score": "0"
-    },
-    {
-        "id": "104",
-        "score": "1"
-    },
-    {
-        "id": "107",
-        "score": "0"
-    },
-    {
-        "id": "108",
-        "score": "0"
-    },
-    {
-        "id": "112",
-        "score": "0"
-    },
-    {
-        "id": "37",
-        "score": "0"
-    },
-    {
-        "id": "44",
-        "score": "1"
-    },
-    {
-        "id": "47",
-        "score": "0"
-    },
-    {
-        "id": "66",
-        "score": "0"
-    },
-    {
-        "id": "72",
-        "score": "0"
-    },
-    {
-        "id": "83",
-        "score": "1"
-    },
-    {
-        "id": "85",
-        "score": "0"
-    },
-    {
-        "id": "93",
-        "score": "1"
-    },
-    {
-        "id": "95",
-        "score": "1"
-    },
-    {
-        "id": "59",
-        "score": "0"
-    },
-    {
-        "id": "60",
-        "score": "1"
-    },
-    {
-        "id": "61",
-        "score": "0"
-    },
-    {
-        "id": "63",
-        "score": "1"
-    },
-    {
-        "id": "64",
-        "score": "1"
-    },
-    {
-        "id": "65",
-        "score": "1"
-    },
-    {
-        "id": "71",
-        "score": "0"
-    },
-    {
-        "id": "76",
-        "score": "1"
-    },
-    {
-        "id": "77",
-        "score": "0"
-    },
-    {
-        "id": "79",
-        "score": "1"
-    },
-    {
-        "id": "57",
-        "score": "1"
-    },
-    {
-        "id": "62",
-        "score": "1"
-    },
-    {
-        "id": "69",
-        "score": "0"
-    },
-    {
-        "id": "74",
-        "score": "1"
-    },
-    {
-        "id": "75",
-        "score": "1"
-    },
-    {
-        "id": "89",
-        "score": "1"
-    },
-    {
-        "id": "51",
-        "score": "0"
-    },
-    {
-        "id": "58",
-        "score": "0"
-    },
-    {
-        "id": "70",
-        "score": "1"
-    },
-    {
-        "id": "90",
-        "score": "1"
-    },
-    {
-        "id": "105",
-        "score": "1"
-    },
-    {
-        "id": "111",
-        "score": "0"
-    },
-    {
-        "id": "119",
-        "score": "1"
-    },
-    {
-        "id": "113",
-        "score": "0"
-    },
-    {
-        "id": "114",
-        "score": "0"
-    },
-    {
-        "id": "116",
-        "score": "1"
-    },
-    {
-        "id": "117",
-        "score": "1"
-    },
-    {
-        "id": "129",
-        "score": "0"
-    },
-    {
-        "id": "130",
-        "score": "0"
-    },
-    {
-        "id": "132",
-        "score": "1"
-    },
-    {
-        "id": "133",
-        "score": "1"
-    },
-    {
-        "id": "140",
-        "score": "0"
-    },
-    {
-        "id": "143",
-        "score": "1"
-    },
-    {
-        "id": "144",
-        "score": "1"
-    },
-    {
-        "id": "148",
-        "score": "1"
-    },
-    {
-        "id": "151",
-        "score": "0"
-    },
-    {
-        "id": "154",
-        "score": "0"
-    },
-    {
-        "id": "155",
-        "score": "0"
-    },
-    {
-        "id": "157",
-        "score": "1"
-    },
-    {
-        "id": "158",
-        "score": "1"
-    },
-    {
-        "id": "159",
-        "score": "0"
-    },
-    {
-        "id": "98",
-        "score": "0"
-    },
-    {
-        "id": "106",
-        "score": "1"
-    },
-    {
-        "id": "110",
-        "score": "1"
-    },
-    {
-        "id": "123",
-        "score": "1"
-    },
-    {
-        "id": "124",
-        "score": "0"
-    },
-    {
-        "id": "127",
-        "score": "0"
-    },
-    {
-        "id": "137",
-        "score": "1"
-    },
-    {
-        "id": "142",
-        "score": "1"
-    },
-    {
-        "id": "146",
-        "score": "1"
-    },
-    {
-        "id": "80",
-        "score": "0"
-    },
-    {
-        "id": "81",
-        "score": "0"
-    },
-    {
-        "id": "84",
-        "score": "1"
-    },
-    {
-        "id": "86",
-        "score": "1"
-    },
-    {
-        "id": "87",
-        "score": "0"
-    },
-    {
-        "id": "96",
-        "score": "1"
-    },
-    {
-        "id": "109",
-        "score": "0"
-    },
-    {
-        "id": "128",
-        "score": "1"
-    },
-    {
-        "id": "141",
-        "score": "1"
-    },
-    {
-        "id": "152",
-        "score": "0"
-    },
-    {
-        "id": "91",
-        "score": "0"
-    },
-    {
-        "id": "97",
-        "score": "1"
-    },
-    {
-        "id": "103",
-        "score": "1"
-    },
-    {
-        "id": "115",
-        "score": "1"
-    },
-    {
-        "id": "118",
-        "score": "1"
-    },
-    {
-        "id": "125",
-        "score": "1"
-    },
-    {
-        "id": "120",
-        "score": "0"
-    },
-    {
-        "id": "121",
-        "score": "1"
-    },
-    {
-        "id": "122",
-        "score": "1"
-    },
-    {
-        "id": "131",
-        "score": "1"
-    },
-    {
-        "id": "134",
-        "score": "0"
-    },
-    {
-        "id": "135",
-        "score": "0"
-    },
-    {
-        "id": "145",
-        "score": "0"
-    },
-    {
-        "id": "160",
-        "score": "1"
-    },
-    {
-        "id": "164",
-        "score": "1"
-    },
-    {
-        "id": "165",
-        "score": "0"
-    },
-    {
-        "id": "166",
-        "score": "1"
-    },
-    {
-        "id": "169",
-        "score": "1"
-    },
-    {
-        "id": "172",
-        "score": "1"
-    },
-    {
-        "id": "174",
-        "score": "1"
-    },
-    {
-        "id": "176",
-        "score": "1"
-    },
-    {
-        "id": "178",
-        "score": "1"
-    },
-    {
-        "id": "179",
-        "score": "0"
-    },
-    {
-        "id": "182",
-        "score": "0"
-    },
-    {
-        "id": "185",
-        "score": "1"
-    },
-    {
-        "id": "187",
-        "score": "1"
-    },
-    {
-        "id": "191",
-        "score": "0"
-    },
-    {
-        "id": "192",
-        "score": "0"
-    },
-    {
-        "id": "194",
-        "score": "1"
-    },
-    {
-        "id": "195",
-        "score": "1"
-    },
-    {
-        "id": "147",
-        "score": "0"
-    },
-    {
-        "id": "149",
-        "score": "0"
-    },
-    {
-        "id": "153",
-        "score": "1"
-    },
-    {
-        "id": "156",
-        "score": "0"
-    },
-    {
-        "id": "175",
-        "score": "0"
-    },
-    {
-        "id": "180",
-        "score": "0"
-    },
-    {
-        "id": "186",
-        "score": "1"
-    },
-    {
-        "id": "193",
-        "score": "0"
-    },
-    {
-        "id": "198",
-        "score": "1"
-    },
-    {
-        "id": "161",
-        "score": "1"
-    },
-    {
-        "id": "162",
-        "score": "0"
-    },
-    {
-        "id": "167",
-        "score": "1"
-    },
-    {
-        "id": "171",
-        "score": "0"
-    },
-    {
-        "id": "177",
-        "score": "1"
-    },
-    {
-        "id": "181",
-        "score": "1"
-    },
-    {
-        "id": "183",
-        "score": "0"
-    },
-    {
-        "id": "184",
-        "score": "1"
-    },
-    {
-        "id": "196",
-        "score": "1"
-    },
-    {
-        "id": "199",
-        "score": "0"
-    },
-    {
-        "id": "126",
-        "score": "1"
-    },
-    {
-        "id": "136",
-        "score": "0"
-    },
-    {
-        "id": "138",
-        "score": "1"
-    },
-    {
-        "id": "139",
-        "score": "1"
-    },
-    {
-        "id": "173",
-        "score": "0"
-    },
-    {
-        "id": "188",
-        "score": "1"
-    },
-    {
-        "id": "150",
-        "score": "1"
-    },
-    {
-        "id": "163",
-        "score": "1"
-    },
-    {
-        "id": "168",
-        "score": "1"
-    },
-    {
-        "id": "170",
-        "score": "0"
-    },
-    {
-        "id": "189",
-        "score": "1"
-    },
-    {
-        "id": "190",
-        "score": "1"
-    },
-    {
-        "id": "197",
-        "score": "1"
-    }
-]

eval_milebench/EgocentricNavigation.log

@@ -1,401 +0,0 @@
-language_model.model.layers.0 4
-language_model.model.layers.1 4
-language_model.model.layers.2 4
-language_model.model.layers.3 4
-language_model.model.layers.4 4
-language_model.model.layers.5 4
-language_model.model.layers.6 4
-language_model.model.layers.7 4
-language_model.model.layers.8 4
-language_model.model.layers.9 4
-language_model.model.layers.10 4
-language_model.model.layers.11 4
-language_model.model.layers.12 4
-language_model.model.layers.13 4
-language_model.model.layers.14 4
-language_model.model.layers.15 4
-language_model.model.layers.16 4
-language_model.model.layers.17 4
-language_model.model.layers.18 4
-language_model.model.layers.19 4
-language_model.model.layers.20 4
-language_model.model.layers.21 4
-language_model.model.layers.22 4
-language_model.model.layers.23 4
-vision_model.encoder.layers.0 0
-vision_model.encoder.layers.1 0
-vision_model.encoder.layers.2 0
-vision_model.encoder.layers.3 0
-vision_model.encoder.layers.4 0
-vision_model.encoder.layers.5 0
-vision_model.encoder.layers.6 0
-vision_model.encoder.layers.7 0
-vision_model.encoder.layers.8 0
-vision_model.encoder.layers.9 0
-vision_model.encoder.layers.10 0
-vision_model.encoder.layers.11 0
-vision_model.encoder.layers.12 0
-vision_model.encoder.layers.13 0
-vision_model.encoder.layers.14 0
-vision_model.encoder.layers.15 0
-vision_model.encoder.layers.16 0
-vision_model.encoder.layers.17 0
-vision_model.encoder.layers.18 0
-vision_model.encoder.layers.19 0
-vision_model.encoder.layers.20 0
-vision_model.encoder.layers.21 0
-vision_model.encoder.layers.22 0
-vision_model.encoder.layers.23 0
-vision_model.embeddings 0
-mlp1 0
-language_model.model.tok_embeddings 4
-language_model.model.norm 4
-language_model.output 4
-language_model.model.embed_tokens 4
-language_model.lm_head 4
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task EgocentricNavigation, devices: {device(type='cuda', index=3), device(type='cuda', index=7)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task EgocentricNavigation, devices: {device(type='cuda', index=0), device(type='cuda', index=4)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task EgocentricNavigation, devices: {device(type='cuda', index=2), device(type='cuda', index=6)}
-Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
-Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task EgocentricNavigation, devices: {device(type='cuda', index=1), device(type='cuda', index=5)}
-Initialization Finished
-Predicting EgocentricNavigation Using internvl
-Proceeding 52-length images samples | Num: 8
-Initialization Finished
-Predicting EgocentricNavigation Using internvl
-Proceeding 52-length images samples | Num: 8
-Initialization Finished
-Predicting EgocentricNavigation Using internvl
-Proceeding 52-length images samples | Num: 8
-Initialization Finished
-Predicting EgocentricNavigation Using internvl
-Proceeding 52-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:07<00:07,  7.49s/it]
-Proceeding 73-length images samples | Num: 2
-Proceeding 73-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:08<00:08,  8.08s/it]
-Proceeding 73-length images samples | Num: 2
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:07<00:07,  7.88s/it]
-Proceeding 73-length images samples | Num: 2
-
-Proceeding 48-length images samples | Num: 5
-
-Proceeding 48-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 38-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 38-length images samples | Num: 9
-Proceeding 48-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 48-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.37s/it]
-Proceeding 54-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 38-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.41s/it]
-Proceeding 54-length images samples | Num: 5
-Proceeding 38-length images samples | Num: 9
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 7
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:05<00:05,  5.50s/it]
-Proceeding 54-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 33-length images samples | Num: 5
-Proceeding 54-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.07s/it]
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 30-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 53-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 53-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 44-length images samples | Num: 3
-
-Proceeding 39-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:09<00:09,  9.43s/it]
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 53-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 44-length images samples | Num: 3
-Proceeding 30-length images samples | Num: 7
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.78s/it]
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 44-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 9
-Proceeding 33-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 39-length images samples | Num: 9
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.96s/it]
-Proceeding 31-length images samples | Num: 7
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 36-length images samples | Num: 10
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.62s/it]
-Proceeding 34-length images samples | Num: 8
-Proceeding 53-length images samples | Num: 5
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:04<00:04,  4.16s/it]
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.34s/it]
-Proceeding 32-length images samples | Num: 10
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.43s/it]
-Proceeding 31-length images samples | Num: 7
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:02<00:02,  2.04s/it]
-Proceeding 31-length images samples | Num: 7
-Proceeding 44-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.48s/it]
-Proceeding 36-length images samples | Num: 10
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.06s/it]
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.67s/it]
-Proceeding 36-length images samples | Num: 10
-Proceeding 42-length images samples | Num: 3
-Proceeding 39-length images samples | Num: 9
-
-Proceeding 56-length images samples | Num: 3
-Proceeding 55-length images samples | Num: 4
-
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.11s/it]
-Proceeding 32-length images samples | Num: 10
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 46-length images samples | Num: 4
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:05,  2.98s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.82s/it]
-Proceeding 32-length images samples | Num: 10
-Proceeding 34-length images samples | Num: 8
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.57s/it]
-Proceeding 42-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 6
-Proceeding 31-length images samples | Num: 7
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:03<00:06,  3.00s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.74s/it]
-Proceeding 42-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 56-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 16
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 56-length images samples | Num: 3
-Proceeding 36-length images samples | Num: 10
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 55-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 55-length images samples | Num: 4
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:04<00:14,  4.72s/it]
 50%|█████     | 2/4 [00:05<00:04,  2.44s/it]
 75%|███████▌  | 3/4 [00:08<00:02,  2.89s/it]
-Proceeding 47-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 46-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 46-length images samples | Num: 4
-Proceeding 32-length images samples | Num: 10
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 6
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 45-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.41s/it]
-Proceeding 62-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 35-length images samples | Num: 16
-
-Proceeding 43-length images samples | Num: 4
-Proceeding 42-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 3
-
-Proceeding 58-length images samples | Num: 4
-Proceeding 56-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:01<00:01,  1.59s/it]
-Proceeding 35-length images samples | Num: 16
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 5
-Proceeding 55-length images samples | Num: 4
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:01<00:05,  1.93s/it]
 50%|█████     | 2/4 [00:03<00:03,  1.59s/it]
 75%|███████▌  | 3/4 [00:07<00:03,  3.01s/it]
-Proceeding 47-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 41-length images samples | Num: 5
-Proceeding 46-length images samples | Num: 4
-
  0%|          | 0/4 [00:00<?, ?it/s]
 25%|██▌       | 1/4 [00:04<00:14,  4.93s/it]
 50%|█████     | 2/4 [00:08<00:07,  3.85s/it]
 75%|███████▌  | 3/4 [00:09<00:02,  2.54s/it]
-Proceeding 47-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 57-length images samples | Num: 5
-Proceeding 45-length images samples | Num: 6
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.77s/it]
-Proceeding 62-length images samples | Num: 1
-
-Proceeding 43-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 11
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.63s/it]
-Proceeding 62-length images samples | Num: 1
-
-Proceeding 43-length images samples | Num: 4
-Proceeding 35-length images samples | Num: 16
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 58-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 51-length images samples | Num: 3
-
  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:03<00:03,  3.94s/it]
-Proceeding 80-length images samples | Num: 1
-
-Proceeding 90-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 5
-
-Proceeding 65-length images samples | Num: 1
-
-Proceeding 40-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 58-length images samples | Num: 4
-
-Proceeding 76-length images samples | Num: 1
-
-Proceeding 60-length images samples | Num: 1
-
-Proceeding 49-length images samples | Num: 2
-Proceeding 47-length images samples | Num: 8
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 41-length images samples | Num: 5
-
-Proceeding 70-length images samples | Num: 1
-
-Proceeding 109-length images samples | Num: 1
-
-Proceeding 69-length images samples | Num: 1
-
-Proceeding 59-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 50-length images samples | Num: 5
-
-Proceeding 68-length images samples | Num: 2
-
-Proceeding 61-length images samples | Num: 2
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 57-length images samples | Num: 5
-Proceeding 71-length images samples | Num: 1
-
-
-Proceeding 63-length images samples | Num: 2
-
-Proceeding 26-length images samples | Num: 1
-Proceeding 77-length images samples | Num: 1
-
-
-Proceeding 62-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 11
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 57-length images samples | Num: 5
-Proceeding 43-length images samples | Num: 4
-Proceeding 51-length images samples | Num: 3
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:03<00:06,  3.22s/it]
 67%|██████▋   | 2/3 [00:04<00:02,  2.11s/it]
-Proceeding 80-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 37-length images samples | Num: 11
-
-Proceeding 90-length images samples | Num: 1
-
-Proceeding 65-length images samples | Num: 1
-
-Proceeding 40-length images samples | Num: 3
-Proceeding 58-length images samples | Num: 4
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 76-length images samples | Num: 1
-
-Proceeding 60-length images samples | Num: 1
-Proceeding 50-length images samples | Num: 5
-
-Proceeding 49-length images samples | Num: 2
-
-Proceeding 70-length images samples | Num: 1
-
  0%|          | 0/3 [00:00<?, ?it/s]
 33%|███▎      | 1/3 [00:02<00:04,  2.26s/it]
 67%|██████▋   | 2/3 [00:03<00:01,  1.45s/it]
-Proceeding 80-length images samples | Num: 1
-
-Proceeding 109-length images samples | Num: 1
-
-Proceeding 90-length images samples | Num: 1
-
-Proceeding 69-length images samples | Num: 1
-
-Proceeding 65-length images samples | Num: 1
-
-Proceeding 59-length images samples | Num: 3
-
-Proceeding 40-length images samples | Num: 3
-Proceeding 41-length images samples | Num: 5
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 76-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 68-length images samples | Num: 2
-
-Proceeding 60-length images samples | Num: 1
-
-Proceeding 61-length images samples | Num: 2
-
-Proceeding 49-length images samples | Num: 2
-
-Proceeding 71-length images samples | Num: 1
-
-Proceeding 63-length images samples | Num: 2
-
-Proceeding 26-length images samples | Num: 1
-
-Proceeding 77-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 70-length images samples | Num: 1
-
-Proceeding 57-length images samples | Num: 5
-
-Proceeding 109-length images samples | Num: 1
-
-Proceeding 69-length images samples | Num: 1
-
-Proceeding 59-length images samples | Num: 3
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 68-length images samples | Num: 2
-Proceeding 37-length images samples | Num: 11
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 61-length images samples | Num: 2
-Proceeding 80-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 71-length images samples | Num: 1
-
-Proceeding 63-length images samples | Num: 2
-Proceeding 90-length images samples | Num: 1
-
  0%|          | 0/1 [00:00<?, ?it/s]
-Proceeding 26-length images samples | Num: 1
-
-Proceeding 77-length images samples | Num: 1
-
-Proceeding 65-length images samples | Num: 1
-Proceeding 40-length images samples | Num: 3
-Proceeding 76-length images samples | Num: 1
-Proceeding 60-length images samples | Num: 1
-Proceeding 49-length images samples | Num: 2
-Proceeding 70-length images samples | Num: 1
-Proceeding 109-length images samples | Num: 1
-Proceeding 69-length images samples | Num: 1
-Proceeding 59-length images samples | Num: 3
-Proceeding 68-length images samples | Num: 2
-Proceeding 61-length images samples | Num: 2
-Proceeding 71-length images samples | Num: 1
-Proceeding 63-length images samples | Num: 2
-Proceeding 26-length images samples | Num: 1
-Proceeding 77-length images samples | Num: 1
-evaluating EgocentricNavigation ...
-Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/EgocentricNavigation/EgocentricNavigation_240803234751.json
-python eval/milebench/evaluate.py                      --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d                     --dataset EgocentricNavigation                     --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/EgocentricNavigation 
-internvl:  EgocentricNavigation:  {'Accuracy': 0.33, 'image_quantity_level-Accuracy': {'Few': 0, 'Medium': 0.4, 'Many': 0.32432432432432434}, 'image_quantity_level-Result': {'Few': [0, 0], 'Medium': [6, 15], 'Many': [60, 185]}}

eval_milebench/EgocentricNavigation/eval.json

@@ -1 +0,0 @@
-{"Accuracy": 0.33, "image_quantity_level-Accuracy": {"Few": 0, "Medium": 0.4, "Many": 0.32432432432432434}, "image_quantity_level-Result": {"Few": [0, 0], "Medium": [6, 15], "Many": [60, 185]}}