| The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. | |
| The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. | |
| language_model.model.layers.0 4 | |
| language_model.model.layers.1 4 | |
| language_model.model.layers.2 4 | |
| language_model.model.layers.3 4 | |
| language_model.model.layers.4 4 | |
| language_model.model.layers.5 4 | |
| language_model.model.layers.6 4 | |
| language_model.model.layers.7 4 | |
| language_model.model.layers.8 4 | |
| language_model.model.layers.9 4 | |
| language_model.model.layers.10 4 | |
| language_model.model.layers.11 4 | |
| language_model.model.layers.12 4 | |
| language_model.model.layers.13 4 | |
| language_model.model.layers.14 4 | |
| language_model.model.layers.15 4 | |
| language_model.model.layers.16 4 | |
| language_model.model.layers.17 4 | |
| language_model.model.layers.18 4 | |
| language_model.model.layers.19 4 | |
| language_model.model.layers.20 4 | |
| language_model.model.layers.21 4 | |
| language_model.model.layers.22 4 | |
| language_model.model.layers.23 4 | |
| vision_model.encoder.layers.0 0 | |
| vision_model.encoder.layers.1 0 | |
| vision_model.encoder.layers.2 0 | |
| vision_model.encoder.layers.3 0 | |
| vision_model.encoder.layers.4 0 | |
| vision_model.encoder.layers.5 0 | |
| vision_model.encoder.layers.6 0 | |
| vision_model.encoder.layers.7 0 | |
| vision_model.encoder.layers.8 0 | |
| vision_model.encoder.layers.9 0 | |
| vision_model.encoder.layers.10 0 | |
| vision_model.encoder.layers.11 0 | |
| vision_model.encoder.layers.12 0 | |
| vision_model.encoder.layers.13 0 | |
| vision_model.encoder.layers.14 0 | |
| vision_model.encoder.layers.15 0 | |
| vision_model.encoder.layers.16 0 | |
| vision_model.encoder.layers.17 0 | |
| vision_model.encoder.layers.18 0 | |
| vision_model.encoder.layers.19 0 | |
| vision_model.encoder.layers.20 0 | |
| vision_model.encoder.layers.21 0 | |
| vision_model.encoder.layers.22 0 | |
| vision_model.encoder.layers.23 0 | |
| vision_model.embeddings 0 | |
| mlp1 0 | |
| language_model.model.tok_embeddings 4 | |
| language_model.model.norm 4 | |
| language_model.output 4 | |
| language_model.model.embed_tokens 4 | |
| language_model.lm_head 4 | |
| The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. | |
| The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| Rank [2] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ObjectExistence, devices: {device(type='cuda', index=2), device(type='cuda', index=6)} | |
| Initialization Finished | |
| Predicting ObjectExistence Using internvl | |
| Proceeding 5-length images samples | Num: 200 | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| Rank [1] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ObjectExistence, devices: {device(type='cuda', index=1), device(type='cuda', index=5)} | |
| Initialization Finished | |
| Predicting ObjectExistence Using internvl | |
| Proceeding 5-length images samples | Num: 200 | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| Rank [3] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ObjectExistence, devices: {device(type='cuda', index=3), device(type='cuda', index=7)} | |
| Initialization Finished | |
| Predicting ObjectExistence Using internvl | |
| Proceeding 5-length images samples | Num: 200 | |
| Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. | |
| Rank [0] Begin to eval model work_dirs/share_internvl/InternVL2-2B on task ObjectExistence, devices: {device(type='cuda', index=0), device(type='cuda', index=4)} | |
| Initialization Finished | |
| Predicting ObjectExistence Using internvl | |
| Proceeding 5-length images samples | Num: 200 | |
| 0%| | 0/50 [00:00<?, ?it/s] 2%|▏ | 1/50 [00:01<01:15, 1.55s/it] 4%|▍ | 2/50 [00:01<00:35, 1.36it/s] 6%|▌ | 3/50 [00:02<00:26, 1.80it/s] 8%|▊ | 4/50 [00:02<00:18, 2.47it/s] 10%|█ | 5/50 [00:02<00:22, 2.01it/s] 12%|█▏ | 6/50 [00:03<00:18, 2.37it/s] 14%|█▍ | 7/50 [00:03<00:14, 2.95it/s] 16%|█▌ | 8/50 [00:03<00:11, 3.53it/s] 18%|█▊ | 9/50 [00:03<00:10, 4.05it/s] 20%|██ | 10/50 [00:04<00:12, 3.26it/s] 22%|██▏ | 11/50 [00:04<00:13, 2.98it/s] 24%|██▍ | 12/50 [00:04<00:10, 3.47it/s] 26%|██▌ | 13/50 [00:04<00:09, 3.97it/s] 28%|██▊ | 14/50 [00:05<00:11, 3.01it/s] 30%|███ | 15/50 [00:05<00:13, 2.53it/s] 32%|███▏ | 16/50 [00:06<00:14, 2.31it/s] 34%|███▍ | 17/50 [00:06<00:11, 2.84it/s] 36%|███▌ | 18/50 [00:06<00:09, 3.33it/s] 38%|███▊ | 19/50 [00:06<00:08, 0%| | 0/50 [00:00<?, ?it/s] 2%|▏ | 1/50 [00:01<01:18, 1.59s/it] 4%|▍ | 2/50 [00:01<00:36, 1.32it/s] 6%|▌ | 3/50 [00:02<00:25, 1.86it/s] 8%|▊ | 4/50 [00:02<00:18, 2.49it/s] 10%|█ | 5/50 [00:02<00:21, 2.12it/s] 12%|█▏ | 6/50 [00:03<00:21, 2.02it/s] 14%|█▍ | 7/50 [00:03<00:17, 2.53it/s] 16%|█▌ | 8/50 [00:03<00:17, 2.47it/s] 18%|█▊ | 9/50 [00:04<00:16, 2.46it/s] 20%|██ | 10/50 [00:04<00:13, 3.01it/s] 22%|██▏ | 11/50 [00:04<00:11, 3.53it/s] 24%|██▍ | 12/50 [00:05<00:13, 2.75it/s] 26%|██▌ | 13/50 [00:05<00:15, 2.41it/s] 28%|██▊ | 14/50 [00:06<00:15, 2.36it/s] 30%|███ | 15/50 [00:06<00:12, 2.80it/s] 32%|███▏ | 16/50 [00:06<00:10, 3.29it/s] 34%|███▍ | 17/50 [00:07<00:13, 2.39it/s] 36%|███▌ | 18/50 [00:07<00:11, 2.91it/s] 38%|███▊ | 19/50 [00:08<00:12, 0%| | 0/50 [00:00<?, ?it/s] 2%|▏ | 1/50 [00:01<01:19, 1.62s/it] 4%|▍ | 2/50 [00:02<00:43, 1.11it/s] 6%|▌ | 3/50 [00:02<00:27, 1.74it/s] 8%|▊ | 4/50 [00:02<00:27, 1.65it/s] 10%|█ | 5/50 [00:03<00:22, 2.05it/s] 12%|█▏ | 6/50 [00:03<00:17, 2.57it/s] 14%|█▍ | 7/50 [00:03<00:13, 3.16it/s] 16%|█▌ | 8/50 [00:03<00:11, 3.74it/s] 18%|█▊ | 9/50 [00:04<00:12, 3.28it/s] 20%|██ | 10/50 [00:04<00:15, 2.64it/s] 22%|██▏ | 11/50 [00:04<00:12, 3.17it/s] 24%|██▍ | 12/50 [00:05<00:14, 2.63it/s] 26%|██▌ | 13/50 [00:05<00:16, 2.31it/s] 28%|██▊ | 14/50 [00:06<00:17, 2.11it/s] 30%|███ | 15/50 [00:06<00:13, 2.63it/s] 32%|███▏ | 16/50 [00:06<00:10, 3.12it/s] 34%|███▍ | 17/50 [00:07<00:13, 2.37it/s] 36%|███▌ | 18/50 [00:08<00:15, 2.12it/s] 38%|███▊ | 19/50 [00:08<00:11, 3.86it/s] 40%|████ | 20/50 [00:07<00:10, 2.97it/s] 42%|████▏ | 21/50 [00:08<00:12, 2.36it/s] 44%|████▍ | 22/50 [00:08<00:09, 2.83it/s] 46%|████▌ | 23/50 [00:08<00:07, 3.38it/s] 48%|████▊ | 24/50 [00:08<00:06, 3.91it/s] 50%|█████ | 25/50 [00:08<00:05, 4.40it/s] 52%|█████▏ | 26/50 [00:08<00:05, 4.48it/s] 54%|█████▍ | 27/50 [00:09<00:04, 4.79it/s] 56%|█████▌ | 28/50 [00:09<00:05, 4.25it/s] 58%|█████▊ | 29/50 [00:09<00:04, 4.71it/s] 60%|██████ | 30/50 [00:09<00:03, 5.09it/s] 62%|██████▏ | 31/50 [00:09<00:03, 5.36it/s] 64%|██████▍ | 32/50 [00:10<00:03, 5.54it/s] 66%|██████▌ | 33/50 [00:10<00:03, 5.61it/s] 68%|██████▊ | 34/50 [00:10<00:02, 5.70it/s] 70%|███████ | 35/50 [00:10<00:02, 5.84it/s] 72%|███████▏ | 36/50 [00:10<00:02, 5.9 2.40it/s] 40%|████ | 20/50 [00:08<00:10, 2.87it/s] 42%|████▏ | 21/50 [00:08<00:08, 3.35it/s] 44%|████▍ | 22/50 [00:08<00:07, 3.82it/s] 46%|████▌ | 23/50 [00:08<00:06, 4.25it/s] 48%|████▊ | 24/50 [00:08<00:05, 4.58it/s] 50%|█████ | 25/50 [00:09<00:06, 3.85it/s] 52%|█████▏ | 26/50 [00:09<00:05, 4.28it/s] 54%|█████▍ | 27/50 [00:09<00:04, 4.67it/s] 56%|█████▌ | 28/50 [00:09<00:04, 4.98it/s] 58%|█████▊ | 29/50 [00:10<00:04, 5.18it/s] 60%|██████ | 30/50 [00:10<00:03, 5.29it/s] 62%|██████▏ | 31/50 [00:10<00:03, 5.35it/s] 64%|██████▍ | 32/50 [00:10<00:03, 5.46it/s] 66%|██████▌ | 33/50 [00:10<00:03, 5.61it/s] 68%|██████▊ | 34/50 [00:10<00:02, 5.77it/s] 70%|███████ | 35/50 [00:11<00:02, 5.89it/s] 72%|███████▏ | 36/50 [00:11<00:03, 4.5 2.61it/s] 40%|████ | 20/50 [00:08<00:09, 3.15it/s] 42%|████▏ | 21/50 [00:08<00:07, 3.70it/s] 44%|████▍ | 22/50 [00:08<00:06, 4.14it/s] 46%|████▌ | 23/50 [00:08<00:06, 4.27it/s] 48%|████▊ | 24/50 [00:09<00:06, 3.85it/s] 50%|█████ | 25/50 [00:09<00:06, 4.10it/s] 52%|█████▏ | 26/50 [00:09<00:05, 4.53it/s] 54%|█████▍ | 27/50 [00:09<00:04, 4.92it/s] 56%|█████▌ | 28/50 [00:09<00:04, 5.24it/s] 58%|█████▊ | 29/50 [00:10<00:03, 5.41it/s] 60%|██████ | 30/50 [00:10<00:03, 5.47it/s] 62%|██████▏ | 31/50 [00:10<00:03, 5.48it/s] 64%|██████▍ | 32/50 [00:10<00:03, 5.55it/s] 66%|██████▌ | 33/50 [00:10<00:03, 5.60it/s] 68%|██████▊ | 34/50 [00:10<00:02, 5.65it/s] 70%|███████ | 35/50 [00:11<00:03, 4.30it/s] 72%|███████▏ | 36/50 [00:11<00:03, 4.47it/s] 74%|███████▍ | 37/50 [00:10<00:02, 6.08it/s] 76%|███████▌ | 38/50 [00:11<00:01, 6.19it/s] 78%|███████▊ | 39/50 [00:11<00:02, 4.54it/s] 80%|████████ | 40/50 [00:11<00:02, 3.89it/s] 82%|████████▏ | 41/50 [00:12<00:02, 3.21it/s] 84%|████████▍ | 42/50 [00:12<00:02, 3.30it/s] 86%|████████▌ | 43/50 [00:12<00:01, 3.61it/s] 88%|████████▊ | 44/50 [00:12<00:01, 4.15it/s] 90%|█████████ | 45/50 [00:13<00:01, 4.59it/s] 92%|█████████▏| 46/50 [00:13<00:00, 4.95it/s] 94%|█████████▍| 47/50 [00:13<00:00, 3.59it/s] 96%|█████████▌| 48/50 [00:14<00:00, 2.81it/s] 98%|█████████▊| 49/50 [00:14<00:00, 2.97it/s] 100%|██████████| 50/50 [00:14<00:00, 3.50it/s] 100%|██████████| 50/50 [00:14<00:00, 3.40it/s] | |
| 7it/s] 74%|███████▍ | 37/50 [00:11<00:03, 3.94it/s] 76%|███████▌ | 38/50 [00:12<00:03, 3.32it/s] 78%|███████▊ | 39/50 [00:12<00:03, 3.39it/s] 80%|████████ | 40/50 [00:12<00:02, 3.77it/s] 82%|████████▏ | 41/50 [00:12<00:02, 4.26it/s] 84%|████████▍ | 42/50 [00:12<00:01, 4.69it/s] 86%|████████▌ | 43/50 [00:13<00:01, 5.00it/s] 88%|████████▊ | 44/50 [00:13<00:01, 3.46it/s] 90%|█████████ | 45/50 [00:14<00:01, 2.55it/s] 92%|█████████▏| 46/50 [00:14<00:01, 2.96it/s] 94%|█████████▍| 47/50 [00:14<00:00, 3.48it/s] 96%|█████████▌| 48/50 [00:14<00:00, 4.00it/s] 98%|█████████▊| 49/50 [00:14<00:00, 4.50it/s] 100%|██████████| 50/50 [00:15<00:00, 4.93it/s] 100%|██████████| 50/50 [00:15<00:00, 3.30it/s] | |
| 7it/s] 74%|███████▍ | 37/50 [00:11<00:02, 4.40it/s] 76%|███████▌ | 38/50 [00:12<00:03, 3.54it/s] 78%|███████▊ | 39/50 [00:12<00:03, 3.56it/s] 80%|████████ | 40/50 [00:12<00:02, 3.68it/s] 82%|████████▏ | 41/50 [00:12<00:02, 4.09it/s] 84%|████████▍ | 42/50 [00:13<00:01, 4.41it/s] 86%|████████▌ | 43/50 [00:13<00:02, 3.22it/s] 88%|████████▊ | 44/50 [00:14<00:02, 2.91it/s] 90%|█████████ | 45/50 [00:14<00:01, 2.79it/s] 92%|█████████▏| 46/50 [00:14<00:01, 3.31it/s] 94%|█████████▍| 47/50 [00:14<00:00, 3.76it/s] 96%|█████████▌| 48/50 [00:14<00:00, 4.26it/s] 98%|█████████▊| 49/50 [00:15<00:00, 4.73it/s] 100%|██████████| 50/50 [00:15<00:00, 5.06it/s] 100%|██████████| 50/50 [00:15<00:00, 3.27it/s] | |
| evaluating ObjectExistence ... | |
| Results saved to work_dirs/share_internvl/InternVL2-2B/eval_milebench/ObjectExistence/ObjectExistence_240803234647.json | |
| python eval/milebench/evaluate.py --data-dir /mnt/inspurfs/share_data/wangweiyun/share_data/long-context-benchmark/MileBench/datasets--FreedomIntelligence--MileBench/snapshots/53c7a58051ef88bacf76541d91f03f5ba2d71e7d --dataset ObjectExistence --result-dir work_dirs/share_internvl/InternVL2-2B/eval_milebench/ObjectExistence | |
| internvl: ObjectExistence: {'Accuracy': 0.815, 'image_quantity_level-Accuracy': {'Few': 0.815, 'Medium': 0, 'Many': 0}, 'image_quantity_level-Result': {'Few': [163, 200], 'Medium': [0, 0], 'Many': [0, 0]}} | |