Model	Open Ended VQA: % Human Rating	Multiple Choice VQA: % Accuracy	Hints-Multiple Choice VQA: % Accuracy 	Attributions-Multiple Choice VQA: % Accuracy 	Refernce Based-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings	Refernce Free-Automatic Evaluation: Accuracy of Judge Prediction Compared to Human Ratings	Automatic Evaluation: % Auto-Rater Ratings	Hints-Automatic Evaluation: % Auto-Rater Ratings	Attributions-Automatic Evaluation: % Auto-Rater Ratings
Humans	82						78		
Gemini Pro 1.5	40	38	66	72	87	52	53	62	29
Gemini Pro Vision	30	41	62		75	38	34	47	
GPT4	34	45	69	82	86	51	38	61	25
LlaVA-1.6-34B	15	24	30		76	43	21	16	
LlaVA-1.5-7B	13	17	29		70	35	19	30	
InstructBlip	13						20	28	
Gemini Pro 1.5 Caption _ Gemini Pro 1.5	23								
Human (Oracle) Caption _ Gemini Pro 1.5	50								
Claude 3.5 Sonnet		46	45				39		
GPT4o		55	83				50		
Qwen-VL-Max		35	53				26		
Molmo-7B		34	42				36