AutoBench-Leaderboard / data /avg_latency.csv
PeterKruger's picture
Add application code, data, requirements, and ignore venv
61cd432
model_name,coding,creative writing,current news,general culture,grammar,history,logics,math,science,technology,Average (All Topics)
claude-3.5-haiku-20241022,15.48,11.25,10.95,10.37,9.97,10.82,8.59,11.27,9.6,9.76,10.8
claude-3.7-sonnet,23.57,16.24,14.73,16.55,13.08,17.49,10.55,14.46,13.77,14.83,15.53
claude-3.7-sonnet:thinking,71.38,35.11,30.36,34.95,34.29,39,58.43,85.34,34.01,35.17,45.8
deepSeek-R1,132.9,72.63,45.19,49.89,63.33,45.66,136.12,205.27,49.02,47.69,84.77
deepSeek-V3,71.53,28.24,28.23,32.61,27.73,26.69,32.57,44.95,23.61,29.47,34.57
deepSeek-V3-0324,60.1,29.61,38.87,31.23,31.3,30.26,49.36,70.08,34.02,47.99,42.28
gemini-2.0-flash-001,10.77,3.26,4.94,4.9,5.14,4.94,5.33,7.55,5.5,5.24,5.76
gemini-2.5-pro-preview-03-25,51.62,23.1,25.99,29.23,32.35,29.55,49.82,68.76,27.3,27.97,36.57
gemma-3-27b-it,57.3,18.12,26.05,21.7,24.51,25.17,23.42,40.69,34.57,28.76,30.03
gpt-4.1-mini,24.38,9.05,11.06,11.79,14.19,12.07,17.77,30.85,11.08,11.55,15.38
gpt-4o-mini,16.86,11.6,10.77,11.06,10.29,10.93,11.29,18.05,10.2,10.68,12.17
grok-2-1212,16.88,8.21,9.83,10.24,9.54,10.44,12.2,20.29,9.47,10.32,11.74
grok-3-beta,44.1,28.57,28.82,30.47,35.2,30.32,37.7,42.02,26.85,35.39,33.94
llama-3.1-Nemotron-70B-Instruct-HF,35.44,17.3,21.43,23.43,23.41,23.64,24.97,37.67,21.89,21.21,25.04
llama-3.3-70B-Instruct,42.84,19.57,26.71,33.2,26.5,27.23,31.8,42.4,32.56,27.52,31.03
llama-3_1-Nemotron-Ultra-253B-v1,70.17,23.7,23.43,24.11,31.86,21.37,80.5,116.39,24.5,22.37,43.84
llama-4-Maverick-17B-128E-Instruct-FP8,19.09,5.48,7.29,8.48,7.92,7.71,11.39,15.29,7.93,6.97,9.76
llama-4-Scout-17B-16E-Instruct,15.31,5.74,7.21,7.8,6.92,7.76,8.66,11.21,7.25,7.04,8.49
mistral-large-2411,52.36,24.34,26.9,29.83,22.78,28.73,25.14,33.72,19.13,28.87,29.18
mistral-small-24b-instruct-2501,20.56,13.11,13.6,11.5,11.19,10.28,11.95,22.39,12.28,13.1,13.99
nova-lite-v1,6.84,4.6,5.93,4.74,4.62,4.55,4.67,6.3,4.74,5.24,5.22
nova-pro-v1,9.29,6.08,4.63,5.19,5.01,4.59,5.11,7.41,4.64,4.54,5.65
o3-mini-2025-01-31,15.17,7.85,7.25,8.29,8.8,7.57,15.26,22.95,7.32,6.45,10.69
o4-mini-2025-04-16,25.56,14.83,14.82,15.96,19.49,15.95,21.58,34.85,14.75,13.26,19.1
qwen-plus,52.6,40.48,28.34,28.7,29.42,28.43,33.31,49.76,26.57,29.67,34.73