Towards Game-Playing AI Benchmarks via Performance Reporting Standards
  While games have been used extensively as milestones to evaluate game-playing
AI, there exists no standardised framework for reporting the obtained
observations. As a result, it remains difficult to draw general conclusions
about the strengths and weaknesses of different game-playing AI algorithms. In
this paper, we propose reporting guidelines for AI game-playing performance
that, if followed, provide information suitable for unbiased comparisons
between different AI approaches. The vision we describe is to build benchmarks
and competitions based on such guidelines in order to be able to draw more
general conclusions about the behaviour of different AI algorithms, as well as
the types of challenges different games pose.
