Title: Avoiding a Tragedy of the Commons in the Peer Review Process,
Abstract: Peer review is the foundation of scientific publication, and the task of
reviewing has long been seen as a cornerstone of professional service. However,
the massive growth in the field of machine learning has put this community
benefit under stress, threatening both the sustainability of an effective
review process and the overall progress of the field. In this position paper,
we argue that a tragedy of the commons outcome may be avoided by emphasizing
the professional aspects of this service. In particular, we propose a rubric to
hold reviewers to an objective standard for review quality. In turn, we also
propose that reviewers be given appropriate incentive. As one possible such
incentive, we explore the idea of financial compensation on a per-review basis.
We suggest reasonable funding models and thoughts on long term effects.