Improving the Peer Review Process

Peer-reviewed journals have the great potential to improve the quality of published papers. Most scholars value them for this reason. But how can we make the process better? Bee has been doing a very nice series on improving peer review over at Backreaction. My favorite two of her many suggestions are:

• Online interface for anonymous author-reviewer communication. Why keep the slow (and frankly archaic) editor-mediated communication between author and reviewer, when everyone has access to the interwebs? An anonymized online interface would be quicker, easier, and more useful. In particular, it would allow for quick clarificatory questions, and even back-and-forth discussions of important results between author and reviewer, before a finalized report is submitted to the editor.
• Incentives for high-quality reviews. Most journals don’t offer you an incentive to do a good job in a timely manner. A monetary compensation for well-done reviews is the obvious thing to do. Authors might even be willing to pay a submission fee for this cause, if it improves the quality and timeliness of the report (I know I would!).
Journals in many disciplines, including philosophy and HPS, could immediately adopt the first idea to great benefit. The second idea requires some structural changes to raise the funds. To this end, I’d only suggest:
• Stop printing paper-copies of journals. It’s a waste of money. Everybody prefers to access articles electronically anyway. The primary role of a journal should be as a peer-review agency. Spend the money on incentives for high-quality reviews instead.
Any other ideas out there?

Soul Physics is authored by Bryan W. Roberts. Thanks for subscribing.
Want more Soul Physics? Try the Soul Physics Tweet.

3 thoughts on “Improving the Peer Review Process”

1. Bee

Thanks for the link. I disagree on stopping printing paper-copies of journals. I submit my papers for publication because I want them to be printed.

2. Bryan

You want them to be printed or you want them to be read? I just don’t see what purpose the former could possibly serve if the latter is what’s at stake.

3. Anonymous

There’s been a cool proposal floating around, that for scientific studies, the introduction and methods section should be submitted and pass review *before* the results are in. Some people there there is a strong bias towards positive results in the literature, meaning that either papers with (useful!) negative results are rejected, or results are cherry-picked to support hypotheses. Presumably, these problems should be avoided.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 1, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.18010106682777405, "perplexity": 2269.3245239009657}, "config": {"markdown_headings": false, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2015-35/segments/1440645294817.54/warc/CC-MAIN-20150827031454-00249-ip-10-171-96-226.ec2.internal.warc.gz"}