Three professors have published a brief guide to refereeing papers. Though based in business schools, Jonathan Berk (Stanford), Campbell Harvey (Duke), and David Hirshleifer (UC Irvine) have produced a document that provides good general advice for referees across the disciplines.

Some excerpts:

The main job of the referee is not:
1)  To help write the paper as a quasi‐coauthor
2)  Make an unpublishable paper publishable by directing the research
3)  To ensure that the paper cites the referee’s work…

Take a scientific stance in your report. Do not insult the authors, or use overly emotional or accusatory language. Avoid ascribing bad intent to authors (“The authors were trying for a cheap publication,” “The authors were trying to brush past literature/conflicting findings under the rug…”) and focus on the substance of the paper…

It is not your job to “make work” for the authors…

It is not your job to be copy editor…

After completing your draft report, please reread it carefully and think about what a critic might make of your arguments. Too often, reports make inconsistent arguments in different parts of the report. This is just one symptom of the common problem of not thinking things through adequately. The authors have typically spent months on their paper. It is easy to form quick opinions, but if this is done casually it is unlikely to result in a valid critique of work done by good authors…

Beware of the behavioral bias of looking for evidence that confirms your view (supports your prior research) and discounting evidence that goes against your view (undermines your prior research). Your job is to determine whether the research will be of interest to the profession. It is not to promote or shut down discussion on a particular agenda…

Good research changes people’s beliefs‐‐outstanding research even more so.  That means that simply having a strong prior against the conclusions of the research is not a reason for rejection. If you recommend rejection, make sure it is based upon strong arguments rather than strong priors.

The whole piece, “Preparing a Referee Report: Guidelines and Perspectives,” can be downloaded here.

(via MR)

(image: detail of “99° (Fever) Hold Me Tight” by Serena Bocchino)