Peer Review or Perish: The Problem of Free Riders in Philosophy (guest post by Elizabeth Hannon)

“Here’s a radical suggestion, using the only weapon/motivational device editors have: If someone fails to fulfill their duties as referee, the journal will not accept submissions from that referee.”

The following is a guest post* by Elizabeth Hannon (LSE), assistant editor of the British Journal for the Philosophy of Science (BJPS), regarding “bad behavior” by referees. It puts forth for public discussion a proposal about how to address such behavior—the proposal has not yet been adopted as an official journal policy. (A version of this post first appeared on the BJPS blog, Auxiliary Hypotheses.)

Peer Review or Perish: The Problem of Free Riders in Philosophy
by Elizabeth Hannon

As any journal editor will tell you (at length, possibly via the medium of rant), the trickiest part of the job is not the papers, not the authors, and not even the typesetters. It’s the referees. It is no mean feat to secure referees who are, first, reliable in their academic judgment, second, responsive to emails, and third, willing to return reports when they say they will. But the frustrations of editors aside, the far more pressing concern is for the career prospects of early-career researchers. Jobs and funding can depend on timely decisions. Indeed, whether an early-career researcher gets to become a mid- or late-career researcher can depend on whether a decision is made in a reasonable amount of time.

Common bad behavior from referees includes (but is not limited to!):

  1. Failing to respond to invites in a timely fashion (where timeliness is calculated in days not weeks), even if it’s only to decline the invitation;
  2. Agreeing to act as referee and to return the report within an agreed timeframe (in the BJPS’s case, four weeks), only to substantially exceed this timeframe (by weeks, sometimes months) and
    a. asking for this substantial extra time for the weakest of reasons*;
    b. not communicating with the relevant editors whatsoever;
  3. Returning a report long past the agreed timeframe, and that report being almost useless;
  4. Not returning the report and not responding to emails enquiring about the report.

Opinions differ on the obligations of academics as referees. Is it unpaid labour, an act of charity towards the community that ought only to be gratefully received? As much a part of the job as teaching and writing? Something in between? Whatever the answer, authors need more from referees than they ever have done; more depends on papers being reviewed in a professional, timely manner. And at the very least, there’s a ‘pay it forward’ case to be made: A paper sent to the BJPS that isn’t desk rejected can be expected to be read by at least six people (and that’s not counting the work that goes into any resubmissions). For every paper an author submits, other people have attended to their work in detail. The author, qua referee, might be expected to return the favour.

I’ve been lucky to witness some extremely productive philosophical engagement between authors and referees. When it’s good, it’s so good. The only shame is that so much of this is hidden. The process viewed en masse—the view one gets as an editor—is primarily one of cooperation and collegiality, and it’s a wonder that puts the lie to the notion of philosophy as anything like an individualistic endeavour.

But what to do about the bad referees, the system’s free riders? Relentless pestering and various forms of emotional blackmail fall on deaf ears. At the BJPS, we operate a flag system for persistent offenders, but all this amounts to is bad referees being asked to perform fewer reviews, while good referees carry more of the load.

So here’s a radical suggestion, using the only weapon/motivational device editors have: If someone fails to fulfill their duties as referee, the journal will not accept submissions from that referee, for some period of time to be determined. The time period should reflect the severity of the dereliction of duties. For instance, agreeing to act as a referee but then disappearing off the radar might warrant the most substantial ban. Delivering a meager report that’s extremely late, and without communicating with the relevant editor about the delay, might mean some shorter period of time on the bench. First-time offenders surely deserve different treatment to persistent re-offenders. And the embargo period will need to be substantial enough to be effective (too short and it will have no real impact; too long and it’s probably not practical due to the changes in the editorial team). The details can be ironed out.

It’s not just badly behaved referees that stand to suffer here. There’s a risk for the journal in question too: bad referees aren’t necessarily bad authors, and we risk losing good papers to other journals by refusing those authors’ papers. But the problem is so rife and its upshot so dire for early-career researchers that maybe something more radical is required to make clear what is expected of referees and ameliorate, at least to some degree, the problem of free-riders. All thoughts on this proposal very welcome!

— — —

* ‘I decided to go on holidays’ and ‘I have other deadlines that I decided to prioritise after agreeing to referee this paper’ are the problems, not the excuse. On the other hand, there are perfectly good reasons to be delayed in returning a report. Not only do we understand, we’ve been there. You are not the droids we’re looking for.

Art: Yelitza Diaz, “Transformation” (installation) (photo by J. Weinberg)

Your email address will not be published. Required fields are marked *

Please enter an e-mail address