“There are just too many papers for which editors are seeking reviews.” What can be done about that?
In the following guest post, Max Hayward (Sheffield) proposes some changes that may help address the problem.
How To Alleviate the Referee Crisis: A Proposal
by Max Hayward
Every editor in philosophy—and, by this stage, almost every author—is aware that there is a referee crisis in philosophy. There are just too many papers for which editors are seeking reviews, and the demands on philosophers to referee papers are getting unmanageable.
But as Neil Sinhababu (NUS) has pointed out, part of this is because of the low acceptance rates at philosophy journals. There are simply more very good papers than there are spots in top journals. This means that many excellent, publishable papers will be sent out for review but then rejected many times, as they bounce around different journals and referees, until they find a home.
Here’s a common dynamic. A really good paper will get sent to Top Journal A. It’s read by the Editor, who thinks “yeah this is pretty good”. The Editor then sends it to an Associate Editor, who thinks “yeah this is pretty good”. The Associate Editor and then sends it out to referees, one of whom says “great, publish!” and the other says “this is pretty good, but I have XYZ reservations”. Because the journal has a tiny acceptance rate, they reject in such cases. So the journal writes to the author saying “your paper was good, but we don’t have space for it, sorry”. Since it’s true that lots of great papers get rejected like this and go on to be published in similarly good journals, the author just thinks “better luck next time” and sends it out to Top Journal B. And then Top Journal C. And then Top Journal D—who decide to publish it.
And it can get far worse than this! A good paper that’s submitted, refereed, and rejected by nine journals before it gets accepted at the tenth may have taken up the time of ten editors, ten associate editors, and twenty referees! And most of the work of the editors and referees involved will have done little, if anything, to improve the paper. The collective upshot of low acceptance rates is massive inefficiency.
So one thing top journals could do is increase their acceptance rates. But how can they do this within their page constraints? (To an extent, the move to open access may result in publishers putting pressure on journals to print more pages anyway. And online journals also don’t have the same constraints. But there are also limits when it comes to the labor involved in production and typesetting, etc. So let’s just assume for the moment that most journals have fairly fixed page limits.)
Another trend in philosophy of the past few decades is that journal articles have gotten much longer (despite some pleas for shorter articles). To a great extent, this is a response to the referee process—the prospect of low acceptance rates (which encourage referees to be uncharitable) and the knowledge that you’re going to have to please a lot of different people, encourages authors to write in a self-defensive, teflony style, with exhaustive clarifications that could only interest the “clinically-literal minded”, and lengthy responses to the most uncharitable potential objections. As Bernard Williams pointed out, we end up publishing papers that are like houses with the scaffolding left on. Yes, scholarly rigor demands that all these issues have been thought through. But they don’t have to be in the published paper. This is often material that is not of interest to the vast majority of readers.
But here’s another trend from the past few decades—the rise of the internet. All journals have an online presence. And yet most journals do basically nothing with their websites! They just provide links to the published papers.
So here’s the proposal. Journals should systematically reduce word limits,* and accept and publish more pieces. BUT they also create an online file for each paper, where the author can publish an appendix detailing clarifications and responses to objections. These would have no word limit (or a very high word limit). These would be a part of the publication—read by editors and referees, published online in a finalized form, and maintained by the journal. But they wouldn’t go into the print edition, and they would be demarcated in the online edition so that readers could clearly see what was the main paper and what was an appendix.
Indeed, journals could even allow referees to publish their critiques online, too. This would give referees an incentive to be constructive and fair-minded. Furthermore, I know a lot of referees put a great deal of effort into their reports, and in many cases it is a great shame that these never see the light of day. While most readers won’t want to read a lengthy back-and-forth on every paper, there will be some who do—and they will be able to find it easily and immediately on the journal website.
This will require a bit more hands-on input from editors, to help authors identify which bits of their work belong in the paper and which belong in the appendix. But the overall effect will be to massively reduce the amount of work editors and referees have to do, since each excellent paper will pass through far fewer journals, and thus require much less review time.
It will also make philosophy papers much easier to read, and allow readers to keep abreast of developments in a much wider range of areas without having to wade through details that are probably only relevant to a handful of specialists who are actively writing on the topic (while still preserving those details for those who want to wade through them!). This should help guard against the move towards hyper-specialisation in philosophy, without sacrificing scholarly rigour.
And it will also provide meaningful rewards, in the form of publication of their comments, for referees, who currently get nothing for their extensive service, and make clear their scholarly contribution to the development of papers.
For these latter reasons, I want to say that even if page constraints weren’t a problem, it would still be a good idea to do this—to use journal websites to create appendices, while making the main body of most papers shorter and more focussed on the main contributions of the paper.
*An even better option would be to follow the Australasian Journal of Philosophy’s approach to word limits; AJP has a soft word limit of 8,000 words, and above that the standards for acceptance rise proportionate to length. My proposal would allow journals to run some very long pieces, while realizing that most papers don’t need to be extremely long in order to convey their core arguments and insights. Shortening most papers could also free up more space for the papers which really need to be long.
“A Little Rough Data About Journal Refereeing in Philosophy”
“How to Fix the Referee Crisis in Professional Philosophy”
“Why a Crowd-Sourced Peer-Review System Would Be Good for Philosophy”
“Should You Referee the Same Paper Twice, for Different Journals?”
“How to Write a Referee Report”
“Peer Review or Perish: The Problem of Free Riders in Philosophy“