Jonathan Weisberg, associate professor of philosophy at the University of Toronto and managing editor of Ergo, notes that by the time a paper is published in one journal, it has likely made the rounds at a few others, and hence has been reviewed by several people whose opinions on it are not publicly available. These people have already “thought about [the paper’s] strengths and weaknesses, and they’ve generated valuable insights and assessments that could save others time and trouble. Yet only the handling editors and the authors get the direct benefit of that labour.”
The current system even encourages authors to waste editors’ and referees’ time. Unless they’re in a rush, authors can start at the top of the journal-prestige hierarchy and work their way down. You don’t even have to perfect your paper before starting this incredibly inefficient process. With so many journals to try, you’ll basically get unlimited kicks at the can. So you might as well let the referees do your homework for you.
It’s an inefficient process that has led to an arrangement that “is sagging low under the weight of premature, mediocre, even low-quality submissions.”
What can be done? Weisberg entertains the following idea: journals should publicly assign grades to submissions—letter grades, like A, B-, D+, etc.
Here are some advantages to the proposal:
- Authors might be more realistic in deciding where to submit. They might also wait until their paper is truly ready for public consumption before imposing on editors and referees.
- Readers would also benefit from seeing a paper’s transcript. Not only could it inform their decision about whether to read the paper, it could aid their sense of how its contribution is received by peers and experts.
- Editors could even limit submissions based on their grade-history, e.g. “no submissions already graded by two other journals”, or “no submissions with an average grade less than a B”. (Ideally, different journals would have different policies here, to allow some variety.)
He considers some downsides of the proposal, too, noting that those who are disadvantaged by a lack well-developed professional networks for sharing and receiving feedback on their work prior to sending it out for review may end up with papers that have worse grade transcripts, compounding their disadvantage. Overall, he is not sure what grade to give this proposal, but, he says, “we can’t keep going as we have been.”
You can read more about the idea here.
Suggestions, modifications, criticisms, alternatives welcome.