Nature to Include Reviewer Reports & Author Replies with Published Articles
From now on, “new submissions of manuscripts that are published as research articles in Nature will automatically include a link to the reviewers’ reports and author responses.”
The identity of the reviewers will not be revealed, unless the reviewers themselves choose otherwise.
For several years, authors at Nature have had the option to have the peer-review file for the manuscript accompany their publication. Making this the standard mode of publication, Nature‘s editors write, helps achieve their aim “to open up what many see as the ‘black box’ of science, shedding light on how a research paper is made. This serves to increase transparency and (we hope) to build trust in the scientific process.”
Is this “open” or “transparent” peer review a model that should be standard for philosophy journals ? There are a few reasons in favor:
- “shedding light” on the process (e.g., makes the collaborative aspects of philosophy more visible)
- better record of the sources of ideas/moves for intellectual history (especially if the reviewers choose to reveal their identity)
- better understanding of why a paper is the way it is
- possible incentive for better peer-reviewer behavior.
Discussion welcome. (See also “The open peer review experiment in Educational Philosophy and Theory” with reflections from Michael A. Peters, Susanne Brighouse, Marek Tesar, Sean Sturm, and Liz Jackson.)
Many people complain too much is published nowadays so they do not have time to read all those papers. So now instead of having to read too many papers we should read the referee reports of those too many papers as well?
You don’t *have* to read any of this.
Presumably what Räsänen means is that you have to read more than you have time to do, generally speaking, to keep up with research in your sub-field – let alone to digest and reflect on it to an appropriate extent.
Of course, no one has to do that in the sense that no one is morally obliged to be a researcher in the first place, but insofar as you want to do your due diligence and want your arguments to be more than bite-sized, you kind of have to read an awful lot and arguably too much already. Hence, arguably, being a good researcher does entail reading borderline too much already.
I am not sure if this is a compelling argument for not publishing peer reviews and replies to reviewers, but I certainly see the force of the point. Philosophy would do well with fewer words to read, at least as long as we can maintain or – ideally – increase the quality of the words you do have to read.
The reviews are made available to those who want to read them. Those who don’t want to read them have literally nothing more to read than they already had.
True. But you don’t have to read all the papers people publish either. We should remember this the next time someone complains there are too many papers published in philosophy.
That’s right. The difference is, if you want to stay current on the literature, you have to know of the papers, and maybe read them. You don’t have to read the reviews.
No, we shouldn’t. It’s just about having the option. (I sure hope you never get to review any of my papers)
From a reader’s perspective, I’d find it useful to have the referee reports available. But I am worried that this may further disincline reviewers to write reports, even if they can remain anonymous.
I wonder how portable this is to philosophy. Nature gives referees no more than two weeks to complete their reports. Under our current system, philosophers take a good deal longer than this. Knowing their reports will be scrutinised more widely, even if anonymously, we might expect philosophers to take even longer again.
I also wondered about whether this system allows for editorial intervention. Let’s say a referee writes a report that’s hugely insightful about how to improve an argument, and the editor is keen that the author take these comments on board. But the same report is also littered with condensation, personal remarks, and so on. Responsible editors might remove such irrelevant content, sharing with the author only the material relevant to their paper. If reports are to be published, presumably editors will either have to publish the reports intact, to the potential detriment of the author, or engage the referee in some back-and-forth to edit the review. This would also add delays to the process (and quite likely sour the relationship with at least some of the referees).
And I’m not convinced that reports being published would prevent referees writing these sorts of reports. Not because referees remain anonymous, but because they don’t think they are doing anything wrong.
I’m curious too about dealing with errors in referee reports and the ‘hygiene’ of our info environment. Referee reports are often dashed off (in terms of their writing; not necessarily in the thought behind them, I hasten to add!). Minimally, typos abound. Will there be corrections issued if, say, a referee report messes up a citation? Or will errors be left in official archives? And, of course, dealing with these things will pile on the work for editors, adding another source of friction slowing down the peer review process.
Everyone will finally be able to see what an odious oik reviewer 2 is.
Hermias,
Referee 2 is not always the bad guy. R2 just did me a great favour. I publish in different fields, including interdisciplinary work (whatever that is). Recently, I submitted a paper that R1 did not like – a classic rude and dismiss referee report. But R2 not only praised it as a significant contribution, they also identified themself in the report (in a slightly veiled) as a very important researcher on the topic (a clear leader). Fortunately, the Editor used their good judgment and gave greater weight to R2’s report. (I never even imagined someone of R2’s stature refereeing my paper for this journal). So say something nice about R2 in your prayers tonight 😉
Gorm, I think that just means that the reviews were mislabeled. The awful reviewer is ALWAYS Reviewer #2. It is a metaphysical fact.
Some counters in the spirit of discussion:
First point: Transparency may serve a purpose in fields like medicine or climate science, where public trust is an issue. But philosophy journals are read almost exclusively by philosophers, who already understand the review process. In that context, what does transparency actually accomplish? Philosophers know that papers reflect dialogue and revision, even if that process isn’t fully visible.
Second point: Treating referee reports as part of a paper’s intellectual history could change how reviewers write. Some might craft mini-essays or arguments of their own, especially if they sign their names. That blurs the line between reviewing and publishing. Maybe that’s a good thing, but it’s worth asking: do we really need to trace who suggested what?
Third point: I’m not sure publishing revise-and-resubmit suggestions adds much value. The final paper should be assessed on its merits. Occasionally it’s interesting to see how a paper evolved, but readers shouldn’t need to audit its full path to publication.
Fourth point: Openness might encourage performance. If reviewers imagine a broader audience, feedback may become less candid and more stylized. That seems like a step away from the kind of honest criticism revision depends on. I’ve heard anecdotal evidence about bad experiences with peer review, but mine have been fine. Sometimes a bit snarky, but not unprofessional. I suspect those that truly cross the line are the exception. To me, that doesn’t seem like grounds to overhaul the system.
So the reports will now have to written with a view to the fact that they might be published. That changes things – surely, writing a review for internal use and writing a review to be potentially published are two very different things. And I assume it would be mostly positive reviews?
I think it is a good initiative for this reason and related reasons, including constructiveness and rigor.
However: for a flagship journal, it might work, but otherwise peer review quality is nowadays so low and random that I doubt publishing review reports would do any good for journals themselves.
Another point is that it should perhaps not be the published papers for which reviews should be published but the rejected ones. It might be doable in the future as more and more people publish pre-prints. Such a practice would help at dealing with junk science on one hand and exposing the low-quality reject-button reviews on the other hand.