Bad Reviewer Experiences
A couple of years ago, we had a discussion of “Philosophy Journal Horror Stories“. Most of the experiences shared were from the perspective of authors. But authors aren’t the only participants in the academic publishing system with complaints (from which we might, one hopes, learn something).
Won’t someone think of the poor reviewers?
Seriously. Though much-more complained about (112 comments) than appreciated (24 comments), peer reviewers play an essential role in the publication of academic articles and books. Yet we haven’t paid much attention to what counts as appropriate treatment of reviewers and their work.
This post was prompted by an email from someone who refereed a manuscript for a journal and was dissatisfied with the editor’s failure to take seriously a problem with it that he pointed out in his report. He says:
I recently reviewed for [a journal], recommending something in between a rejection and an R&R. The author was given a chance to revise the manuscript and after revisions the editor asked me to either accept (with minor revisions) or reject. The author was not able to deal with the issues I raised and so I recommended rejection.
The paper was later accepted in a form close to the original submission. While it is fine that an editor disagrees with an individual reviewer’s judgment, I had noted a number of errors in the way the author described the literature and as far as I can see none of them were addressed. I later confronted the editor on this point and his simple response was that I was the only one raising this concern and that there could be reasonable academic disagreement in this case. I though this was shocking and as I told the editor: my claims could have easily been verified (or, for that matter, proven false) and if the editor doubted what I was saying, they could have simply asked me for textual evidence.
It seems to me as if this is a problematic decision by the editor irrespectively of whether I was right or wrong about this factual claim. That is, if a reviewer claims that the literature is misrepresented, should the editor not take this seriously enough to verify the claim?
Readers are welcome to share their own bad experiences as reviewers, reflect on the experiences others share, and discuss norms for how reviewers and their work should be treated by editors and authors. But first let me include a passage from the previous post on journal horror stories:
Before we begin, now would be a good time to invoke my multi-purpose adage, “philosophers are people, too.” That includes the philosophers who are the editors of and referees for academic journals. People make mistakes, people have multiple demands on their time, people get tired, and so on. Further, these people are often volunteers or inadequately compensated, adding to their busy lives the various responsibilities of maintaining a significant portion of our professional ecosystem. So even when we may be sharing stories that reveal their imperfections, I think it is important to register appreciation for all the work they do.
OK. Let ‘er rip.
Related: “Citing the Referees at the Journal that Rejected You“, “Citing (and Thanking) the Referees at the Journal that Rejected You, Part 2“, “A Public Database of Referee Service“, “The BJPS Referee Of The Year Award“, “How Bad Is Reviewer 2, Actually? Data from a Philosophy Journal“, “The Best Reviewer/Editor Comments You’ve Received“, “The Worst Reviewer/Editor Comments You’ve Received“, “An Objection Does Not A Rejection Make“, “Referees With Attitude Problems”
“if the editor doubted what I was saying, they could have simply asked me for textual evidence”
If it is easily demonstrable that an author has misread the literature, provide at least some evidence of that in the review.
a few years ago, when I was just out of grad school, a very good specialist journal asked me to review a paper with a caveat: “The author is very well-known, so we hope this is not inappropriate to ask this: ideally we’d like to get either an R&R or an accept verdict for this paper.”
I gave it R&R with a reply that I did think that the letter was inappropriate. The paper was accepted and it never came back to me.
So, blind review is a joke?
I think it’s inappropriate to generalize from an experience at one journal on one occasion, to a claim that always and everywhere, blind review is a joke. (Various journals I’ve worked for have had various levels of anonymity – many of them now prevent even the editor-in-chief from seeing the authorship until it is accepted, with only the managing editor, often a non-academic or a grad student, seeing it.)
I think Travis’s point was that, in this particular situation, blind review was treated as a joke. I’m not sure where the “always and everywhere” generalization was implied.
Using ‘is’ instead of ‘was’ does give the impression of a generalization.
I see the impression; I don’t see how a charitable reading actually implies the generalization. One way to bring out the charitable reading is to append “blind review is a joke” with “to those editors.” That seems tacit anyway, and avoids the obvious falsehood that blind review is a joke “always and everywhere.”
But neither is there is an existential quantifier to imply “in this particular situation”. Without a quantifier, and the use of the word ‘is’ not ‘was’, the comment (or rhetorical question) in this case does imply a generalisation.
Disagreed. Reading the statement with a tacitly appended “in this situation” (or something like that) is clearly the charitable move, given that it is more appropriate in response to the initial story, that it avoids the obviously false generalization, and that it occurs in an offhand blog comment rather than an exercise in formal regimentation.
“They wanted me to go easy on the paper because it was from a well-known author!”
“Wow, so blind review is just a joke I guess?”
“Well, no, many journals still properly utilize blind review; it would be unfair to generalize from this situation to all journals everywhere.”
“Yeah… I meant that it’s a joke to those editors…”
Not every comment here is meant as a perfectly explicit and literal assertion (contrary to a surprising number of replies I see on this blog). Sometimes, people just want to chat like ordinary folks rather than academic philosophers lol.
Also, the anecdote is a reason to think triply anonymous is a good idea.
Short answer: Unless the journal has a triple blind review policy, yes.
Longer answer: Editors can and do put their fingers on the scale in all sorts of ways and many will do this for big name people or their students. These examples are egregious but the more subtle ones worry me more. For instance, some referees are known to be prone to recommend acceptance or R and R on anything that’s not flat terrible and others are known to reject almost everything. Want an acceptance send to the first. Want a reject send to the second. They can also choose to send papers to referees likely to be hostile to the thesis or sympathetic.
I’ve said it before but I’ll say it again: The system would be much more rational if editors were actually allowed to be editors. The main barriers to this are training and, most importantly, compensation. I do not have time to actually be an editor because I am not compensated for it in any way by anyone.
No specific ‘horror stories’, but I do find it frustrating when, as a reviewer, I am not informed of the editor’s decision to R&R, reject, etc. I appreciate seeing the other reviewers’ reports, but at a minimum, I think the editor should share the decision letter with reviewers.
As an editor for various journals, I believe that at least one journal I edit for does this, but I actually don’t know about the others! There’s not an easy way for an editor to shape who gets what – it’s all up to the software defaults at the journal (which the editor-in-chief might control, but might also not know how to set).
That’s ridiculous (though I have no doubt that it’s true).
For a lot of journals now, you actually can access this information in the relevant submission management system. Journals don’t inform you when a a decision is made, of course, but if you go back after a few months (or however long the decision process is meant to take), you can often see the comments from other reviewers and sometimes the final decision.
I find it frustrating when I initially write a detailed review justifying a rejection only to find that another reviewer has written a brief review review to the tune of “this is perfect, no revisions, accept.” It’s possible for two reviewers to disagree about the quality of a manuscript, of course, and but I find these very brief, overly positive reviews on manuscripts quite suspect. It’s much easier to write a short, positive review that doesn’t ask for any changes than a constructive critical review, and sometimes I have the strong suspicion that the writers of these positive reviews are just being lazy. It’s especially frustrating when some editors decide to assign equal weight to these short positive reviews and return a verdict of “R&R,” leaving the author with the burden of satisfying the critical reviewer, who doesn’t think the paper is salvageable but is forced to review multiple revisions of it anyway (the positive reviewer often remains so throughout the same process).
Now, I’m not saying that reviewers should always ask for changes. I just think they should show in their reviews that they have read the paper carefully, even if they ultimately recommend accept. I would say that minimally, a positive review should give an overview of the argument and an evaluation of the ways in which it is a good contribution to the literature. And editors should treat short, positive reviews that lack such detail as highly suspect.
I just a few months back reviewed a rather poorly written paper on a niche topic for a major journal. It was one of these papers where you could see that there was an idea there, but it would take so much effort to revise it to bring this idea to the forefront, that I reluctantly recommended rejection and wrote a VERY extensive review with suggestions on how to trim the fat, improve perspicuity, strengthen the argument, find better examples, answer objections etc.
It was then flabbergasting to see that another reviewer gave it a glowing two-paragraph review, highlighting the enormous clarity, high importance of the observation, undue neglect of the niche, going to far as to suggest that the paper could be included in an anthology.
I figure that as an editor, I’d just disregard such a review outright. The over the top praise screamed insincerity. It also does not do the author any favours, who might now be less receptive to constructive suggestions for improvement.
FWIW I’m reasonably certain I’m not the unreasonable one because other reviewers shared my assessment. One more cheer for being able to read other reviews. I do like to have evidence regarding whether I’m being reasonable.
Since when did it become praise to suggest a paper could be included in an anthology?
Touché
I’m another of those who submitted a reject recommendation only to see the essay later appear in print, virtually unchanged. (Thanks Manuscrito!) As best I could tell, I was the only referee, and I got nothing but radio silence after I submitted my referee report.
It is fine to disagree with me. Reasonable readers often do disagree, and it is plausible that I’m not always all that reasonable!
However, given the lack of an R’n’R, the lack of any apparent changes in responses to my comments (including comments along the lines of “the theory offered here is the dominant theory in the literature, offered already by the bulk of the Big Figures, and there is no reference to those figures or explanation of what is new”), and the lack of any acknowledgment, I don’t have much reason to think that my referee report—the product of both substantial expertise and a good deal of time—was taken seriously. I won’t be refereeing for Manuscrito again.
I have had experiences like this with various journals that are at the edges of the philosophical journal ecosystem – either journals I hadn’t heard of, or journals mostly in another field that had a philosophical paper to deal with, or some combination of the above.
Sorry, but I have a hard time believing what you just said. I submitted multiple reject suggestions to Manuscrito, and what you just described did not happen to me.
I too have a hard time believing what you just said. I have had many experiences, including many experiences in places where other people have had experiences, and what they described did not happen to me. Ergo, they are wrong about what they say happened to them.
I did not say you are wrong. I said it is hard to believe what you just said. However, I believe it is for the best that the journal will not have you as a reviewer anyway given your comment above.
Like others, I appreciate reading the other reviewer’s report. It is reassuring to have points confirmed by their judgment and useful to see different points raised. [Also, when I review a paper, I sometimes explore the surrounding literature and start generating my own paper ideas or realise that some issue illustrates a point I want to make on a different topic. It can be useful in that respect if someone more familiar with an area tells the author that A and B are essential reading on the issue.]
It is especially important to see the other report when asked to review an R&R. If I don’t know what R1 said was lacking in the initial submission, I won’t know why the author introduces all these changes here and there.
I think it should also be more common for editors to send the reviewers each other’s reports in cases of significant disagreement and ask them if they have any further guidance to resolve the impasse. I don’t believe this will add all that much work, and may make for better editorial decisions. Often the disagreement will persist, and the reviewers may start getting annoyed with each other. But if it does lead to some revision in what R1 or R2 says, then that is more useful to the editor than just trying to make sense of conflicting views.
There have been occasions where, as a reviewer, seeing the other reviewer’s remarks help me explain how I think what the author is doing isn’t actually making a mistake the other reviewer thinks they are, or why the point the other reviewer is asking about is outside the scope of this paper. It’s a little bit of an odd position to be in, but I do think it is helpful, both for the author and the editor.
I once reviewed an article, recommended rejection, and was then asked to review an almost unchanged draft of the same paper for another journal.
That’s fine, my views are not definitive, the author is under no obligation to think my criticisms are valid or worth amending their paper for.
I say “almost unchanged”. Second time around, the author had removed some acknowledgements of helpful feedback from anonymous referees. And no, it wasn’t with a view to restoring such things post-acceptance, because they had left in an acknowledgement to Namedropped Famous Person.
Absolute dick move.
I always decline to review manuscripts for which I’ve already given a reject verdict elsewhere.
Well, the title was the other thing that changed, so I only knew on reading it that it was the same paper.
In such a case you are totally within your rights to contact the editor and asked that the job be reassigned.
Yes, I know. I did. This was not the point of my anecote.
It happened to me a few times that I recommended acceptance and the editor recommended revise & resubmit because I gave some (explicitly marked as optional) *suggestions* for improvement.
I think we have too many R&Rs. They mean additional work for all involved (authors, reviewers, and editors). So I’ve been trying to be more definitive in my verdicts (outright reject, outright accept with optional comments, conditional accept with non-optional corrections).
Optional suggestions for improvement are how I show that I engaged with the paper, to strengthen the force an accept recommendation. (Cf upthread about the problems with too unengaged accept recommendations.) But it seems that (some?) editors are all too ready to read any potential improvement as a need for revision.
Two bad experiences. Same top-10 journal.
Re #2, do you know that your comments made it back to the author? If not, the author has not shown themselves to be untrustworthy. I wouldn’t take the misspelling of the other author’s name as evidence that the original author received your comments, as that’s something that’s likely to have been caught by a copy editor.
Yes, I do know that my comments were returned to the author. As mentioned, with this journal you can see in Editorial Manager what ‘letters’ have been sent to authors. The misspelling was subtle and unlikely to have been picked up by copy editors–the editors used by the journal’s press are not great. There is much more I could say; these were not the only two comments I returned to the author, but they are the only two that stand out right now and I believe all were dealt with apart from my requestion to take better account of the existing literature. I am giving thin details since this is obviously a delicate matter…
I served as a reviewer for an article submitted to a special issue of Synthese. The article was so completely out of touch with the recent literature on the topic that it read as if it had been written 20 years ago. The work was also completely derivative, reading more like a lit review than anything else. I recommended rejection, but I took care to write a very thorough, carefully worded report. The other reviewer assigned to the piece recommended rejection as well.
Later on, I checked on the status of the article, and I was able to read the decision letter that the editor had written to the author. The editor recommended that the author resubmit the article, and also spent a significant chunk of the letter disparaging both me and my fellow reviewer for being overly harsh. They speculated that our relative youth (I’m a tenure-track professor, btw) had something to do with our verdicts (“younger reviewers tend to be very harsh in their assessments”) and suggested that we had rejected the article because of our “very narrow, and sometimes even self-interest-defending, perspective,” whatever exactly that means.
I prefer not to review for journals whose editors feel free to trash their reviewers, thanks.
For what it’s worth, this is against current (rather strict!) Synthese editorial policy about special issues (which, even in the case of invited papers, require peer review and, I am pretty sure, for both referees to recommend at least an R&R–I could be wrong about the details but I am nearly certain what you describe is a violation of the policy)–depending on timing of when this happened to you that either means that (hopefully) this is much less of an issue there than it was when this happened, or that somehow the editor of the special issue got away with something that they shouldn’t have been able to. In either case I think this is probably at the least extremely unusual at this point given the changes Synthese has made to the oversight of their special issues.
This was 2018. I hope things have improved since then!
Ah, yeah. I think that the policies have changed since then/when they overhauled things after their two scandals. In fact, from what I gather Synthese special issues are some of the only venues where it is now somewhat routine for people to be invited to contribute a paper and then have the paper rejected.
I refereed for a niche open access journal (free to readers and authors), and I never even got an acknowledgement of my review (which I completed in about a week). I gather that because they’re open access and trying to do things on shoe-string budget (so no automated software to handle submissions) I should be forgiving and so I didn’t email or bother the editor, who I’m sure is overworked. Still, a bit disappointing.
On another note, I suppose I may have become one of those “lazy” referees that others complain about – I have reduced the number of R & Rs that I give (and the length of my comments)- partly because I’m asked to referee more and more and partly because I’ve come round to the view that if the paper is comparable to the average essay published in the relevant journal, I should probably just say “accept” even if there are ways the paper could be improved. (I do try to indicate these where it is relatively easy to do). After all, there are lots of ways papers that do get published could’ve been better…
(Imaginary report on Quine’s Two Dogmas “Insufficiently clear on how the two dogmas are “at root identical”.
The Author added a footnote to Duhem’s thesis in the “R&R” (Quine added the reference in between the Phil Review version and the From a Logical Point of View version) but shows no awareness of how Duhem’s thesis and his claims are distinct; seems to misunderstand Carnap as a simple “radical reductionist”,etc. etc.)
At least a few of Quine’s works would be brutallized by today’s refereeing standards. Both “Two Dogmas” and “Epistemology Naturalized” are somewhat unclear and shockingly poorly argued, from the current perspective.
As the reviewer in the case above, I want to add to the description in the original post by Justin. Beyond several misrepresentations of the literature, the fact that the paper was published in a form close to the original submission also meant that the author and editor knowingly ignored counterexamples in the literature, since I raised these in the review report (while the author unsuccessfully attempted to deal with these in the revision, that’s not the version that was published).
To me, this is contrary to basic academic standards. Although we cannot always deal with all counterexample or produce flawless papers, we should always recognize counterexamples and potential flaws. The fact that the editors allowed such flaws to be hidden is beyond the pale and I have no respect for that journal.
(If anyone wonders why I write this pseudonymously, it is because if I name myself there’s a chance that the paper could be identified and therefore also the author, which I don’t believe is fair all things considered.)
A few years ago I recommended that an article I refereed be rejected. The editor nevertheless decided that the paper warranted a verdict of revise and resubmit instead and sent the resubmitted version to me. I expressed a strong disinclination to review the revised manuscript on the grounds that, because I didn’t think that revision would yield a publishable paper, it was unfair to have me review it. The editor ended up got very testy with me about this, suggesting that my attitude was unreasonable. I remained (and remain) unmoved.
Many of the journals I review for include a clickbox option indicating a willingness (or not) to review a revision in the event of an R&R editorial decision. If a reviewer elects not to re-review after the first pass, editors can seek out other reviewers if they so choose. This approach respects the considered views of reviewers who are, after all, *donating* their time and expertise to the peer review process.
I had something similar to MoonShine’s experience only a lot worse. When I was a lecturer at a big state school one of the TT profs edited a specialist journal in the field I wrote my dissertation. He asked me to review a few times and there was no pressure and I generally enjoyed it. But then one day he sent me a truly bad paper which I sent back with a verdict that we should reject it with a possibility to revise but no encouragement to do so, and notes on a good four or five things he’d absolutely have to address and two or three more he should. (I should have flat rejected but I was nicer in those days). I should add that by this point the journal editor had been appointed interim head of the department after the last one flamed out. He catches me in the hall one day and says, “Well the guy who wrote that paper is a big deal in X and we’d really like to have a paper by him so I’m going to treat your report as a conditional acceptance. That’s okay right?” After a long awkward silence I say, “Yeah well I guess so.” Him, “Oh that’s good! And since it’s a conditional acceptance you won’t need to worry about reviewing the resubmission.” What do you do when your boss says that? Now this guy never struck me as a bully or as vindictive and I don’t think he’d have punished me for taking a stand if I had. But then again I wouldn’t have thought he’d pull some shady ***t like this until he did. And would you chance it? That still leaves an enormously bad taste in my mouth.
I had a journal send me a request to review an article, and they wanted it done in two weeks. I said I couldn’t do it that quickly, but I could probably get it done in six weeks once my semester ended. The editor agreed, but then after two weeks, he kept sending me emails every couple days hounding me for the review. I finally told him that if he wanted a quick verdict, I’d say reject it based on my quick reading of it. He then came back with something like, “Are you sure? The author is very highly regarded, and we thought this would be an easy acceptance.” I stuck to my initial stance that I could give it a more thorough look in a few weeks, if he wanted, but I never heard back. It felt like the author was big name and/or friend of the editor, for whom the editor just wanted a couple quick rubber stamps from reviewers to publish it.
On the plus side, that journal hasn’t asked me to review anything since then.
The lesson I’m taking away from many of these stories is that once you’re a “big name” you’re exempt from real peer review. I guess I kinda assumed that, but depressing to get confirmation.
Definitely not universally true. I referee 30-odd papers a year and I’ve never come across this.
I don’t referee quite that many, I think (probably closer to 15-20 most years) and I have also never had this experience, even though I’ve had a number of cases where the authors turned out to be pretty big (sometimes very big) names.
I’m not sure what you mean by this “universally true.” Do you mean that not every paper by a big name is judged by different lower standards at every journal? No one ever said they were and if that’s what you mean then your rejoinder is very much taking on a straw man. But if it’s common enough in many areas it’s a pretty big problem. After all to say that there’s a problem with violent crime in city x is not to say every single person in city x will get murdered or even assaulted. Just that this bad thing is much more common than it should be. In the same vein I couldn’t well argue that just because I live in city x and have never personally seen anyone get assaulted or murdered there’s no crime problem much less that violent crime in city x doesn’t exist. I’d also wager that editors prone to do this kind of thing hesitate to do it with established people because they’re more willing and able to push back.
But the question is how common it is. David is just providing a non-trivial amount of data (given the number of papers he has refereed), that it is not universal, at least in phil physics and science. His evidence is more than “not every paper…” that’s also understates the counter-evidence he and Matt (and I) are providing. I’ve also refereed over 100 papers and never had this experience. Sure, all of these are just a small bucket in the grand scheme, but if it was relatively common you’d at least think one of us would’ve had this happen one time. But maybe we’ve just been lucky?
But if your point is that it is bad if it even happens once, then of course I would agree. But I gather junior scholar is worried about it being a regular occurrence. We need more evidence for that, which you have not provided (nor does this comment thread, since the OP asked for horror stories, and so you’re primarily going to hear the bad cases here!)
I reviewed a paper for a journal (revise and resubmit) and when the second round of reviews began I logged into the journal system to read the other reviews (like many others here have noted, this is useful when trying to understand the changes authors make) only to find that the system also presents to reviewers the reply to the authors from the editor… which has the names of the authors unredacted.
I informed the editor that this should be fixed, and I was told it would be. The paper got another revise and resubmit… and the problem still wasn’t fixed; the reply to the unredacted authors from the editor was still visible on the system.
This same journal also managed to publish an uncorrected first draft of an article which went through several rounds of revisions; when this was pointed out to the editors it took them three months to issue a correction, and now the now retracted first draft is getting cited by papers I’m currently reviewing…
I once reviewed an MS for an extremely prestigious generalist journal, one that didn’t have a questionnaire soliciting a verdict (Accept, Reject, R&R, etc.). So I wrote a letter explaining why the thing wasn’t publishable. Lots of good stuff in it, I wrote, but its main thesis had been scooped — argued almost verbatim, and for almost the exact same reasons — in a prominent journal a few years earlier. It was clear the author simply missed the pub; an understandable but still fatal mistake. My letter made it plain the piece couldn’t be published with its current thesis and argument, however insightful some of the side points were.
Sadly, the editors took this to mean the author should simply come back with a new (altogether different) main thesis and argument. This led to predictable disasters: the author, eager to publish in this super-elite venue, scrambled to come up with a whole new paper from scratch, which consequently had many new problems, rushed as it was. So new referee report, even more critical. And again, it was sent out, with the author producing yet a new, even shoddier argument, and so it went for a little while longer (the other review, btw, was equally negative).
The lesson? For reviewers, state right off the bat; “this piece should unequivocally be rejected” (I thought “needs a different thesis + argument” would suffice), and minimize praise even if you mean it, lest it be mistaken for mixed feelings.
For the journal, I’d say have a questionnaire, heed its outcome, and don’t go out of your way for an author just because of what else they may have done.
The author, whom I gathered was an excellent — if in this case, unlucky — philosopher, deserved better, and I’ve always wanted to apologize to them for the hell we put them through, for no benefit whatever.
So, the author didn’t plagiarize the earlier paper, simply didn’t read it?
I’m not sure if that should be a fatal flaw. Nobody can read everything.
Surely it’s fatal just because it’s no longer making a contribution to scholarship, given the previous paper (the contribution has already been made).
I think we need more details to conclude anything like this (that the flaw is fatal). Having a thesis scooped is sort of par for the course in a field as old as ours. Perhaps making the exact same argument would qualify as fatal. But one of the cool things about a field like ours is that different folks can explain roughly the same thing in different ways and yet make a contribution.
I should clarify my initial post, because I fully agree – had the author merely taken a position that others had taken, that wouldn’t be in any way fatal. The problem is that the author’s main theses – and the main aims of the paper as they introduced it – were (1) everyone until now claims P; (2) I will now (for the first time) show that Not P. [This several times presented as their main contribution.] (3) For reasons X. (4) Instead, Q.
As it turned out, at least two articles in very prominent journals (at the level of, say, PPR and NOUS) had just recently called out the prior literature about P and argued that it is false; and at least one of them did so for the very same reasons X trumpeted in this particular manuscript, and argued for the same Q as the alternative.
Again, I didn’t think the author culpable in this redundancy; indeed, I, too, almost missed those others, and I’m supposed to be (broadly) in this area. But it still made their contribution, or at least the main one even as they themselves presented it, gratuitous. Or so it seemed to me and (I later saw) my fellow reviewer.
Many, if not all of the comments, are about recommending rejection and then being ignored. I wonder if there are also cases of recommending *acceptance* and being ignored? Or at least cases where the paper being reviewed was considered good by the reviewer?
A few years back I reviewed a paper for the Journal of Applied Philosophy that I thought was really good – one of the more interesting papers I’d read in a while and on a topic that didn’t have a lot written on it. It needed a few changes, so I asked for those but said that I thought it should be published with a few additions/changes, and that I didn’t take those to be major or hard to make. (Mostly some clarifications and additions at a few points.) The paper was just flat rejected without any further commentary or reason given. Thankfully (and unsurprisingly) it was soon thereafter accepted in another good journal. But, I wasn’t willing to review for the Journal of Applied Philosophy after that until the editors changed, since it was clear they didn’t actually value my judgment. That was their choice, of course, but if they didn’t value it, I didn’t feel like I needed to take the time to give it, I thought.
I’m probably far too late to get into the discussion, so a quick comment.
I’m struck by how most of the bad experiences/horror stories recounted in this thread have to do with editorial decisions more favourable to the author than recommended. For instance, the reviewer may have thought that some references were missed and when the paper appeared, these references were still missing.
I am really not sure why something of this sort counts as a bad experience. There are lots of reasons why an editor might take this path. It doesn’t necessarily mean she thinks the referee’s views are wrong or not worth taking seriously. (Maybe, she thought that the references didn’t add enough to merit the space they took, or that their omission was not mandatory. One needn’t feel slighted or badly treated by this.