The reason your paper is listed as ‘editor assigned’ is that I’m going to review it myself.
In the wake of the recent discussion here about the editorial practices at philosophy journals, Jonathan Jenkins Ichikawa (UBC) recounts a story, set about five years ago, in which he submitted a paper to a journal with a policy of double-anonymous reviewing—Philosophical Studies—but which clearly, and quite blatantly, did not make its decision using such a procedure. Instead, the editor, Stewart Cohen, who became Ichikawa’s colleague during the process, engaged in various email and in-person exchanges about the article, revisions to it, and its ultimate acceptance. Ichikawa recounts the story at his blog. He writes:
The editor of Philosophical Studies seems to be treating submissions in his own area of research interest very differently from the way he treats other submissions, sometimes deciding to referee papers himself. As far as I know, Cohen was the sole referee for my submission… On the whole I’m very confident in asserting that in at least one instance, Philosophical Studies engaged in seriously problematic editorial procedures. I don’t know how atypical my story is.
UPDATE: Stewart Cohen, editor of Philosophical Studies, has asked me to post the following:
Philosophical Studies is now in the process of instituting a triple-blind review system. Neither the editor, nor the associate editors will know the identity of the authors until after the decision has been made (except for special issues and symposia which are invited). [Edit: Cohen asked that the preceding struck-through text be replaced with the following:] All submissions will be triple-blind reviewed. Invited papers will be labeled as such.
UPDATE 2 (2/4/15): I would like to get a better sense of the extent and types of these kinds of problems. Publicizing the problematic episodes and naming the journals will help with that, serving to raise awareness of the problems as well as possibly deter them. If a paper of yours received what seems like inappropriate treatment by a member of the editorial team at a journal (as with Ichikawa, Anonymous (comment #4), Clayton (#12), anonymous junior prof (#21), Grad @ the time (#35)), please describe it in the comments. I encourage you to name the journal. Anonymous comments will be accepted, provided that they are submitted with a real email address by which I can verify the identity of the author. Your email does not get published when your comment is approved, and, as usual, I will not reveal the identity of anonymous commenters. Please refrain from posting second-hand stories (e.g., “this once happened to a friend of mine…”). NOTE: I have received a few comments recounting alleged editorial malpractice in which the journals are named, but the authors have failed to provide real email addresses. These comments will not get posted. Feel free to resubmit your comment with a real email address I can use to verify your identity, if I need to.
UPDATE 3 (2/7/15): One of the journals receiving a lot of attention in the discussion below is Philosophy and Public Affairs. Its editor, Alan Patten, has provided the following statement:
Readers of the blog might be interested in the statement about editorial practices that is posted on Wiley’s Philosophy & Public Affairs site under Author Guidelines.
Some of the questions about the journal’s procedures that have been raised in this forum are addressed in the statement, including questions about our policies on anonymity, on lengthy submissions, and on submissions by associate editors. Concerns about fairness and bias are certainly valid in this area, and there are some genuinely tough questions such as whether to publish articles by our associate editors. I’m sure we could do better in some respects, but on the whole I think our procedures do a reasonable job of balancing some of the competing goals and demands of running a journal.
I’m disturbed by the insinuation in one or two posts that there was something corrupt about our decision to publish Niko Kolodny’s recent two-part article. I acknowledge that there might be reasonable disagreement about whether a journal should have a blanket policy of not publishing papers by current editors. But that point aside our adherence to our editorial policies could not have been more rigorous in this case. In addition to sending the paper to an external reader (as described in the statement) I confirmed with that reader at the start of the process that s/he was not already familiar with the paper, did not know the identity of the author, and had no presumptions either way. I should add that it’s not as if there’s been a great many articles published by associate editors in recent years. We’ve accepted a grand total of two such articles (including the two-parter by Niko) under my editorship (which began in 2010). We’ve also declined several submissions by editors in this timespan. In any case, I’m proud that we were able to publish Niko’s article. If you take the time to read it, I doubt that you will conclude that the best explanation of why we published it is anything other than its excellence.
Kudos to Ichikawa.
If this didn’t happen all the time one would be hard pressed to explain the recurrent contributions by certain people to journals like Journal of Philosophy, Philosophy & Public Affairs and Journal of Political Philosophy. Those are just some of the most obvious examples.
Philosophy and Public Affairs just published a two-part article by one of its Associate Editors. It may be a fine article, but I would have thought it obvious that every journal should have a policy of never publishing work by its main editors or reviews of their books. (It was certainly the policy of The Canadian Journal of Philosophy when I was an editor there.) I’d be interested to know if others share this view. Philosophy and Public Affairs evidently doesn’t, nor in the past has Ethics. But the claim that review has been blind in this kind of case is rarely credible — you really had no idea who the author was??? — and in any case I think it’s important that journals not allow even the appearance of favouritism. That’s a standard we hold public officials to — they can’t say they received the gift from the developer but weren’t influenced by it — and I think journals should abide by it too. I’m amazed when they don’t.
Cohen refereed a paper that I submitted to Phil Studies some years ago. Very unprofessional: an editor should follow the editorial policy printed in the journal.
[Note: this anonymous post was submitted from a verified, legitimate email, and so is not anonymous to me. No unconfirmable accusations or rumors will be permitted.]
Tom, it certainly raises a fair question. Of course, top journals (especially) will often receive submissions that many people in the field would easily know the authorship of (this doesn’t address Jonathan’s case), so that cannot be entirely avoided even if editors were barred. Also, it’s not clear that we want journals to be edited only by people who are willing to forgo publishing there. It’s not absurd, but it’s not clear. I have just signed on as an Associate Editor with P&PA (though I have not yet begun my duties). All things considered, I think that stricture might have tipped me into declining, at least if the expectation is (as it currently is) that I will try to serve for more than a couple of years. Not that I’d be such a great loss, but it might be that others whose service to such journals is very desirable would feel the same way. One work-around might be to expect that sacrifice, but ask for only a two-year commitment, but that has disadvantages too, as a journal burns through their top choices for editors in the course of a few years.
Tom Hurka said, “I would have thought it obvious that every journal should have a policy of never publishing work by its main editors or reviews of their books.”
I don’t always agree with Hurka, but this seems to me to be pretty clear, at least if journals don’t want to have a reputation for various bits of favoritism or in-dealing. (Some seem not to mind this.) But, it seems to in fact to be controversial. In way way I think this is a pathology of philosophy – the ability to construct arguments tends to make people able to construct arguments that are at least _able to convince them_ that, say, they are being impartial,even when it’s in fact quite implausible that they are (and certainly not at all obvious to those outside!)
People at the center of circles of privilege may not think it’s that much of a big deal if X get a spot in PPA as opposed to Y. Sure, let’s help a friend out, what goes around and all that. People with the right sort of pedigree will make it regardless, right? Who cares if some nobody’s entire livelihood has been destroyed by the fact that someone in this “top” journal’s inner circle took over half an issue with his musings, in all likelihood refereed by another friend and newly appointed associate editor, whose work takes center stage in the gargantuan 73-page article?
I got the same email from Cohen, and my paper was ultimately rejected. I was a graduate student. The comments that were included were very helpful–I’m not sure whether they were from Cohen or a second referee.
Regarding the P&PA article: I don’t think that David Estlund’s response adequately addresses Tom Hurka’s concerns, which I take to focus on articles having the combination of (i) effectively non-anonymous and (ii) written by editors. Estlund responds to each point separately, but not their combination: their combination can be avoided by allowing editors to submit works only if they have taken all reasonable steps to ensure anonymity e.g. by not sharing the work in progress with other editors. The problem of their combination is obvious if you imagine what it would be in the position of an editor reviewing the work of another editor when (a) you know it is likely that they know it is you reviewing; and (b) you plan on submitting your own work for them to review. As well as the question of what best practices are for a journal (and what is clear to Hurka is also what is clear to me), there is a question of what the individual editor is morally required to do. If an editor intends to submit a paper to their own journal, would it really be such a hardship to avoid presenting it in venues where one can predict one’s fellow editors to be likely to be present?
If you look at the thanks for that paper, you will see that Niko Kolodny thanks for written comments, Alan Patten (the main editor of P&PA), as well as Arthur Ripstein and Jay Wallace (associate editors) and that he presented the paper at UCLA (the university of Barbara Herman and Seana Shiffrin who are also associate editors). This leaves at most exactly one member of the editorial board (Stuart White) who Kolodny would not expect to be aware of the paper. Given that a significant amount of the reviewing at the journal is done by editors, Hurka is surely right that the journal has allowed at least “the appearance of favouritism,” particularly when it publishes a two-part paper of 83 pages and its website states “We prefer papers of fewer than 12,000 words (including notes) and may return longer papers to their authors unreviewed with the suggestion that they be resubmitted in shorter form.”
As well as the obvious issue of fairness, my impression is that this behavior is very damaging for the journal’s reputation — I remember conference chat of many years ago about P&PA’s reputation for publishing papers by editors and their grad students. I’m sure I’m not the only person to have reflected on the fact that the political philosophers published in Ethics are much more diverse than those published in P&PA. As the online journal rankings repeatedly show, Ethics’s reputation has exceeded P&PA’s for a while now. I wouldn’t be surprised if these facts are linked.
Responding to Anonymous [9]: Sometimes a lead editor will recognize the authorship of a paper even if it has been anonymized. In that case fully blind review is impossible. One could argue that the journal should immediately reject in that case, though I’m not at all sure that’s reasonable. But suppose the editor also realizes that it is by an associate editor. Again, I doubt that it’s reasonable to let this immediately disqualify it, though I’d appreciate hearing from those who disagree. I do think that in that case it would be important to get reports from people who are as independent as possible. Suppose Nagel submits a paper while he is associate editor, and it is easy for the editor to realize who it is by. I do believe it should receive serious and independent review so far is possible, but I’m not sure it should be disqualified.
Whenever an associate editor publishes in the journal, I suppose it is open to people to suspect that it was accepted only through favoritism. The ethical requirement is to make sure that that is not the case. It is also important to the profession to avoid the appearance of favoritism, but the bare possibility of favoritism is not the appearance of favoritism. There is a limit to how publicly this can be policed, and in the end what matters is the quality of the published work. Blind review is an excellent instrument toward ensuring acceptance of the best work, but (I believe) few if any journals disqualify referees who know the authorship of the reviewed piece (Ethics does not), and this is because that would work against the goal of reviewing and publishing the best work. I think similar leeway is sensible in the case of publishing work by a journal’s editors so long as it is reviewed as independently as possible. The fact that an excellent journal such as Ethics is often ranked even more highly than PPA is pretty slim evidence that PPA is corrupt.
As for the thinly veiled suggestion by Lumpen [7] that I played any role at all, formally or informally or in any way, in the review of either of Kolodny’s pieces, I have excellent news: the conjecture is entirely mistaken. Lumpen also depressingly conjectures that I or someone thinks it doesn’t matter whether favoritism plays a role or not. On what basis, I have no idea. Of course it matters enormously. Tom Hurka’s question about submissions by a journal’s own editors is (as I indicated) a serious one, and not simple.
Finally, I would be curious to know which journals formally forbid publication by any editors or associate editors. If that is working for some top journals then other journals ought to consider it. (It would not present the same challenges to a lower-tier journal for obvious reasons.)
By the way, I don’t know about others, but I always decline to referee papers that centrally involve my own work (not that this issue arises all that often), or that is by or centrally concerns the work of a student of mine. (In any case, I was never even consulted at any stage about the papers in question.) I wonder if people think we should all abide by that.
Same experience as Jonathan, sans the acceptance. Why aren’t we putting pressure on journals to be transparent and practice triple-blind review?
Not being an academic, I don’t understand how submissions from relatively well-published people could be anonymous. Isn’t it customary to refer the reader, via footnotes, to one’s other publications addressing the issues that are pertinent to, yet not the focus of, the current article? I would be surprised, for instance, if Tim Scanlon wrote an article about any issue in normative ethics without referencing ‘What We Owe to Each Other’ or ‘Being realistic about reasons’ somewhere in the text. And for good reason (to show that he isn’t just assuming certain things without argument; to direct the interested/skeptical reader to relevant/prior arguments elsewhere etc.). Are papers sent to journals without such footnotes? Or with such footnotes but with empty placeholders instead of the actual names of the articles/books? If the former, I can’t imagine how big assumptions, controversial theses or apparent gaps in argument could get past the editor’s desk. If the latter, any competent referee would be able to figure out who the writer is. Admittedly, Scanlon is a pretty big name, hence not an ordinary case but I am guessing that editors and especially referees for top journals could do this for a LOT of the people working in their areas of research. What am I missing? (The only other alternative I can think of is explicitly referring to the texts but in third person as if you were someone else, but that is just absurd to me for a number of reasons.)
You get it right with your parenthetical at the end. The practice for papers undergoing anonymous review is to either refer to such texts in the third person or to leave the citations blank, with these corrected or completed after the manuscript is accepted.
“Whenever an associate editor publishes in the journal, I suppose it is open to people to suspect that it was accepted only through favoritism.”
Why *only*? Nobody thinks that. But still there is a reasonable suspicion of favoritism.
Stewart Cohen’s statement in the update is encouraging. However, it raises another question: Why do journals have special issues? There’s already an abundance of edited volumes by regular academic presses where it’s transparent that the standards for who gets to contribute are different from the way they are (or should be) in the case of journals. Why dilute the signaling power of having an article accepted in a journal by having special issues?
Anon Grad Student: Not all special issues are invite-based. Some, such as the forthcoming April 2015 Res Philosophica issue on Transformative Experiences, involve double- (or even triple-)anonymous review of submissions. I say this to say that not all ‘special issues’ are “dilut[ting] the signaling power of having an article accepted in a journal.” So just because an article appears in a special issue of a journal, that’s not always a reliable means for inferring that it wasn’t anonymously peer-reviewed.
Rachel McKinnon: That’s helpful, thank you. In that case, would be helpful to be upfront about this? Journals that have a two-tier system where most papers go through the normal anonymous review process and some papers don’t might have a note at the beginning of each paper that indicates how the decision to publish was reached. It strikes me that if this were to become the norm, it would solve many issues, including some that have been touched upon in the comments here. I would expect that not many, not even well established authors, want to have their paper published in a journal that officially acknowledges that the paper didn’t go through the usual process.
At Phil Review, we don’t forbid editors or other Cornell-affiliates (other faculty, grad students, recent grad students) from submitting, though it doesn’t happen often. The blind review process means that in principle we might not know that a paper is from an ‘affiliated’ person, but in practice, one of the editors usually recognizes such papers. We send them out for external review, preferably to two reviewers (unless we desk reject). If anything the bar is higher.
Rachel–that’s a bit misleading, as according to the Res Philosophica website there are six invited papers in that special issue.
I thought I would share my own story (though unlike JJI, I would prefer to remain anonymous). I wrote a paper as a grad student that was published in a “top 5” journal (i.e. either J Phil, Phil Rev, Nous, Mind, or PPR). My very well connected advisor wrote the editor of the journal and said the paper was great. It was accepted without even being looked at! The editor simply said that my advisor’s word was gold and my advisor’s high opinion of the paper was sufficient. I wasn’t going to complain either: I went out on the market the year everything collapsed and though I had also published without favors, I’m sure it didn’t hurt to also have another publication in such a good journal. I’m not sure how often this happens, but I know first-hand that it occasionally does.
Anonymous person: But the invited papers are clearly noted by the journal (on the website), so it’s not misleading. The other papers were anonymous-refereed (mine is one of the papers that went through the review process, for full disclosure). Do you still think it’s misleading?
http://www.resphilosophica.org/calls/transformative/
Can I add something to Anonymous Junior Prof’s story. I’ve been forwarded an email from someone who forgot (?) to delete an email from a former advisor that strongly suggested that they could arrange something similar to what Anon Junior Prof benefitted from. (In this case, they were told to try to get their work into a top journal but that they could use a certain very nice journal as an option if these didn’t work out. There wasn’t any explicit promise, but it was strongly suggested that the acceptance was in the bag if the person needed to use that option.)
Can we please, please take seriously the calls for transparency and oversight!
Rachel–I meant that your *comment* was misleading, not that the journal’s website was misleading.
David Estlund writes: “By the way, I don’t know about others, but I always decline to referee papers that centrally involve my own work (not that this issue arises all that often), or that is by or centrally concerns the work of a student of mine…I wonder if people think we should all abide by that.”
I agree it’s a question worth discussing. In a decision email, the editor of one of the journals already mentioned above wrote to me: “As is my practice, I also solicited comments on the piece from the author under discussion – not as a referee as such, merely as an interested party. [The author under discussion] generously waived anonymity, so I also include his comments below for your information.”
I agree that there should be transparency. But I’m not sure that there is necessarily a problem with this: “The editor of Philosophical Studies seems to be treating submissions in his own area of research interest very differently from the way he treats other submissions.” Won’t all editors be in a better position to know whether a paper is unsuitable for publication if the paper is in his or her own area of research interest, and won’t all editors desk-reject those papers that they know are unsuitable for publication? In other words, won’t all editors (who do at least some initial vetting) treat submissions in his or her own area of research differently in that they will be more likely to desk-reject those submissions given their greater competency with respect to such submissions? So although I agree that their should be transparency, that the editor should not be told the name of the author if he or she does some of the initial vetting, and that those submissions that meet some reasonable threshold should be normally be sent out for review, I don’t know that I agree that editors should never treat submissions in their own areas differently in the sense that I mention above. Do others disagree with me? I’m also curious whether it would be a mistake for an editor to desk accept a paper that she or he feels is clearly superb. But, of course, they could form such a clear opinion only if the submission in his or her own area.
I agree with Douglas Portmore that it is perfectly appropriate for editors to take advantage of their own philosophical expertise in their editorial capacities, and that there is a sense in which this means that an editor will appropriately respond differentially to a submission in the areas of his or her expertise. But there is also a sense in which an editor relying on expertise can treat all submissions the same: he or she can use his or her knowledge and judgment to assess the submission as carefully and objectively as possible.
The difference I was complaining about wasn’t that Cohen employed the additional knowledge about the field he happened to have; it’s that, prior to the review process, he decided which kind of process to employ, based in part on the topic of the paper. Whether a submission is anonymously refereed, for instance, shouldn’t depend on whether it’s on a topic in which the editor is particularly interested.
In response to Anonymous Junior Prof – Wow! Am I really naive, or is everyone else as shocked as me by these stories? To be clear: if I was in Junior Prof’s situation I am sure I would have accepted the good journal publication in a shot. (Perhaps with some tiny moral qualms in the back of my head, but still in a heartbeat.) But I am totally amazed that the ‘well-connected advisor’ would do that. That is some seriously dubious shit, no? She/he MUST have known how wrong that kind of influence-wielding special pleading is. Likewise I am pretty appalled and amazed that the editor of one of the top 5 journals would accept an article on that basis.
Hi Doug,
I think the issue is, in part, that there’s treating things differently and then there’s treating things differently. It’s one thing to desk reject and another to serve as a (solo?) referee repeatedly in a particular area. (I posted a version of this on facebook, so forgive me if you’ve seen this already.)
Some of us are better at distancing ourselves from things and judging papers on their merits and some of us are worse at it. Some of us think that we’re too likely to evaluate the merits of a paper on the basis of whether we antecedently like the conclusion. Some of us worry that we’re likely to be like that and overcompensate. You can pick whichever suboptimal editor you like, but I’ll go with a hypothetical editor that suffers from two vices: is too quick to dismiss the work of people who disagree with him and is too quick to judge that work is good if it supports conclusions he likes. (Vary as you see fit, but part of this fit part of Jonathan’s worry when he noted that the part of the paper he got the greatest resistance on was the part where he disagreed with the editor’s previous work. (As others have noted, this isn’t the kiss of death. Jonathan and others have had their work published in spite of this, but that’s consistent with the further claim that it’s an uphill battle that others might not face. That’s enough to get the worry off of the ground). My hypothetical editor is a crude caricature used to make a point. I’m not claiming that this is the editor in question. As I see it, it’s a distraction to focus on the editor in this case. This should be about general standards.)
Suppose this is how the journal runs. When papers on X come in, the editor reviews them on his own without going through the usual refereeing process. (We can do this with anonymous review or not. For this point, it doesn’t matter so much.) When non-X papers come in, the editor dishes them out at random to referees who, like the editor, suffer from two vices (i.e., is too quick to dismiss the work of people who disagree with them and too quick to judge that work is good if it supports views that they like).
Now, two points. First, if I don’t see eye to eye with the editor, I’d much rather be in the position of the people who get to submit non-X papers to this journal. It’s likely I’ll have to deal with referees who suffer from the vices described above, but there’s at least a chance that I don’t have to and there’s something good about the possibility that the forces can be mitigated a bit. Why should I be denied that opportunity? Wouldn’t it be nice if everyone had that opportunity?
Second, if this is an important journal, I’m not thrilled that I’m competing for jobs with people who have a lower bar to cross to get into the literature. Why shouldn’t we have the opportunity to compete on even terms for space in a prestigious journal?
These aren’t the complaints that Jonathan raised, but they are complaints that I think can be legitimately raised if an editor undercuts the standard review process and serve as a referee. (In the case Jonathan describes, the editorial process is not anonymous, but is that supposed to help?)
Like I said, you can vary the case in lots of ways. We can imagine more and less extreme versions of the above. We can imagine editors that overcompensate their suspected biases and give too much deference to people that disagree with them. The same basic kind of problem remains, I think. In my experience with this journal, I have received some feedback from the editor that has been useful. Some people have reported positive experiences. That shouldn’t be denied. So far as I can tell, none of the good that Phil Studies has done requires an editor taking over the responsibility of referees. And, so far as I can tell, we have some good reason to think that competition for a scarce and valuable good would be fairer if the journal practiced the normal sort of refereeing practice (i.e., shipping things to referees to get more eyes on a paper before deciding whether it’s in or out). As someone who sends papers to that journal, I’d like my work to get checked out by referees. It’s one thing to face the possibility of desk rejection for unpublishable work, but quite another not to get a chance to have your paper refereed in the normal way if it’s not the kind of work that should be desk rejected. (I didn’t see my rejections as desk rejections in the normal sense. They were lengthy exchanges about the merits of the editor’s objections.)
This is professional malfeasance pure and simple.
The swift response from Phil Studies was encouraging. Perhaps more journals would follow suit if others came forward with similar stories.
I think that I agree with what Clayton and Jonathan say in response to me. I was just making sure that people don’t object to editors treating such submissions differently in the minimal sense of using their specific expertise to discern that certain submissions fall below minimal acceptable standards while relying on reviewers when the submission falls outside of their expertise. Since I’m an editor now, I want to be as clear as possible about what the appropriate general standards are. And it seems to me appropriate for an editor to desk-reject certain submissions because he or she knows that they fall below acceptable scholarly standards. And it may be that the editor knows this only because he or she is very familiar with the relevant literature and thus competent to judge this without even sending it out to reviewers.
@ David Estlund [10], I accept there are limits to how blind one can reasonably expect reviews sometimes to be. That said, there is still a further question as to what the ethical requirements are on editors when it comes to submitting their work to their own journals. I don’t have a problem if they’ve made reasonable efforts to keep it anonymous from their own journal. If they haven’t (e.g. by presenting it at their fellow editors universities), then they ought to submit it to a different journal. Given that’s what I think the ethical requirements are on the editors regarding submitting their own work, I think the best practice for a journal would be to encourage their editors to meet these requirements.
Moreover, I don’t agree that we’re dealing with “the bare possibility of favoritism.” If an unknown author submitted a 83-page two-part epic to a journal with the length-policy I quoted, would it get sent out for review or would it be immediately returned as too long? I think it’s reasonable to assign non-trivial credence to the answer that it’d be returned, in which case the reasonable inference is that Kolodny was favorably treated. What is your credence that the unknown author’s article would get returned as too long?
But her comment wasn’t misleading. It is not misleading to say that the issue in question involved blind refereeing. It did involve blind refereeing — indeed, Rachel’s paper (among others) were blind refereed and, further, those papers that were not blind refereed are clearly marked as such. So, it seems like she was exactly right when she said that, “just because an article appears in a special issue of a journal, that’s not always a reliable means for inferring that it wasn’t anonymously peer-reviewed.”
I realize I’m coming late to this, but I had a similar experience as Jonathan with a prestigious philosophy journal many years ago, but with what I think are some important differences that are worth adding to the conversation. (I agree that focusing on which journal/editor isn’t what’s relevant, since we’re trying to move forward, so I won’t specify further.)
After six or so months, I inquired about my paper through the journal’s online system and was told by direct email from the editor that there would be a decision within two weeks. After more than two weeks passed, I inquired again, this time to the editor’s direct email, and received a rejection later the same day, with some comments, none of which seemed to me like grounds for rejection (they either required only easy fixes to non-central points or engendered some misunderstandings – or so it seemed to me at least). So I wrote up harsh replies to each point – I just reread them and had some good laughs at the level of my nerve – and sent them to the editor. Even though the paper had been rejected, I said, perhaps he/she could get some use out of my replies, since I had been told he/she was also currently working on the topic.
He/she replied that the paper had indeed been rejected, but as editor he/she was leaving open the possibility of rescinding the decision, and also noted he/she was the author of the comments. He/she said that he/she was indeed working on the topic, sent me his/her paper in case I would be willing to offer comments, and mentioned that we actually say similar things about one aspect of the topic.
We had a back and forth about my paper for a couple weeks, until I sent him/her about 2000 words of comments on his/her draft. Then the correspondence ended – he/she never replied to my last set of responses, to his/her responses, to my replies, to… So I just dropped it. After a couple more bad experiences with refs working in that subfield, I dropped the project and swore off the subfield.
Some differences between my case and Jonathan’s: 1) I was still in graduate school at a 20-30 Gourmet ranked program at the time, not a minted PhD from a top program. 2) I inadvertently rushed the editor-reviewer on his/her decision, which may have had an effect on things. 3) Not realizing the editor was the reviewer, I sent snarky replies as my first step in the “take back the rejection” process (really, not even realizing that’s what process we were in!). 4) The editor was just starting his/her work on the topic, as opposed to having already been well published in it. (In fact, it took a few years for his/her paper on the topic to appear in print, and I just looked at the two versions, and it looks like quite of bit of vetting of the ideas occurred in the meantime.) 5) My article presented some insights that the editor was also trying to get in print for himself/herself. And 6) the whole process played some role in my refocusing on a different area of research.
Glad we’re finally getting serious about these issues.
I appreciate Ichikawa’s anecdote. Recently, I submitted a paper to Phil Studies and settled in for the expected months-long wait (unless the paper was desk-rejected as an earlier paper of mine had been). I watched for months the status of the paper remain at editor invited, thinking that this was the journal’s parlance for referees assigned. The status never changed until I received an email notifying me that the paper had been rejected with no associated referee comments. Now, of course, it’s possible that referees had been assigned and the paper had been reviewed and rejected without comments, but I am suspicious that the paper never even got a read, given Ichikawa’s experience. I don’t have the advantage of knowing the editors or working near them. Therefore, a related problem with practices such as this is not only that the process is unfair, but that submitters don’t even know whether their paper got a fair shot or not. I genuinely would have valued a comment in the email that no referees had actually been assigned. At least then, I would know that the rejection had no bearing on the quality of the paper.
Ok, I am also now pretty amazed by Grad @ the time’s story (35 above).
I have NEVER heard of an editor rejecting a paper but then when the author sends a bunch of outraged replies to the referees comments, changing his/her mind and allowing that maybe the paper will be accepted after all! Has that sort of thing happened to anyone else?! Does everyone else send replies back in response to the referee’s comments when they get rejected, hoping perhaps the decision will be reversed? I have, on a number of occasions, received referee comments that I thought were dumb and unfair and totally missed my point etc. But it never occurred to me that I should start emailing my grievances to the journal’s editor in case I might convince them that the referee got it wrong.
In all seriousness – do other people do that? Should I start?
Philosophy and Public Affairs seems to me to have a real problem with bias. Because I suspected that it had such a problem, I went through back issues last year, looking at volumes 37, 38, 39, 40 and 41, in which there are 57 papers. Of those 57 papers, 17 had at least one author who received their doctorate from the same institution as the current Editor, Oxford. Another 13 were written by authors whose doctorates were from Harvard, where a number of editorial staff work or studied. The next most common place for authors to have received their doctorate is Princeton, where the journal is based. Of the ten papers in those five volumes published by authors who were last year employed by institutions in the western United States, four are by members of the editorial staff, three are by former visitors at Princeton, where the journal is based, and two are by people who were supervised by members of editorial staff. Only one of the ten has an author with no obvious links with editorial staff. It is impossible to be sure, simply on the basis of that sort of data, that there is anything untoward about its editorial policy, let alone anything deliberately so. I suggest though that it places the burden of proof on those who claim that it assesses all submissions in the same way.
“But it never occurred to me that I should start emailing my grievances to the journal’s editor in case I might convince them that the referee got it wrong. In all seriousness – do other people do that? Should I start?”
Sure, why not? I’ve had mixed luck with this. It helps to do it in the right way, though. Use sparingly and write respectfully. In my experience, you’re not going to get anywhere with this if the issue is a substantive one. I suspect that if your objection to the report is that the referee’s objections were poor, you’re out of luck. Editors don’t usually have the time or inclination to get involved in that. If, however, the referee’s reasons for rejecting are poor and you can demonstrate this without having to get the editor involved in some substantive philosophical issue, a respectful email just might turn things around. On more than one occasion, I’ve turned rejections into R&Rs when I can show that the referee didn’t accurately describe my view or just overlooked some crucial section of the paper. (Most recently, I had a referee say that while my objections clearly ruled out X, there was another view, Y, that survived these objections. I pointed to a passage in the first part of the paper where I clearly stated that said that my aim was to show that X was mistaken and that Y was a better view. The editors agreed that the referee’s complaint simply amounted to the claim that my paper accomplished precisely what I claimed it would so they sent it to a different referee.)
I think that the appropriateness of the practice you describe depends on a few things. I think that non-blinded desk rejections are a worrisome practice, though I imagine they are somewhat common. Desk rejections involve a quick judgment of the worth of a paper, and typically are not the result of a careful reading of the paper. If the person making that decision can see the name and affiliation of the author, it is hard to imagine there not being large biases in favor of better known authors or more prestigious institutions. I just can’t imagine someone who knows philosopher X and thinks X’s work is generally good, really avoiding being influenced by that in deciding whether to desk reject. This puts more junior authors and less well-known authors at a massive disadvantage for even getting past the initial gate-keeping stage. If the editor is blind with respect to authorship and affiliation, then it seems like a reasonable practice, however, even in that case, the editor should have some conscious policy in place to help them avoid some of Clayton’s worries. If 80% of submissions in a particular topic have to get through the approval of one specific person before even being refereed, it really matters whether that person is disposed to think poorly of papers about particular views or debates. It seems much better for that editor’s views on the merits of the view or debate to come into play later, after other expert advice has been given, so that they have a check against their own (potentially unjust) inclinations.
My desk-rejections are blind. A managerial editor sends me the anonymized submission and then I decide whether to send it out for review or give it a desk-rejection.
I think that the issue raised in 38 above about PaPA is slightly different from the issues raised in this thread. The patterns identified (prevalence of Oxford and Harvard students, etc) are consistent with the editors not being aware of the authors’ identities, not engaging in the strangely personal feedback given by Cohen etc, not being influenced by famous advisers, etc. Instead, the patterns probably point to that journal functioning as the key publication venue for a particular brand of political philosophy, which tends to be strongest at Oxford, Harvard, Princeton etc. None of this is to say that the publication patterns aren’t potentially problematic, because to the extent that publication in PaPA really boosts employment chances in political philosophy they might lead to a narrow set of interests dominating the field, but it is to say that picking on PaPA in this context seems unfair and unwarranted.
Let me get this straight: 57 papers, 30 of which from people with doctorates from either Harvard or Oxford? Seriously, people?
Why do we still consider a publication in PPA a career-making achievement? This data ought to be brought to the attention of the APA, the journal’s publisher, or whoever has the power to do anything about this. Which is probably nobody, and so the philosophical upper crust continues to do as it pleases, and we all have to play along with the pretend meritocracy, or else.
I think it’s important to separate out two different sets of issues:
1. Fairness in access to journal venues, understood as key factors in employment, promotion, and professional reputation.
2. The effect of bias and possible bias on the promotion and dissemination of good scholarship.
Having separated them out, I think we have to acknowledge that one of these goals may have to be traded off against the other. For example, while there are some epistemic costs to a system that makes it more likely that editors and their close associates will get papers published in it, it’s not clear that these would outweigh the costs of either a) not allowing such editors (and their close associates) to publish in journals that are the best fit for their work or b) requiring such editors (and their close associates) to keep their works in progress secret to preserve the blindness of the review process. But from the standpoint of fairness in access to professionally important goods, neither a) or nor b) are likely to carry much weight (those in editorial positions at prestigious journals already having sufficient access to such goods).
Surely the *type* of political philosophy done at Harvard, Oxford and Princeton is also done in the vast majority of other departments. If not those places wouldn’t be leading the discipline. Oh, you say, but the best journal carries papers by the best people, who got their PhD from the best departments. We know they’re the best people because they publish in the best journal. Or was it because of where they got their PhD?
I truly hope this is recognized for the scandal it is. It threatens the basic integrity of the profession. 30 of 57 articles from people with doctorates from THREE programs directly related to PPA’s editorial board? This confirms what many of us have long known. The system is rigged, and something must be done about it.
Two programs, actually: Harvard and Oxford. If we add Princeton the number raises.
Here’s a thought: maybe the problem is not exclusively journal *policies* (although, as the multitude of troubling comments here demonstrate that *too* is a problem), but also the very fact that we continue to *rank* journals. Especially after reading this thread, I strongly suspect, that if someone were so inclined, they could show how journal rankings are just as ephemeral and just as problematic as overall philosophy rankings (see Mitchell Aboulafia’s excellent work in this regard).
Professional philosophy seems to have a perverse addiction to rankings — maybe it is time to seek help.
In regards to “well connected” professors helping their graduate students to get published in top journals: this would presumably help the student, after completion, to walk into an academic job; and would in turn help departments who publish their previous students employment records.
The privileged will also be invited to give talks at other departments and conferences, thereby making many professional connections, and to publish their work in edited volumes. And so the Matthew Effect will make all these decisions appear obviously correct in retrospect.
Thank you for doing this work.