Guarding the Guardians (or Editors) (3 updates)


The reason your paper is listed as ‘editor assigned’ is that I’m going to review it myself.

In the wake of the recent discussion here about the editorial practices at philosophy journals,  Jonathan Jenkins Ichikawa (UBC) recounts a story, set about five years ago, in which he submitted a paper to a journal with a policy of double-anonymous reviewing—Philosophical Studies—but which clearly, and quite blatantly, did not make its decision using such a procedure. Instead, the editor, Stewart Cohen, who became Ichikawa’s colleague during the process, engaged in various email and in-person exchanges about the article, revisions to it, and its ultimate acceptance. Ichikawa recounts the story at his blog. He writes:

The editor of Philosophical Studies seems to be treating submissions in his own area of research interest very differently from the way he treats other submissions, sometimes deciding to referee papers himself. As far as I know, Cohen was the sole referee for my submission… On the whole I’m very confident in asserting that in at least one instance, Philosophical Studies engaged in seriously problematic editorial procedures. I don’t know how atypical my story is.

UPDATE: Stewart Cohen, editor of Philosophical Studies, has asked me to post the following:

Philosophical Studies is now in the process of instituting a triple-blind review system.  Neither the editor, nor the associate editors will know the identity of the authors until after the decision has been made (except for special issues and symposia which are invited). [Edit: Cohen asked that the preceding struck-through text be replaced with the following:]  All submissions will be triple-blind reviewed.  Invited papers will be labeled as such.

UPDATE 2 (2/4/15): I would like to get a better sense of the extent and types of these kinds of problems. Publicizing the problematic episodes and naming the journals will help with that, serving to raise awareness of the problems as well as possibly deter them. If a paper of yours received what seems like inappropriate treatment by a member of the editorial team at a journal (as with Ichikawa, Anonymous (comment #4), Clayton (#12), anonymous junior prof (#21), Grad @ the time (#35)), please describe it in the comments. I encourage you to name the journal. Anonymous comments will be accepted, provided that they are submitted with a real email address by which I can verify the identity of the author. Your email does not get published when your comment is approved, and, as usual, I will not reveal the identity of anonymous commenters. Please refrain from posting second-hand stories (e.g., “this once happened to a friend of mine…”). NOTE: I have received a few comments recounting alleged editorial malpractice in which the journals are named, but the authors have failed to provide real email addresses. These comments will not get posted. Feel free to resubmit your comment with a real email address I can use to verify your identity, if I need to.

UPDATE 3 (2/7/15): One of the journals receiving a lot of attention in the discussion below is Philosophy and Public Affairs. Its editor, Alan Patten, has provided the following statement:

Readers of the blog might be interested in the statement about editorial practices that is posted on Wiley’s Philosophy & Public Affairs site under Author Guidelines

Some of the questions about the journal’s procedures that have been raised in this forum are addressed in the statement, including questions about our policies on anonymity, on lengthy submissions, and on submissions by associate editors. Concerns about fairness and bias are certainly valid in this area, and there are some genuinely tough questions such as whether to publish articles by our associate editors. I’m sure we could do better in some respects, but on the whole I think our procedures do a reasonable job of balancing some of the competing goals and demands of running a journal.

I’m disturbed by the insinuation in one or two posts that there was something corrupt about our decision to publish Niko Kolodny’s recent two-part article. I acknowledge that there might be reasonable disagreement about whether a journal should have a blanket policy of not publishing papers by current editors. But that point aside our adherence to our editorial policies could not have been more rigorous in this case. In addition to sending the paper to an external reader (as described in the statement) I confirmed with that reader at the start of the process that s/he was not already familiar with the paper, did not know the identity of the author, and had no presumptions either way. I should add that it’s not as if there’s been a great many articles published by associate editors in recent years. We’ve accepted a grand total of two such articles (including the two-parter by Niko) under my editorship (which began in 2010). We’ve also declined several submissions by editors in this timespan. In any case, I’m proud that we were able to publish Niko’s article. If you take the time to read it, I doubt that you will conclude that the best explanation of why we published it is anything other than its excellence.

Warwick University MA in Philosophy
Subscribe
Notify of
guest

119 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
ER
ER
9 years ago

Kudos to Ichikawa.

Sal Mandry
Sal Mandry
9 years ago

If this didn’t happen all the time one would be hard pressed to explain the recurrent contributions by certain people to journals like Journal of Philosophy, Philosophy & Public Affairs and Journal of Political Philosophy. Those are just some of the most obvious examples.

Tom Hurka
Tom Hurka
9 years ago

Philosophy and Public Affairs just published a two-part article by one of its Associate Editors. It may be a fine article, but I would have thought it obvious that every journal should have a policy of never publishing work by its main editors or reviews of their books. (It was certainly the policy of The Canadian Journal of Philosophy when I was an editor there.) I’d be interested to know if others share this view. Philosophy and Public Affairs evidently doesn’t, nor in the past has Ethics. But the claim that review has been blind in this kind of case is rarely credible — you really had no idea who the author was??? — and in any case I think it’s important that journals not allow even the appearance of favouritism. That’s a standard we hold public officials to — they can’t say they received the gift from the developer but weren’t influenced by it — and I think journals should abide by it too. I’m amazed when they don’t.

Anonymous
Anonymous
9 years ago

Cohen refereed a paper that I submitted to Phil Studies some years ago. Very unprofessional: an editor should follow the editorial policy printed in the journal.

[Note: this anonymous post was submitted from a verified, legitimate email, and so is not anonymous to me. No unconfirmable accusations or rumors will be permitted.]

Dave Estlund
9 years ago

Tom, it certainly raises a fair question. Of course, top journals (especially) will often receive submissions that many people in the field would easily know the authorship of (this doesn’t address Jonathan’s case), so that cannot be entirely avoided even if editors were barred. Also, it’s not clear that we want journals to be edited only by people who are willing to forgo publishing there. It’s not absurd, but it’s not clear. I have just signed on as an Associate Editor with P&PA (though I have not yet begun my duties). All things considered, I think that stricture might have tipped me into declining, at least if the expectation is (as it currently is) that I will try to serve for more than a couple of years. Not that I’d be such a great loss, but it might be that others whose service to such journals is very desirable would feel the same way. One work-around might be to expect that sacrifice, but ask for only a two-year commitment, but that has disadvantages too, as a journal burns through their top choices for editors in the course of a few years.

Matt
9 years ago

Tom Hurka said, “I would have thought it obvious that every journal should have a policy of never publishing work by its main editors or reviews of their books.”

I don’t always agree with Hurka, but this seems to me to be pretty clear, at least if journals don’t want to have a reputation for various bits of favoritism or in-dealing. (Some seem not to mind this.) But, it seems to in fact to be controversial. In way way I think this is a pathology of philosophy – the ability to construct arguments tends to make people able to construct arguments that are at least _able to convince them_ that, say, they are being impartial,even when it’s in fact quite implausible that they are (and certainly not at all obvious to those outside!)

Lumpen
Lumpen
9 years ago

People at the center of circles of privilege may not think it’s that much of a big deal if X get a spot in PPA as opposed to Y. Sure, let’s help a friend out, what goes around and all that. People with the right sort of pedigree will make it regardless, right? Who cares if some nobody’s entire livelihood has been destroyed by the fact that someone in this “top” journal’s inner circle took over half an issue with his musings, in all likelihood refereed by another friend and newly appointed associate editor, whose work takes center stage in the gargantuan 73-page article?

Anonymous2
Anonymous2
9 years ago

I got the same email from Cohen, and my paper was ultimately rejected. I was a graduate student. The comments that were included were very helpful–I’m not sure whether they were from Cohen or a second referee.

Anonymous political philosopher
Anonymous political philosopher
9 years ago

Regarding the P&PA article: I don’t think that David Estlund’s response adequately addresses Tom Hurka’s concerns, which I take to focus on articles having the combination of (i) effectively non-anonymous and (ii) written by editors. Estlund responds to each point separately, but not their combination: their combination can be avoided by allowing editors to submit works only if they have taken all reasonable steps to ensure anonymity e.g. by not sharing the work in progress with other editors. The problem of their combination is obvious if you imagine what it would be in the position of an editor reviewing the work of another editor when (a) you know it is likely that they know it is you reviewing; and (b) you plan on submitting your own work for them to review. As well as the question of what best practices are for a journal (and what is clear to Hurka is also what is clear to me), there is a question of what the individual editor is morally required to do. If an editor intends to submit a paper to their own journal, would it really be such a hardship to avoid presenting it in venues where one can predict one’s fellow editors to be likely to be present?

If you look at the thanks for that paper, you will see that Niko Kolodny thanks for written comments, Alan Patten (the main editor of P&PA), as well as Arthur Ripstein and Jay Wallace (associate editors) and that he presented the paper at UCLA (the university of Barbara Herman and Seana Shiffrin who are also associate editors). This leaves at most exactly one member of the editorial board (Stuart White) who Kolodny would not expect to be aware of the paper. Given that a significant amount of the reviewing at the journal is done by editors, Hurka is surely right that the journal has allowed at least “the appearance of favouritism,” particularly when it publishes a two-part paper of 83 pages and its website states “We prefer papers of fewer than 12,000 words (including notes) and may return longer papers to their authors unreviewed with the suggestion that they be resubmitted in shorter form.”

As well as the obvious issue of fairness, my impression is that this behavior is very damaging for the journal’s reputation — I remember conference chat of many years ago about P&PA’s reputation for publishing papers by editors and their grad students. I’m sure I’m not the only person to have reflected on the fact that the political philosophers published in Ethics are much more diverse than those published in P&PA. As the online journal rankings repeatedly show, Ethics’s reputation has exceeded P&PA’s for a while now. I wouldn’t be surprised if these facts are linked.

Dave Estlund
9 years ago

Responding to Anonymous [9]: Sometimes a lead editor will recognize the authorship of a paper even if it has been anonymized. In that case fully blind review is impossible. One could argue that the journal should immediately reject in that case, though I’m not at all sure that’s reasonable. But suppose the editor also realizes that it is by an associate editor. Again, I doubt that it’s reasonable to let this immediately disqualify it, though I’d appreciate hearing from those who disagree. I do think that in that case it would be important to get reports from people who are as independent as possible. Suppose Nagel submits a paper while he is associate editor, and it is easy for the editor to realize who it is by. I do believe it should receive serious and independent review so far is possible, but I’m not sure it should be disqualified.

Whenever an associate editor publishes in the journal, I suppose it is open to people to suspect that it was accepted only through favoritism. The ethical requirement is to make sure that that is not the case. It is also important to the profession to avoid the appearance of favoritism, but the bare possibility of favoritism is not the appearance of favoritism. There is a limit to how publicly this can be policed, and in the end what matters is the quality of the published work. Blind review is an excellent instrument toward ensuring acceptance of the best work, but (I believe) few if any journals disqualify referees who know the authorship of the reviewed piece (Ethics does not), and this is because that would work against the goal of reviewing and publishing the best work. I think similar leeway is sensible in the case of publishing work by a journal’s editors so long as it is reviewed as independently as possible. The fact that an excellent journal such as Ethics is often ranked even more highly than PPA is pretty slim evidence that PPA is corrupt.

As for the thinly veiled suggestion by Lumpen [7] that I played any role at all, formally or informally or in any way, in the review of either of Kolodny’s pieces, I have excellent news: the conjecture is entirely mistaken. Lumpen also depressingly conjectures that I or someone thinks it doesn’t matter whether favoritism plays a role or not. On what basis, I have no idea. Of course it matters enormously. Tom Hurka’s question about submissions by a journal’s own editors is (as I indicated) a serious one, and not simple.

Finally, I would be curious to know which journals formally forbid publication by any editors or associate editors. If that is working for some top journals then other journals ought to consider it. (It would not present the same challenges to a lower-tier journal for obvious reasons.)

Dave Estlund
9 years ago

By the way, I don’t know about others, but I always decline to referee papers that centrally involve my own work (not that this issue arises all that often), or that is by or centrally concerns the work of a student of mine. (In any case, I was never even consulted at any stage about the papers in question.) I wonder if people think we should all abide by that.

Clayton
9 years ago

Same experience as Jonathan, sans the acceptance. Why aren’t we putting pressure on journals to be transparent and practice triple-blind review?

ProbablyMissingSomething
ProbablyMissingSomething
9 years ago

Not being an academic, I don’t understand how submissions from relatively well-published people could be anonymous. Isn’t it customary to refer the reader, via footnotes, to one’s other publications addressing the issues that are pertinent to, yet not the focus of, the current article? I would be surprised, for instance, if Tim Scanlon wrote an article about any issue in normative ethics without referencing ‘What We Owe to Each Other’ or ‘Being realistic about reasons’ somewhere in the text. And for good reason (to show that he isn’t just assuming certain things without argument; to direct the interested/skeptical reader to relevant/prior arguments elsewhere etc.). Are papers sent to journals without such footnotes? Or with such footnotes but with empty placeholders instead of the actual names of the articles/books? If the former, I can’t imagine how big assumptions, controversial theses or apparent gaps in argument could get past the editor’s desk. If the latter, any competent referee would be able to figure out who the writer is. Admittedly, Scanlon is a pretty big name, hence not an ordinary case but I am guessing that editors and especially referees for top journals could do this for a LOT of the people working in their areas of research. What am I missing? (The only other alternative I can think of is explicitly referring to the texts but in third person as if you were someone else, but that is just absurd to me for a number of reasons.)

anon
anon
9 years ago

“Whenever an associate editor publishes in the journal, I suppose it is open to people to suspect that it was accepted only through favoritism.”

Why *only*? Nobody thinks that. But still there is a reasonable suspicion of favoritism.

Anon Grad Student
Anon Grad Student
9 years ago

Stewart Cohen’s statement in the update is encouraging. However, it raises another question: Why do journals have special issues? There’s already an abundance of edited volumes by regular academic presses where it’s transparent that the standards for who gets to contribute are different from the way they are (or should be) in the case of journals. Why dilute the signaling power of having an article accepted in a journal by having special issues?

Rachel McKinnon
Rachel McKinnon
9 years ago

Anon Grad Student: Not all special issues are invite-based. Some, such as the forthcoming April 2015 Res Philosophica issue on Transformative Experiences, involve double- (or even triple-)anonymous review of submissions. I say this to say that not all ‘special issues’ are “dilut[ting] the signaling power of having an article accepted in a journal.” So just because an article appears in a special issue of a journal, that’s not always a reliable means for inferring that it wasn’t anonymously peer-reviewed.

Anon Grad Student
Anon Grad Student
9 years ago

Rachel McKinnon: That’s helpful, thank you. In that case, would be helpful to be upfront about this? Journals that have a two-tier system where most papers go through the normal anonymous review process and some papers don’t might have a note at the beginning of each paper that indicates how the decision to publish was reached. It strikes me that if this were to become the norm, it would solve many issues, including some that have been touched upon in the comments here. I would expect that not many, not even well established authors, want to have their paper published in a journal that officially acknowledges that the paper didn’t go through the usual process.

Karen Bennett
Karen Bennett
9 years ago

At Phil Review, we don’t forbid editors or other Cornell-affiliates (other faculty, grad students, recent grad students) from submitting, though it doesn’t happen often. The blind review process means that in principle we might not know that a paper is from an ‘affiliated’ person, but in practice, one of the editors usually recognizes such papers. We send them out for external review, preferably to two reviewers (unless we desk reject). If anything the bar is higher.

anonymous person
anonymous person
9 years ago

Rachel–that’s a bit misleading, as according to the Res Philosophica website there are six invited papers in that special issue.

anonymous junior prof
anonymous junior prof
9 years ago

I thought I would share my own story (though unlike JJI, I would prefer to remain anonymous). I wrote a paper as a grad student that was published in a “top 5” journal (i.e. either J Phil, Phil Rev, Nous, Mind, or PPR). My very well connected advisor wrote the editor of the journal and said the paper was great. It was accepted without even being looked at! The editor simply said that my advisor’s word was gold and my advisor’s high opinion of the paper was sufficient. I wasn’t going to complain either: I went out on the market the year everything collapsed and though I had also published without favors, I’m sure it didn’t hurt to also have another publication in such a good journal. I’m not sure how often this happens, but I know first-hand that it occasionally does.

Rachel McKinnon
Rachel McKinnon
9 years ago

Anonymous person: But the invited papers are clearly noted by the journal (on the website), so it’s not misleading. The other papers were anonymous-refereed (mine is one of the papers that went through the review process, for full disclosure). Do you still think it’s misleading?

http://www.resphilosophica.org/calls/transformative/

A Different Anonymouse
A Different Anonymouse
9 years ago

Can I add something to Anonymous Junior Prof’s story. I’ve been forwarded an email from someone who forgot (?) to delete an email from a former advisor that strongly suggested that they could arrange something similar to what Anon Junior Prof benefitted from. (In this case, they were told to try to get their work into a top journal but that they could use a certain very nice journal as an option if these didn’t work out. There wasn’t any explicit promise, but it was strongly suggested that the acceptance was in the bag if the person needed to use that option.)

Can we please, please take seriously the calls for transparency and oversight!

anonymous person
anonymous person
9 years ago

Rachel–I meant that your *comment* was misleading, not that the journal’s website was misleading.

Paul Kelleher
Paul Kelleher
9 years ago

David Estlund writes: “By the way, I don’t know about others, but I always decline to referee papers that centrally involve my own work (not that this issue arises all that often), or that is by or centrally concerns the work of a student of mine…I wonder if people think we should all abide by that.”

I agree it’s a question worth discussing. In a decision email, the editor of one of the journals already mentioned above wrote to me: “As is my practice, I also solicited comments on the piece from the author under discussion – not as a referee as such, merely as an interested party. [The author under discussion] generously waived anonymity, so I also include his comments below for your information.”

Douglas W. Portmore
9 years ago

I agree that there should be transparency. But I’m not sure that there is necessarily a problem with this: “The editor of Philosophical Studies seems to be treating submissions in his own area of research interest very differently from the way he treats other submissions.” Won’t all editors be in a better position to know whether a paper is unsuitable for publication if the paper is in his or her own area of research interest, and won’t all editors desk-reject those papers that they know are unsuitable for publication? In other words, won’t all editors (who do at least some initial vetting) treat submissions in his or her own area of research differently in that they will be more likely to desk-reject those submissions given their greater competency with respect to such submissions? So although I agree that their should be transparency, that the editor should not be told the name of the author if he or she does some of the initial vetting, and that those submissions that meet some reasonable threshold should be normally be sent out for review, I don’t know that I agree that editors should never treat submissions in their own areas differently in the sense that I mention above. Do others disagree with me? I’m also curious whether it would be a mistake for an editor to desk accept a paper that she or he feels is clearly superb. But, of course, they could form such a clear opinion only if the submission in his or her own area.

Jonathan Ichikawa
9 years ago

I agree with Douglas Portmore that it is perfectly appropriate for editors to take advantage of their own philosophical expertise in their editorial capacities, and that there is a sense in which this means that an editor will appropriately respond differentially to a submission in the areas of his or her expertise. But there is also a sense in which an editor relying on expertise can treat all submissions the same: he or she can use his or her knowledge and judgment to assess the submission as carefully and objectively as possible.

The difference I was complaining about wasn’t that Cohen employed the additional knowledge about the field he happened to have; it’s that, prior to the review process, he decided which kind of process to employ, based in part on the topic of the paper. Whether a submission is anonymously refereed, for instance, shouldn’t depend on whether it’s on a topic in which the editor is particularly interested.

Anonymous Post Doc
Anonymous Post Doc
9 years ago

In response to Anonymous Junior Prof – Wow! Am I really naive, or is everyone else as shocked as me by these stories? To be clear: if I was in Junior Prof’s situation I am sure I would have accepted the good journal publication in a shot. (Perhaps with some tiny moral qualms in the back of my head, but still in a heartbeat.) But I am totally amazed that the ‘well-connected advisor’ would do that. That is some seriously dubious shit, no? She/he MUST have known how wrong that kind of influence-wielding special pleading is. Likewise I am pretty appalled and amazed that the editor of one of the top 5 journals would accept an article on that basis.

Clayton
9 years ago

Hi Doug,

I think the issue is, in part, that there’s treating things differently and then there’s treating things differently. It’s one thing to desk reject and another to serve as a (solo?) referee repeatedly in a particular area. (I posted a version of this on facebook, so forgive me if you’ve seen this already.)

Some of us are better at distancing ourselves from things and judging papers on their merits and some of us are worse at it. Some of us think that we’re too likely to evaluate the merits of a paper on the basis of whether we antecedently like the conclusion. Some of us worry that we’re likely to be like that and overcompensate. You can pick whichever suboptimal editor you like, but I’ll go with a hypothetical editor that suffers from two vices: is too quick to dismiss the work of people who disagree with him and is too quick to judge that work is good if it supports conclusions he likes. (Vary as you see fit, but part of this fit part of Jonathan’s worry when he noted that the part of the paper he got the greatest resistance on was the part where he disagreed with the editor’s previous work. (As others have noted, this isn’t the kiss of death. Jonathan and others have had their work published in spite of this, but that’s consistent with the further claim that it’s an uphill battle that others might not face. That’s enough to get the worry off of the ground). My hypothetical editor is a crude caricature used to make a point. I’m not claiming that this is the editor in question. As I see it, it’s a distraction to focus on the editor in this case. This should be about general standards.)

Suppose this is how the journal runs. When papers on X come in, the editor reviews them on his own without going through the usual refereeing process. (We can do this with anonymous review or not. For this point, it doesn’t matter so much.) When non-X papers come in, the editor dishes them out at random to referees who, like the editor, suffer from two vices (i.e., is too quick to dismiss the work of people who disagree with them and too quick to judge that work is good if it supports views that they like).

Now, two points. First, if I don’t see eye to eye with the editor, I’d much rather be in the position of the people who get to submit non-X papers to this journal. It’s likely I’ll have to deal with referees who suffer from the vices described above, but there’s at least a chance that I don’t have to and there’s something good about the possibility that the forces can be mitigated a bit. Why should I be denied that opportunity? Wouldn’t it be nice if everyone had that opportunity?

Second, if this is an important journal, I’m not thrilled that I’m competing for jobs with people who have a lower bar to cross to get into the literature. Why shouldn’t we have the opportunity to compete on even terms for space in a prestigious journal?

These aren’t the complaints that Jonathan raised, but they are complaints that I think can be legitimately raised if an editor undercuts the standard review process and serve as a referee. (In the case Jonathan describes, the editorial process is not anonymous, but is that supposed to help?)

Like I said, you can vary the case in lots of ways. We can imagine more and less extreme versions of the above. We can imagine editors that overcompensate their suspected biases and give too much deference to people that disagree with them. The same basic kind of problem remains, I think. In my experience with this journal, I have received some feedback from the editor that has been useful. Some people have reported positive experiences. That shouldn’t be denied. So far as I can tell, none of the good that Phil Studies has done requires an editor taking over the responsibility of referees. And, so far as I can tell, we have some good reason to think that competition for a scarce and valuable good would be fairer if the journal practiced the normal sort of refereeing practice (i.e., shipping things to referees to get more eyes on a paper before deciding whether it’s in or out). As someone who sends papers to that journal, I’d like my work to get checked out by referees. It’s one thing to face the possibility of desk rejection for unpublishable work, but quite another not to get a chance to have your paper refereed in the normal way if it’s not the kind of work that should be desk rejected. (I didn’t see my rejections as desk rejections in the normal sense. They were lengthy exchanges about the merits of the editor’s objections.)

Anonymous
Anonymous
9 years ago

This is professional malfeasance pure and simple.

Sacco
Sacco
9 years ago

The swift response from Phil Studies was encouraging. Perhaps more journals would follow suit if others came forward with similar stories.

Douglas W. Portmore
9 years ago

I think that I agree with what Clayton and Jonathan say in response to me. I was just making sure that people don’t object to editors treating such submissions differently in the minimal sense of using their specific expertise to discern that certain submissions fall below minimal acceptable standards while relying on reviewers when the submission falls outside of their expertise. Since I’m an editor now, I want to be as clear as possible about what the appropriate general standards are. And it seems to me appropriate for an editor to desk-reject certain submissions because he or she knows that they fall below acceptable scholarly standards. And it may be that the editor knows this only because he or she is very familiar with the relevant literature and thus competent to judge this without even sending it out to reviewers.

Anonymous political philosopher
Anonymous political philosopher
9 years ago

@ David Estlund [10], I accept there are limits to how blind one can reasonably expect reviews sometimes to be. That said, there is still a further question as to what the ethical requirements are on editors when it comes to submitting their work to their own journals. I don’t have a problem if they’ve made reasonable efforts to keep it anonymous from their own journal. If they haven’t (e.g. by presenting it at their fellow editors universities), then they ought to submit it to a different journal. Given that’s what I think the ethical requirements are on the editors regarding submitting their own work, I think the best practice for a journal would be to encourage their editors to meet these requirements.

Moreover, I don’t agree that we’re dealing with “the bare possibility of favoritism.” If an unknown author submitted a 83-page two-part epic to a journal with the length-policy I quoted, would it get sent out for review or would it be immediately returned as too long? I think it’s reasonable to assign non-trivial credence to the answer that it’d be returned, in which case the reasonable inference is that Kolodny was favorably treated. What is your credence that the unknown author’s article would get returned as too long?

LLH
LLH
9 years ago

But her comment wasn’t misleading. It is not misleading to say that the issue in question involved blind refereeing. It did involve blind refereeing — indeed, Rachel’s paper (among others) were blind refereed and, further, those papers that were not blind refereed are clearly marked as such. So, it seems like she was exactly right when she said that, “just because an article appears in a special issue of a journal, that’s not always a reliable means for inferring that it wasn’t anonymously peer-reviewed.”

Grad @ the time
Grad @ the time
9 years ago

I realize I’m coming late to this, but I had a similar experience as Jonathan with a prestigious philosophy journal many years ago, but with what I think are some important differences that are worth adding to the conversation. (I agree that focusing on which journal/editor isn’t what’s relevant, since we’re trying to move forward, so I won’t specify further.)

After six or so months, I inquired about my paper through the journal’s online system and was told by direct email from the editor that there would be a decision within two weeks. After more than two weeks passed, I inquired again, this time to the editor’s direct email, and received a rejection later the same day, with some comments, none of which seemed to me like grounds for rejection (they either required only easy fixes to non-central points or engendered some misunderstandings – or so it seemed to me at least). So I wrote up harsh replies to each point – I just reread them and had some good laughs at the level of my nerve – and sent them to the editor. Even though the paper had been rejected, I said, perhaps he/she could get some use out of my replies, since I had been told he/she was also currently working on the topic.

He/she replied that the paper had indeed been rejected, but as editor he/she was leaving open the possibility of rescinding the decision, and also noted he/she was the author of the comments. He/she said that he/she was indeed working on the topic, sent me his/her paper in case I would be willing to offer comments, and mentioned that we actually say similar things about one aspect of the topic.

We had a back and forth about my paper for a couple weeks, until I sent him/her about 2000 words of comments on his/her draft. Then the correspondence ended – he/she never replied to my last set of responses, to his/her responses, to my replies, to… So I just dropped it. After a couple more bad experiences with refs working in that subfield, I dropped the project and swore off the subfield.

Some differences between my case and Jonathan’s: 1) I was still in graduate school at a 20-30 Gourmet ranked program at the time, not a minted PhD from a top program. 2) I inadvertently rushed the editor-reviewer on his/her decision, which may have had an effect on things. 3) Not realizing the editor was the reviewer, I sent snarky replies as my first step in the “take back the rejection” process (really, not even realizing that’s what process we were in!). 4) The editor was just starting his/her work on the topic, as opposed to having already been well published in it. (In fact, it took a few years for his/her paper on the topic to appear in print, and I just looked at the two versions, and it looks like quite of bit of vetting of the ideas occurred in the meantime.) 5) My article presented some insights that the editor was also trying to get in print for himself/herself. And 6) the whole process played some role in my refocusing on a different area of research.

Glad we’re finally getting serious about these issues.

Out of work young PhD
Out of work young PhD
9 years ago

I appreciate Ichikawa’s anecdote. Recently, I submitted a paper to Phil Studies and settled in for the expected months-long wait (unless the paper was desk-rejected as an earlier paper of mine had been). I watched for months the status of the paper remain at editor invited, thinking that this was the journal’s parlance for referees assigned. The status never changed until I received an email notifying me that the paper had been rejected with no associated referee comments. Now, of course, it’s possible that referees had been assigned and the paper had been reviewed and rejected without comments, but I am suspicious that the paper never even got a read, given Ichikawa’s experience. I don’t have the advantage of knowing the editors or working near them. Therefore, a related problem with practices such as this is not only that the process is unfair, but that submitters don’t even know whether their paper got a fair shot or not. I genuinely would have valued a comment in the email that no referees had actually been assigned. At least then, I would know that the rejection had no bearing on the quality of the paper.

Anonymous Post Doc
Anonymous Post Doc
9 years ago

Ok, I am also now pretty amazed by Grad @ the time’s story (35 above).
I have NEVER heard of an editor rejecting a paper but then when the author sends a bunch of outraged replies to the referees comments, changing his/her mind and allowing that maybe the paper will be accepted after all! Has that sort of thing happened to anyone else?! Does everyone else send replies back in response to the referee’s comments when they get rejected, hoping perhaps the decision will be reversed? I have, on a number of occasions, received referee comments that I thought were dumb and unfair and totally missed my point etc. But it never occurred to me that I should start emailing my grievances to the journal’s editor in case I might convince them that the referee got it wrong.
In all seriousness – do other people do that? Should I start?

A Political Philosopher
A Political Philosopher
9 years ago

Philosophy and Public Affairs seems to me to have a real problem with bias. Because I suspected that it had such a problem, I went through back issues last year, looking at volumes 37, 38, 39, 40 and 41, in which there are 57 papers. Of those 57 papers, 17 had at least one author who received their doctorate from the same institution as the current Editor, Oxford. Another 13 were written by authors whose doctorates were from Harvard, where a number of editorial staff work or studied. The next most common place for authors to have received their doctorate is Princeton, where the journal is based. Of the ten papers in those five volumes published by authors who were last year employed by institutions in the western United States, four are by members of the editorial staff, three are by former visitors at Princeton, where the journal is based, and two are by people who were supervised by members of editorial staff. Only one of the ten has an author with no obvious links with editorial staff. It is impossible to be sure, simply on the basis of that sort of data, that there is anything untoward about its editorial policy, let alone anything deliberately so. I suggest though that it places the burden of proof on those who claim that it assesses all submissions in the same way.

Dr. No
Dr. No
9 years ago

“But it never occurred to me that I should start emailing my grievances to the journal’s editor in case I might convince them that the referee got it wrong. In all seriousness – do other people do that? Should I start?”

Sure, why not? I’ve had mixed luck with this. It helps to do it in the right way, though. Use sparingly and write respectfully. In my experience, you’re not going to get anywhere with this if the issue is a substantive one. I suspect that if your objection to the report is that the referee’s objections were poor, you’re out of luck. Editors don’t usually have the time or inclination to get involved in that. If, however, the referee’s reasons for rejecting are poor and you can demonstrate this without having to get the editor involved in some substantive philosophical issue, a respectful email just might turn things around. On more than one occasion, I’ve turned rejections into R&Rs when I can show that the referee didn’t accurately describe my view or just overlooked some crucial section of the paper. (Most recently, I had a referee say that while my objections clearly ruled out X, there was another view, Y, that survived these objections. I pointed to a passage in the first part of the paper where I clearly stated that said that my aim was to show that X was mistaken and that Y was a better view. The editors agreed that the referee’s complaint simply amounted to the claim that my paper accomplished precisely what I claimed it would so they sent it to a different referee.)

Lewis Powell
Lewis Powell
9 years ago

I think that the appropriateness of the practice you describe depends on a few things. I think that non-blinded desk rejections are a worrisome practice, though I imagine they are somewhat common. Desk rejections involve a quick judgment of the worth of a paper, and typically are not the result of a careful reading of the paper. If the person making that decision can see the name and affiliation of the author, it is hard to imagine there not being large biases in favor of better known authors or more prestigious institutions. I just can’t imagine someone who knows philosopher X and thinks X’s work is generally good, really avoiding being influenced by that in deciding whether to desk reject. This puts more junior authors and less well-known authors at a massive disadvantage for even getting past the initial gate-keeping stage. If the editor is blind with respect to authorship and affiliation, then it seems like a reasonable practice, however, even in that case, the editor should have some conscious policy in place to help them avoid some of Clayton’s worries. If 80% of submissions in a particular topic have to get through the approval of one specific person before even being refereed, it really matters whether that person is disposed to think poorly of papers about particular views or debates. It seems much better for that editor’s views on the merits of the view or debate to come into play later, after other expert advice has been given, so that they have a check against their own (potentially unjust) inclinations.

Douglas W. Portmore
9 years ago

My desk-rejections are blind. A managerial editor sends me the anonymized submission and then I decide whether to send it out for review or give it a desk-rejection.

S
S
9 years ago

I think that the issue raised in 38 above about PaPA is slightly different from the issues raised in this thread. The patterns identified (prevalence of Oxford and Harvard students, etc) are consistent with the editors not being aware of the authors’ identities, not engaging in the strangely personal feedback given by Cohen etc, not being influenced by famous advisers, etc. Instead, the patterns probably point to that journal functioning as the key publication venue for a particular brand of political philosophy, which tends to be strongest at Oxford, Harvard, Princeton etc. None of this is to say that the publication patterns aren’t potentially problematic, because to the extent that publication in PaPA really boosts employment chances in political philosophy they might lead to a narrow set of interests dominating the field, but it is to say that picking on PaPA in this context seems unfair and unwarranted.

another one
another one
9 years ago

Let me get this straight: 57 papers, 30 of which from people with doctorates from either Harvard or Oxford? Seriously, people?

Why do we still consider a publication in PPA a career-making achievement? This data ought to be brought to the attention of the APA, the journal’s publisher, or whoever has the power to do anything about this. Which is probably nobody, and so the philosophical upper crust continues to do as it pleases, and we all have to play along with the pretend meritocracy, or else.

Derek Bowman
Derek Bowman
9 years ago

I think it’s important to separate out two different sets of issues:

1. Fairness in access to journal venues, understood as key factors in employment, promotion, and professional reputation.
2. The effect of bias and possible bias on the promotion and dissemination of good scholarship.

Having separated them out, I think we have to acknowledge that one of these goals may have to be traded off against the other. For example, while there are some epistemic costs to a system that makes it more likely that editors and their close associates will get papers published in it, it’s not clear that these would outweigh the costs of either a) not allowing such editors (and their close associates) to publish in journals that are the best fit for their work or b) requiring such editors (and their close associates) to keep their works in progress secret to preserve the blindness of the review process. But from the standpoint of fairness in access to professionally important goods, neither a) or nor b) are likely to carry much weight (those in editorial positions at prestigious journals already having sufficient access to such goods).

another one
another one
9 years ago

Surely the *type* of political philosophy done at Harvard, Oxford and Princeton is also done in the vast majority of other departments. If not those places wouldn’t be leading the discipline. Oh, you say, but the best journal carries papers by the best people, who got their PhD from the best departments. We know they’re the best people because they publish in the best journal. Or was it because of where they got their PhD?

Anonymous
Anonymous
9 years ago

I truly hope this is recognized for the scandal it is. It threatens the basic integrity of the profession. 30 of 57 articles from people with doctorates from THREE programs directly related to PPA’s editorial board? This confirms what many of us have long known. The system is rigged, and something must be done about it.

another one
another one
9 years ago

Two programs, actually: Harvard and Oxford. If we add Princeton the number raises.

Martin Shuster
9 years ago

Here’s a thought: maybe the problem is not exclusively journal *policies* (although, as the multitude of troubling comments here demonstrate that *too* is a problem), but also the very fact that we continue to *rank* journals. Especially after reading this thread, I strongly suspect, that if someone were so inclined, they could show how journal rankings are just as ephemeral and just as problematic as overall philosophy rankings (see Mitchell Aboulafia’s excellent work in this regard).

Professional philosophy seems to have a perverse addiction to rankings — maybe it is time to seek help.

Rob
Rob
9 years ago

In regards to “well connected” professors helping their graduate students to get published in top journals: this would presumably help the student, after completion, to walk into an academic job; and would in turn help departments who publish their previous students employment records.

on and on and on and on anon
on and on and on and on anon
9 years ago

The privileged will also be invited to give talks at other departments and conferences, thereby making many professional connections, and to publish their work in edited volumes. And so the Matthew Effect will make all these decisions appear obviously correct in retrospect.

Eric Schliesser
Eric Schliesser
9 years ago

Thank you for doing this work.

Komarine
Komarine
9 years ago

This issue comes up time and again (I seem to remember a thread about this at, I think, Leiter a very long time ago). Has nothing changed? Is there more we can do to change things?

Jessica Wilson
9 years ago

This comment is right-on. As soon as one has any degree of prominence, or has any substantive research project, there is effectively no way to de-anonymize one’s papers. Even if the journal editor isn’t familiar with the area and so doesn’t immediately identify the author, most referees to whom the paper will be sent will be able to do so, especially if the author has been workshopping the paper at talks and conferences, as many of us do. As such, attempts by journals to enforce anonymized refereeing are bound to fail, at least (I conservatively estimate) half the time.

Sacco
Sacco
9 years ago

Indeed, many of these issues were discussed 10 years (!) ago regarding Philosophical Perspectives: http://tar.weatherson.org/2004/12/07/philosophical-perspectives/

Matt
9 years ago

That old thread from Weatherson’s blog is well worth looking at again, not least to see more examples of the shockingly low levels of self-awareness displayed by many good philosophers in it. (Hurka was making similar excellent points in that thread as well, all of which are worth looking at again.)

Kate Norlock
9 years ago

I have long wanted clarification on this: “we” “rank” journals? As a grad student pre-Internet, I was not aware of this, and my professors did not make me aware of this. However, by the time I was approaching tenure, I gathered that everyone else had gotten this magical memo. Who’s the we, and what’s the rank, and before y’all answer those first two — I know I can find several blogs that have measured various aspects of impact and rep and so on, but where did the idea come from that there is a we, a rank, and a fixed set of journals? Please advise. (In fact, I’d love a new post just on this topic, Justin!)

ER
ER
9 years ago

If “we” rank job candidates in references, and rank departments, and the media rank universities, and university management crack their whips according to those rankings, and so on, why not rank journals as well? I leave the choice between modus ponens and modus tollens to the reader.

The more interesting question here is whether we should make those rankings more sensitive to the views of experts in the relevant academic fields, or whether we should boycott them to diminish their legitimacy — the weighted lottery of US News vs the rigged lottery of the in-crowd. Given the sorts of problems highlighted here, if “we” go for the former strategy “we” have to be careful about how “we” pick “our” experts.

Basher
Basher
9 years ago

Referees are overworked, so we need a healthy amount of desk rejections. But if that’s the case it is crucial that those be anonymous. The most egregious case of what happens with abundant non-anonymous desk rejections is the Journal of Political Philosophy, which desk-rejects maybe 80-90% of submissions extremely quickly (or at least an unusually large proportion of submissions). As a result the journal contains a large amount of work by people with links to the editorial team, much like PPA, though probably not as bad. I’m not saying this is a deliberate outcome. It may well simply be an effect of the bias that the journal’s policy avoidably allows into the process.

So either desk rejections are limited to obviously awful papers, or authors’ identities need to be concealed from the editors. Given the scarcity of referees, the latter policy seems preferable. The silver lining is that a lot of the new online journals seem to adhere to this policy, and some established ones are moving towards it as well.

Anonymaus
9 years ago

In response to Justin’s UPDATE 2, here is a story that shows problematic journal practices can sometimes be more subtle. Specifically, editors can exercise their power by choosing referees who are likely to be sympathetic, or antagonistic. For one of my journal submissions, I later learned, the manuscript was sent to two referees. One is someone I had co-authored with (but, to be charitable, the paper was just in the process of coming out). The other is someone who I am known to work with on a postdoc. Both referees disclosed their potential conflict of the interest but was told to go ahead anyway. The paper received an R&R and then eventually acceptance. It wasn’t a free pass, but it is easy to imagine that the outcome could have been different with a different set of referees.

Separately, in response to Kate Norlock, a ranking doesn’t need to be the result of a collective. I would be astonished if any person can honestly say that they don’t have some sort of ranking at heart, even just for thinking about their own submission strategy. Even if there were no blogs or websites that attempt formal rankings, there will still be some kind of emergent fuzzy ranking that result from individuals’ own rankings, with perhaps the rankings of people holding more power in various ways holding more sway.

Grad @ the time
Grad @ the time
9 years ago

I’m still convinced that it’s better to stick to discussing the negative repercussions that can come with editorial practices that stray from a journal’s stated policy, and to brainstorming solutions, than to name names like Justin suggested. Notice how quickly the thread devolved into PPA bashing. I feel our goal should be to raise awareness of the many problems, so that other editors who read this aren’t tempted to stray from stated policy in the future, and to devise an avenue for correcting the problems if they do. We’ve heard a lot on the problems side. How about some thoughts on the correction side? Is outing journals/editors the only plan we can come up with?

Lion Rampant
Lion Rampant
9 years ago

One problem with recounting individual problems with journals is that, even if we tell our stories anonymously, editors are likely to recognize them, and us.

So I’m just going to say this: I’ve had a very bad experience with the current (newish) editors of Philosophy, Politics & Economics.

Junior TT Anon
Junior TT Anon
9 years ago

I work in the history of philosophy and have had some frustrations in editorial practices in some of those journals (both general history and more specialized journals), particularly as it surrounds a non-anonymized submission process that leads to a lot of room for editorial bias.

Here’s one situation I’ve recently experienced. I submitted an article to a journal by emailing a copy of the article to the editor (as per the author instructions on the website). The article was prepared for anonymous review, but I emailed the editor directly, so there was no anonymity at that level. The paper received positive comments from both readers with one recommending acceptance and the other recommending an R&R with a few fairly minor revisions. The paper was R&R’d. I made some changes to the paper in response to the comments received and then resubmitted the paper. All evidence from there suggested that the reader who recommended the R&R had approved the paper (for example, the editor/editor’s assistant had made some changes to conform the article to the journal’s style, etc). However, the editor later emailed me to let me know that they were going to send it to one more reader before making a decision. That reader recommended rejection, offering several comments, all but one of which was quite uncharitable to the paper and could have been easily addressed. (For what it’s worth, the one good suggestion/worry was in no way a devastating criticism of the paper. It recommended that I spend more time on a particular point.) And, of course, the paper was then outright rejected.

Publishing is, as we all know, largely a numbers game. If you give a paper to enough people, someone will say ‘no’ to it. And my sense here (which is coming from a fairly biased perspective, I grant you) is that the editor was looking for someone to say ‘no’ to the paper. And so they sent it out until they got the ‘no’ they were looking for.

I’ve talked with some friends and mentors about this and they were taken aback by the series of events. But I don’t know if we’re just in a new world where ever-increasing numbers of articles mean that this will become far more commonplace. I don’t know if what happened was problematic or just something that, while frustrating, is nonetheless rather normal. I do know that I’d feel happier (though still aggrieved) if the process were anonymized, because then at least I could be more confident that this was a decision based on the paper and not on my name, gender, or status within the profession.

I have a deep feeling of helplessness about all of this. I don’t want to ‘out’ this editor in fear that this will color their treatment of any future manuscript that I submit to them. The submission process at this journal is not blind and it is a good journal that I would love to publish in. I am a junior faculty with my job on the line and a tenure clock that is ticking quite loudly. What can I do in this situation? I’ve just chalked this one up to experience and submitted the paper elsewhere. But it has left a very bad taste in my mouth.

(One further thing that frustrates me about this is that it seems like the initial reviewers’ time was not well respected insofar as the editor seemed to disregard their recommendations. I put quite a bit of time into reading and writing up my reports when I’m asked to review a paper with the expectation that they’re taken seriously by the editor. I’d be a bit put out were I to discover that that time and effort was trumped by a last-minute final reviewer.)

David Hunter
David Hunter
9 years ago

Tom–you’ll be happy to hear the Canadian Journal of Philosophy still has a policy of not publishing work submitted by its editors.

We make an exception for special issues (and for symposia). Not every special issue contains work by its editor. And the review process differs from one special issue to another. Most have been by invitation, though a few recent ones have involved calls for papers. None has yet involved double blind review. But I will add discussion of this to the agenda for our board meeting.

AnonGradStudent asks a good question: why do journals publish special issues? Why not just stick with regular issues? I think the basic answer is that special issues can provide interesting thematically related content for our readers. But it should be made clear that the papers are not put through the normal review process. I will make sure this is clearly and prominently stated on the CJP’s websites.

Since March 2014 we have been using an online submission management system, Editorial Manager. There are many advantages to it, both for us and for our authors. But somewhat surprisingly, it has been extremely difficult to implement our review policy in it. We still use work-arounds to ensure that desk rejects and that our final board balloting process are blind. I’d be curious to hear how other journals find online submission management systems.

junior moral philosopher
junior moral philosopher
9 years ago

On more than a few occasions where I’ve gotten a rejection from Ethics fairly quickly (desk rejections?), someone from the the city where the journal’s editor is based has visited my personal website (where drafts of my work are posted) at least a couple days before. Is this the editor? I can’t know for certain, but it just seems too coincidental. It’s made me wonder whether Ethics really is holding itself up to the ideal of anonymous triple blind review, at least at the the desk-rejection stage. In the internet age, googling to find out the authors of papers just seems too tempting for editors and reviewers…

Out of work young PhD
Out of work young PhD
9 years ago

Without naming specific journals (for prudential reasons), I will say that I have had experiences along the lines of both Junior TT Anon and Junior Moral Philosopher.

Re Junior TT Anon’s experience: I submitted a paper to one journal, whose estimated review time was three months. After four months, the journal emailed me to say that they had decided to send my paper to additional referees. After another six weeks, I received an R&R with four sets of comments. The first two were completely in favor of acceptance, the third totally missed the point of the paper, and the fourth had a substantive objection (which I thought was something that should have appeared in a response paper). I suspect that the editors had been seeking reviewers that wanted to reject the paper, though I guess it’s possible that they were seeking reviewers who would be more charitable to the paper. Either way, the decision was an R&R, but the journal’s policy was to run any R&R papers through the whole review process anew, which made it likely that I would have to wait another six months, a time frame that wouldn’t work for me, as I am and was on the job market.

Re Junior Moral Philosopher’s experience: I submitted a paper to a journal, whose estimated review time was three months. Approaching the end of that time, I got a hit on my website from the country in which the journal is located. The title of my paper would have been easily accessible via Google, since I had presented a similar paper at several conferences. A day or two afterward, I received a rejection, with two sets of comments. The first was substantive and helpful, while the second was mean-spirited and mostly ad hominem (with most of the comments focused on my inexperience). I made changes in light of the first reviewer’s comments and the rejected paper was eventually accepted for publication at another journal, with no suggested changes. Of course, it’s possible that the second reviewer at the first journal was not the person who found my website and that the comments were not made based on my being a newly minted Ph.D., but as Junior Moral Philosopher points out, it does seem too coincidental that the website hit had happened so close to the decision to reject. Maybe it was just the editors wanting to know whose paper they were rejecting, but I suspect it was the second reviewer who had found my website and had gained knowledge of identifying information.

AnonWithTenure
AnonWithTenure
9 years ago

A few years ago my paper was rejected at Australasian Journal of Philosophy, with two sets of comments.

Soon after the rejection, I presented the paper at a conference, where philosopher X was in attendance. After my talk, X asked me when it whether it is published. I told him that it recently got rejected at AJP and it was under review elsewhere now. X was surprised, he said he had refereed my paper for AJP and recommended acceptance (and he assured me he had no idea the paper was mine at the time). He said he asked me if it was going to be published because he didn’t want to assume anything about how it went at AJP, or come off like he knew the paper was mine all along. All of this was a surprise to me, too, since the editor at AJP had said that both referees recommended rejection, and included comments.

I had originally noticed that both of the comments, though one in particular, were really quite positive, but didn’t think much of it at the time. I assumed that the referee was just grumpy or had really high standards, and this was just another annoying rejection. So, I looked up the email I got from AJP on my phone right then and there, and asked X if these were his comments. X was furious. The one, very positive set of comments was indeed his, but the last line of the comments, as sent to me by AJP, was *not*, and it read something like “This paper should be published somewhere, but not at this journal.” According to X, who I trust was not lying (why would he?), he had not written this sentence, and had indicated instead that they should publish my paper. So, the editor at AJP, for whatever reason, actually added the rejecting sentence to what the referee wrote.

I’m not sure that X should have told me that he had refereed my paper; perhaps that was inappropriate (though maybe the wrongness is mitigated by the fact that he had already refereed it and assumed it was accepted). For this reason, X didn’t pursue this with the editor much further (he did send an email asking—feigning ignorance I assume—whether my paper had been accepted, and the editor wrote to him that it was rejected because the other referee’s report was very negative (which, having seen the comments, it was not), so the editor knows he was lying to me), and, at the time (before tenure), I didn’t see the point in getting into a big fight with an editor, so I just dropped it. X vowed never to referee for AJP again and that was that.

Balancing act
Balancing act
9 years ago

I’m sure there are some arguments against triple-anonymous review, but I can’t see them trumping the urgency of mitigating the sorts of problems described here. This is something the APA should incorporate in its code of ethics. As for googling papers, well, one solution could be to mask papers’ titles as well, and just use key words.

Anonymous
Anonymous
9 years ago

I have also received (desk?) rejections from more than a few name journals immediately after someone located in the editor’s city visited my webpage.

Jenny Saul
Jenny Saul
9 years ago

I had a paper accepted within 48 hours once. This worried me. So I wrote back saying “gosh, what fast refereeing!”. Turned out editor had liked the paper and accepted it himself. I asked him to please rescind the acceptance and send it off to referees. He said “that’ll take longer”, I said “yes”. It did get accepted, but the comments from the referees made it much better. (This was after I was already established, so I had the luxury of turning down the acceptance.)

Jenny Saul
Jenny Saul
9 years ago

I should add: their procedures have improved since. Submissions are anonymous to editor now.

Roy T Cook
Roy T Cook
9 years ago

I have a story that is in some ways (but not all) similar to AnonWithTenure’s.

A number of years back I submitted a paper to a relatively highly ranked journal (usually top 10, always top 20, depending on the ranking one consults). The paper was pretty specialized, and involved two distinct arguments for two distinct (but interrelated) claims. I received revise-and-resubmit, which basically said that the paper would be published if I ditched one of the arguments, and further developed the other (being younger, I did this without complaint).

I only received one referee report. The referee’s objections to the second argument amounted to this: I had not taken X’s argument into account. I was aware of X’s argument (which was unconvincing, but that is beside the point), which at that point (both the point at which I wrote the paper, and the point at which I received the referee report) only existed in an online, unpublished draft which clearly had “Draft: do not cite” marked across the top. It seems exceedingly likely that X was the author of this referee report (but again, that is probably beside the point).

Here is the point: A little while after trimming the paper to the journal’s satisfaction, I was chatting with another philosopher – Y – at a conference. Y is by all accounts the undisputed “leading expert” on the topic in question: Y has written multiple monographs on the subject, and no one else has written any (and Y is also someone that I think immensely highly of!) Hence, the conversation naturally turned to that subject. Still thinking the second point was a good one, I presented it to Y, to see what Y thought. Y immediately recognized me as the author of a paper that Y had refereed a while back, and asked if the argument I had just rehearsed was in print yet. I said “yes and no”, since the part of the paper we were discussing had been cut at the journal’s insistence. Y then informed me that this seemed strange, since Y’s report had pretty much been “This is excellent and important – publish it as it is.”

So basically, the journal received two reports: one negative, and one immensely positive (from someone who could conservatively be called the leading figure in the sub-field), yet only sent me the former report. Perhaps not as evil as some of the other stories discussed above, but definitely rather arbitrary and unprofessional.

Anony
Anony
9 years ago

In the early 90s I sent in a paper to a journal with a fairly good reputation (I thought), I guess it would be considered high mid rank. The paper was not accepted; I forget the details. I don’t think I received referees’ comments. I published the paper with the next journal I sent it to, one that is considered a better journal. Low and behold, a few years later I picked up a copy of the first journal and saw that about a year after I had submitted my paper to the journal there was a paper published in it which had exactly my main argument, only done more cleanly and without the discussion building up to it that I had in my paper. I believed (and still believe) that my paper had been plagiarized because the argument is pretty unusual. I have always wondered whether this had come about because of a junior-ish referee having seen my paper (and then written a version of the argument himself). I guess various other explanation did not seem to me at the time to be as likely, for whatever reason. It’s interesting that they did not; other strangers did see versions of the paper, since I had used it in job applications. I never pursued the matter — was not in good shape at the time to take it on. I guess I could have had better evidence if journals all listed the referees used each year. But they don’t.

Anony
Anony
9 years ago

*Lo and behold (of course).

Anon
Anon
9 years ago

My recent submission to Ethics got rejected after being read by one of the editors. The email came from him personally. Although he could offer no comments (fair enough), he recommended that in the future I explain the relationship between this paper and another related paper he found on my website (a useful advice, thanks). This other paper was not cited in my submission, which means he googled my name and checked out my website before rejecting my paper. And he did not even try to hide this lack of blind review.

Lion Rampant
Lion Rampant
9 years ago

Jenny Saul’s story is pretty shocking (thanks for it). How common is it for well known people to bypass the refereeing process?

Junior faculty
Junior faculty
9 years ago

I’ve recently had an experience similar to some of the ones now being reported, but I’m not convinced there’s any wrongdoing involved (in my case). Maybe the devil’s in the details; I’m posting this story because I hope the contrast with Roy T Cook’s story and Junior TT Anon’s story will help highlight what’s wrong with what happened to them.

As far as I know, everything was anonymous all around, all the way through. I went through 2 R&Rs, receiving in total 6 sets of comments (1 on the submission, 3 on the resubmission, 2 on the reresubmission). One of these sets of comments (on the resubmission) was pretty hostile, but the rest were super useful and made my paper much much better. Both of the final comments recommended acceptance. But the paper was rejected by the editor, a non-specialist, for reasons that I think are pretty bad (of course), and don’t get much past page 1 of my paper.

This is frustrating. The paper was tied up for a long time at a journal it just couldn’t get into, because the editor didn’t like the very idea of what I was aiming to do. The editor overrode the judgment of people better placed to judge the work.

But it’s not wrong. This is a print journal with a backlog. They have to reject some papers, even some good papers, even some “deserving” papers, in whatever sense a paper can deserve to be accepted. I have every reason to think my paper was rejected for its content, and not because of who I am.

The online editorial software this journal (now) uses passed on the referees’ comments unedited. The editor chose to reject the paper & told me why, rather than sending it out a third time looking for a referee who would say no. The editor didn’t choose to take the hostile referee’s verdict on the resubmission over the other verdicts. These are relevant differences from what happened to Roy T Cook & Junior TT Anon.

I guess I should name the journal, since I’m saying relatively good things about them. It was Mind.

Anonymous
Anonymous
9 years ago

In the interests of transparency, it might be good for someone to know the acceptance rates that people get when they submit work to their former supervisors’ journals. I’ve been looking at some numbers and found that some editors’ graduate students publish a surprising percentage of their work in these outlets (in one case, it’s above 20%). You can just look at the philosophy family tree, find connections between philosophers and editors, and check CVs to see what the percentages are. I haven’t done this for many editors, so I don’t have a baseline, but it seems like potentially important data to keep an eye on if we care about things like fairness in refereeing. It’s possible that this is down to selection bias (i.e., people submit their work to journals edited by former supervisors more often than other people do because they think that the journal is important or well run or whatever) or down to the fact that some are just better trained than others, but it’s also a possible indication that things aren’t running as well as they should. (It doesn’t matter whether this is malicious or not, this isn’t about motive. This is about causal influences that lead to an imperfect system that might be corrected with minor changes.) This would require significant further study and it’s boring and thankless work. What could I hope to get at the end of it? A long list of enemies? Someone should be monitoring these kinds of things, but I don’t think there’s any body that checks to see if there are things like surprisingly high rates of acceptance when people submit work to journals run by their former supervisor. That information, so far as I can tell, is available only to the editors of the journals, but maybe it should be shared with the APA or some other body for monitoring. There is some data out there and someone could comb through it or at least monitor it for irregularities.

Anonymous
Anonymous
9 years ago

Here’s a proposed test:

Any journal that has a record of publishing mostly essays written by big shots is probably not using the best editorial practices.

Most people in the political philosophy world know that PAPA is, for all intents and purposes, a crooked journal (since it gives big names and students of big names preferential treatment – see the numbers cited above). It is likely that once we consider all essays published in Ethics and not just the articles, we will find a similar sort of bias (I went through the last two years’ worth of Ethics and found approximately 50% of the published pieces to be written by well-established people and their students, with a surprisingly very small number of essays written by people whose names I didn’t recognise).

If people think this might be a decent test, then perhaps people could look at other journals.

My overly casual observations suggest that Phil Imprint, Nous, and PPR are edited in, at least by the sad standards I’ve just proposed, an exemplary fashion. I’d include Mind in this list but for its objectionably slow editorial processes. Hard to know what to say about Phil Review…

Anonymous
Anonymous
9 years ago

As a graduate student, I had a similar experience to Jonathan’s with Phil Studies but with a negative result. I submitted my paper, emailed to check that it hadn’t slipped through the net (it had been on ‘editor assigned’ status for four months), and received a quick, four-line rejection soon thereafter. Stewart Cohen told me later it was he who had refereed my paper. His rejection clearly indicated that he wasn’t sympathetic to my project, so I never submitted papers from that project (my main project) to Phil Studies because I expected them to meet a similar fate.

Anonymous
Anonymous
9 years ago

Small thing in response to Anon@ 5:08, The nobody test will reveal something, but I don’t think that the test can show (much less suggest) that a journal works in an exemplary fashion. Two of the journals you mention are in the top 5 mentioned above (i.e., J Phil, Phil Rev, Nous, Mind, or PPR). If the test worked to establish that something was indeed exemplary, we should be able to narrow the list above down to three of the five mentioned (i.e., the non-exemplary one would be J Phil, Phil Rev, or Mind). Do these journals do poorly on the proposed test? Not obviously.

There are many ways for a journal to be non-exemplary, so I doubt any single test or small set of tests will pick up on the marks that would give us confidence that a journal is being run in an exemplary way. You’re right, though, that you’re picking up on a good mark to worry about. Too many big-shots getting work into print that isn’t particularly novel, inventive, or ground-breaking isn’t a good sign. Too many friends and former students in a journal isn’t a good sign. I think we’ll do better finding the marks of perversity than we will finding the marks of goodness.

Brian Weatherson
Brian Weatherson
9 years ago

I’m obviously biased, but I don’t think the Philosophical Review publishes primarily big shots. I went through the last three years of issues, and counted 37 articles, of which I think 19 were by people who didn’t have tenure at the time. (I had to guess in a couple of cases, so that might be off by one or two.) Indeed, 9 of 37 were by people who weren’t in tenure-track jobs at the time (3 grad students, and 6 post-docs of one kind or another).

And in that 37, it’s published pieces by people from Innsburck, William and Mary, Delaware, Kansas, Ludwig Maximillian, Manitoba and the Korea Institute for Advanced Study. Five of those 7 are among the 18 tenured I listed above, so that only leaves 13 articles by people with tenure at, roughly, top 40 to 50 programs.

It’s true there were papers by people who are ‘big shots’ as most people would understand the term. But I’m not sure there is a pattern. Here is where those papers (i.e., papers from tenured faculty at top departments) were from (with * marking that it was from someone associate level or rough equivalent): Bristol*, Leeds, LSE, UNC, Pitt, Oxford, Arizona, Cambridge*, Brown/Ohio State, Syracuse, UNC*, Notre Dame*, Sydney/Cambridge.

If there’s a pattern of bias that’s showing up there, it isn’t one of the obvious ones. Maybe there’s a mild pro-English bias? That would be bad, but it would also not be the kind of bias you normally expect in an American journal, and hardly something you could diagnose with that sample size.

(I haven’t gone back to check the graduate schools of the authors. I didn’t recognise any Cornell students in there, but I could have missed one.)

Jay Wallace
Jay Wallace
9 years ago

Just a few comments about the editorial practices of Philosophy and Pubic Affairs, where I have been an Associate Editor for about 10 years. I’m sure I won’t be able to allay all the concerns that people seem to have about the journal, but wanted to share some of my experiences, for whatever they might be worth.

The really distinctive thing about the journal is that nearly all the reviewing is done by the Associate Editors themselves. In at least one way, it is a practice that is more transparent than the ones used at most journals; the names are right there in the front matter of the journal, and if you submit to the journal and the paper survives the initial (anonymized) screen by the Editor, you can be pretty sure that your paper will be read by one or two of the people on the list. The people who serve as Associate Editors are a pretty respectable lot (present company perhaps excepted), and they are committed to the journal, and willing to invest a lot of time reading and commenting on submissions etc. All of the submissions, including those from present or past Associate Editors, are of course anonymized (though, as many on this discussion thread have noted, it often isn’t hard for reviewers to find out the author of a paper they have agreed to review in the era of Google, something I myself never even try to do when I am reading a submission). There are pluses and minuses to all possible procedures for reviewing journal submissions, and I’m sure our procedures are not without their flaws. But one advantage to ours is that papers get reviewed fairly quickly: the Associate Editors agree to review submissions they receive within a month, and papers do get turned around fairly swiftly. Another is that reviews are being done by a fairly distinguished and committed group of readers. (In this era of vastly increased academic production, it is often very difficult for editors at most journals to find competent and fair-minded readers who are willing to referee submissions, with the result that the quality of the comments that people receive on submitted work is extremely variable.)

About submissions to the journal by current or past Associate Editors: I can see why it might make sense for some journals to have a blanket policy of never publishing work by people currently associated with the journal. (Though if this is so important, I’m not sure why exceptions should be made for special issues and the like!) But PAPA publishes in a fairly narrow area of normative ethics and political philosophy, and it also has a fairly large group of Associate and Advisory Editors, including people who are producing some very important work in just these areas. (I believe something similar is true of Ethics.) A policy of banning publications by Associate and Advisory Editors would therefore have a more significant effect on access by the journal to important work than would be the case at a more generalist journal such as the Canadian Journal of Philosophy. And as David Estlund remarked above, it would also strongly discourage people from signing on for multi-year terms as Associate Editors (though most of us, when we agree to do this, end up submitting to the journal only very rarely). As I noted earlier, submissions by people involved in the journal are of course anonymized in the standard ways (which isn’t to say that it would be difficult to find out who the author was, if one really wanted to do that: see above). They are also sometimes rejected.

Regarding length of submissions: the journal has length guidelines, but it is open to submitting longer pieces, and also sometimes multi-part works. (The guidelines say only that the journal “prefers” submissions of under 12,000 words.) Another multi-part epic that the journal published in the not too distant past was by David Velleman, who has no association whatsoever with the journal (and whose paper was submitted anonymously to Associate Editors for review in the normal ways). Of course, the bar for publication of such a work will be a bit higher than for a more conventional paper. But if any of you out there have outstanding papers that cannot be condensed into a more conventional format, you should by all means submit them!

Finally Associate and Advisory Editors at PAPA are constantly on the lookout for good work that might be suitable for publication in the journal. So we sometimes encourage people to submit papers we have heard at conferences and the like, and we might also suggest that students or colleagues submit to the journal. In these cases, Associate Editors recuse themselves from reviewing the papers that might eventually be submitted, for the obvious reasons.

Facts and Data
Facts and Data
9 years ago

I don’t know how we are to assess this test, but I think the author of the above post is playing loosely with the facts. Let’s look at the authors of the last 60 papers published in PAPA. I’ve put an asterisk next to the name if it is a paper by a current editor of PAPA (some of these people might not have been editors when the paper was published, I don’t know):

Niko Kolodny*, Daniel Viehoff, John Weymark, Tommie Shelby, David Estlund, Alexander Guerrero, Michael Kates & Ryan Pevnick, Pamela Hieronymi, Victor Tadros, George Tsai, Allen Buchanan, Anna Stilz, Gina Schouten, Frederick Neuhouser, Serena Olsaretti, Charles Beitz*, Michael Blake, Nicholas Vrousalis, Lea Ypi, Ronald Dworkin, Joseph Heath, Avihay Dorfman & Alon Harel, Simon Caney, Collin O’Neil, James Shaw, David Enoch & Levi Spectre & Talia Fisher, Mathias Frisch, Andrea Sangiovanni, Waheed Hussain, Lucas Stanczyk, Seth Lazar, Jonathan Quong, David Luban, Lucy Allais, Julia Nefsky, David Estlund, Andrei Marmor, Christian List, Shelly Kagan, Thomas Christiano, Dan Moller, Martha Nussbaum, Samuel Bowles, F.M. Kamm*, R. Jay Wallace*, Jeff McMahan*, Joseph Mazor, Arthur Isak Applbaum, Kristi Olson, Alexander Guerrero, Seana Valentine Shiffrin*, Sophia Moreau, Seth Lazar, A. John Simmons*, Niko Kolodny*, Christian List & Mathias Koenig-Archibugi, Selim Berker, F.M. Kamm*, Allen Buchanan, Patrick Emerton & Toby Handfield

I work in moral and political philosophy, but I have never heard of many of these people, or do not think of them as “big names” certainly. I’d say maybe 20 of the 60 seem like “big name” people? Is that too high a ratio? What would we expect to see? The more concerning thing, if anything, is that about half of the “big name” people are editors. That might be something PAPA should revisit, policy-wise.

It is possible, of course, that the rest of the people are all (or often) students of big name people. But given that the editorial board includes faculty from pretty much every top political philosophy program: Princeton, UCLA, Berkeley, Toronto, Oxford (for the main editors) and Stanford, NYU, USC, Harvard, Virginia, Michigan, Rutgers, MIT, Penn, Columbia, etc. (for the rest), it is hard to know what to make of this suggestion. It is hardly as if the journal is dominated with the work of recent PhDs from Princeton, UCLA, Berkeley, Toronto, and Oxford. Wouldn’t we expect that some of the work by relatively unknown people would be by people from these 15 (or whatever) programs? I sincerely doubt things would be any different if we ran the numbers for Phil Imprint, Nous, PPR, Phil Review, Mind, etc.

I do think that there are some significant gaps/problems. Why are so few of the editors libertarian, or conservative, or non-white? But these seem like a different set of questions.

The journal seems to be doing decently well by the “no name” test, at least if that test is reasonably administered. Perhaps the author of the above post is suffering from confirmation bias. At least over the recent history discussed above, PAPA does not have a “record of publishing mostly essays written by big shots.”

Anonymous
Anonymous
9 years ago

Hi Facts and Data –

Actually that list supports the contention that PAPA has a problem. A lot of those people are either very well known or very well-connected.

Niko Kolodny*, Tommie Shelby, David Estlund, Alexander Guerrero, Pamela Hieronymi, Victor Tadros, Allen Buchanan, Anna Stilz, Frederick Neuhouser, Serena Olsaretti, Charles Beitz*, Michael Blake, Ronald Dworkin, Joseph Heath, Simon Caney, David Enoch, Andrea Sangiovanni, Waheed Hussain, Jonathan Quong, David Luban, Lucy Allais, Julia Nefsky, David Estlund, Andrei Marmor, Christian List, Shelly Kagan, Thomas Christiano, Dan Moller, Martha Nussbaum, F.M. Kamm*, R. Jay Wallace*, Jeff McMahan*, Arthur Isak Applbaum, Seana Valentine Shiffrin*, A. John Simmons*, Niko Kolodny*, Christian List, Selim Berker, F.M. Kamm*, and Allen Buchanan.

Every one of those people is either very well established or very well connected in the sense that they are either Harvard PhDs or have or had jobs at either Harvard, Princeton, or Oxford. (I’ve also left a few off just to be conservative).

Much more significantly, 9 of those articles are by associate editors?! Wow. So, 15% of PAPA articles are by associate editors. That’s amazing! What are the numbers at Political Theory or APSR or Ethics or Phil Review? I’d be really surprised if 15% of their articles were by editors.

Now, I think that a lot of the articles published in PAPA are actually pretty good! But, I am surprised to discover that there is such a strong correlation between a very small number of institutions and publishable political philosophy. When I read Political Theory and APSR, e.g., I find a lot of really strong essays that are produced by people working at all sorts of universities and colleges.

A Political Philosopher
A Political Philosopher
9 years ago

Comment 84 confuses and obscures the point I made about the pattern I found in authors who had published at P&PA over a five year period – not, incidentally, quite the same five year period that they look at. The point I made was about connections between authors and editorial staff. If over a quarter of papers over a five year have an author who did their doctorate at the same institution as the current editor, any issue there might be is not about “big shots” but about personal and professional connections to the editorial staff. I am happy to provide Dr Weinberg with a copy of my workings for him to check if he would like to do so.

I appreciate Professor Wallace’s generous intervention. As I said, the patterns I found cannot by themselves show misconduct, let alone deliberate misconduct, and so I am genuinely interested in what members of editorial staff at P&PA think about them. Do they make Professor Wallace, for example, wonder about the appropriateness of the journal’s editorial procedures?

I would also be interested to see how the patterns compare to those at similar journals. It would be worth conducting a similar survey, it strikes me, into any connections between authors and editorial staff at say, Ethics, some other top-ranked sub-disciplinary journal, and a top-ranked generalist journal over a similar time frame. Apart from their own intrinsic interest, this would provide a baseline from which to assess the patterns I found at P&PA. Unfortunately, I don’t currently have the time to do so.

Anonymous
Anonymous
9 years ago

Just to follow up on Brian Weatherson’s post. I think some people get worried about philosophers using their connections to get work into good journals and after skimming most of the Cornell grads from 2008 on, I really couldn’t find any that had published in Phil Review. Their grads publish well (by my lights, at least) but they aren’t publishing in the journals that their advisors edit. There’s often a lot of grousing about this journal in online discussions, but it certainly doesn’t look like they are giving their grads any sort of favorable treatment.

Facts and Data
Facts and Data
9 years ago

You were inadequately “conservative” in gathering your data. None of these people on your list are Harvard PhDs or have had jobs at Harvard, Princeton, or Oxford: David Enoch, David Estlund, Alexander Guerrero, Julia Nefsky. And perhaps there are others.

But maybe that’s quibbling. I think it’s a question worth asking, the question: what % of a journal’s authors should be “big names”? What kind of institutional diversity should we want?

I think it’s not surprising or particularly troubling that Harvard, Princeton, and Oxford are well represented, in part because many of the above authors got their PhDs 15+ years ago, when those three schools were particularly dominant in both philosophy and political theory. But it’s a good conversation to have, I think, although I guess I’d prefer to focus on the editorial practices or institutions that we might want to reform. The list gets a lot shorter, for example, if current editors don’t publish in PAPA. Perhaps it would be better to send more things out to a broader pool of referees? Although that might (as R. Jay Wallace suggests) harm both the speed and quality of refereeing. Perhaps it would be good to increase the institutional/philosophical diversity of the editorial board?

It’s also worth asking whether it’s OK for different journals to have different “styles” and correspondingly different networks of influence and power. Political Theory and APSR publish much more work that is from a historical or contintental or Straussian perspective, for example, and much less work that is from an analytical perspective, relative to PAPA. But is that a problem? These are all very prominent journals, and there’s no obvious reason that we should want all three of them to look very similar in terms of what they publish, and the social networks that we can trace out in that regard. If I write a paper on Arendt or Athens, I’m sending it first to APSR or PT, not PAPA. If I write a paper on egalitarianism, Rawls, just war theory, democratic theory, I’m probably sending it to PAPA or APSR, not PT.

Matt
9 years ago

Anonymous at 12:23 said, of a long list of people, “Every one of those people is either very well established or very well connected in the sense that they are either Harvard PhDs or have or had jobs at either Harvard, Princeton, or Oxford.”

It’s not true of many people on the list that they “are either Harvard PhDs or have or had jobs at Harvard, Princeton, or Oxford.” It’s not true of Guerrero, Buchanan, Simmons, Heath, and Christinano, at least (maybe more.) I think there are real reasons to worry about journal practices, and I think that editors should be much more concerned about whether they are being objective than they seem to be from this thread (and other ones.) Introspection really won’t suffice here, and there is a good prima facie case for less than great behavior. But there’s no use is making claims that are not true.

Tom Hurka
Tom Hurka
9 years ago

Since I initiated some of the above discussion, let me respond to some of Dave Estlund’s and Jay Wallace’s remarks.

I was concerned only with submissions by Associate Editors, not Advisory Editors or members of the Editorial Board, because, unless I’m mistaken, only the former are actively involved in evaluating submissions. There are seven of them, and I just don’t see how it’s an enormous sacrifice for a journal to accept that, in any year, there are seven people whose articles it won’t publish. Nor do I see it as a tremendous sacrifice for those people that they forgo publishing in one journal of the dozens available to them for the period they edit it. These just seem like pretty minor concerns.

My concern with Associate Editor submissions is very much linked to the fact, distinctive as Jay says of PAPA, that its evaluations are mostly made by its editors rather than by outside referees. I suspect that in most cases of an Associate Editor submission the editor evaluating it will know who the author is, not by looking anything up but just by the subject matter, style of argument, and so on. And to me there’s a special danger when an author whose identity you know is also an editorial colleague you’ve assessed others’ submissions with in the past and will continue to work with in the future. May it not be harder in this circumstance than in others to say “Sorry, your paper isn’t up to our standard,” if that’s justified? I’m pretty sure it would be harder for me.

There’s also been a lot of discussion lately about implicit bias, and just as there can be implicit bias about race and gender so there can be implicit bias in favour of friends and colleagues. Imagine getting the same paper in two circumstances: one where you know it’s by a friend and colleague and one where you don’t know the author’s identity. Isn’t there a possibility — not a certainty but a serious possibility — that you’ll evaluate it more favourably in the first case? Again I suspect I would. Favouritism doesn’t have to be explicit or conscious; it can operate indirectly. And if you don’t take steps to block it there’s an ever-present danger that it will.

As the comments in the thread above show, PAPA isn’t universally regarded as fair and impartial in its editorial practices. I would have thought its editors would be concerned with its reputation and want to improve it. Why not when the costs of doing so are, to my mind at least, so small?

Blergh!
Blergh!
9 years ago

Matt @1:57

What about well-established? Christiano, Buchanan, Simmons….? If you’re gonna be nit-picky then do it well.

Matt
Reply to  Blergh!
9 years ago

Blergh! – I don’t doubt that they are “well established”, but the claim was that they were “well established” _in a particular way_. That wasn’t true. That’s not nit-picking.

AA
AA
9 years ago

I’m seeing a lot of rationalizations by people associated with PAPA here. Seriously, what are the odds that in a continuous sequence of 57 papers, 30 would be authored by scholars with doctorates from just two institutions? Explain that before you bring up other points that just muddle the waters.

Andrew Moon
9 years ago

I was disappointed to read about the lack of anonymity that occurs in some journals. For all the (perhaps justified) crap that Mind gets for being slow, I highly doubt that I would have published papers there if they had known that I was a grad. student at U. of Missouri. It’s difficult enough for students coming out of lower-ranked schools, and anonymous refereeing is really our main way to get our foots in the door. Taking that away feels like a kick in the face.

Blank
Blank
9 years ago

Just today prominent philosopher X who edits journal Y posted a private cfp (only available to X’s facebook friends) for papers on the same topic as a paper X has forthcoming in journal Y, along with a suggestion that X would “try to make sure they got sent out for review quickly”, because X thinks it would be cool to have multiple papers on the same topic in the issue that X’s paper is coming out in. I wonder if the journal will make clear that this is how these contributions were solicited.

Jonathan D. Jacobs
9 years ago

I don’t want to detract from the main thread of stories about editorial practice, but the issue of marking invited papers as invited came up in regards to a forthcoming special issue of Res Philosophica (and special issues, more generally). So I put up a post at Letters from the Editors for discussion of that issue. I welcome your feedback:

Should Invited Papers be Marked as Such?

Anonymous
Anonymous
9 years ago

In response to comments about Mind’s review process, particularly concerning how long it takes to review a manuscript at this journal, I would like to point out Mind’s recent statement:

http://www.oxfordjournals.org/our_journals/mind/review_procedure.html

According to this statement, the average time for a first decision is less than three months. This seems a reasonable period of time, at least to me, in comparison to other journals which receive 550+ manuscripts a year. It should also be kept in mind that this journal has never had a moratorium on submissions, in contrast to other prominent journals which regularly do so.

grad
grad
9 years ago

Though not ideal, a moratorium is better than a long wait-time. Because with a moratorium, one can plan, e.g. send the article to another journal in the meantime. But I can’t do that while it’s sitting at a journal for 12 months.

Jay Wallace
Jay Wallace
9 years ago

Just a brief follow-up regarding editorial practices at PAPA (partly in response to Tom’s helpful clarification in comment #90 above). I should have mentioned in my original remarks that anonymous submissions by Associate Editors are, as a matter of course, subjected to a different procedure than other submissions; in particular, they are always sent out for review by at least one person who is not among the Associate Editors of the journal. This applies to submissions by both current and past Associate Editors, and it is meant to address the dangers of implicit bias and favoritism to which Tom rightly calls attention. (As I noted earlier, such submissions are sometimes rejected.)

I agree with Tom that a ban on considering submissions by Associate Editors, during the period when they are actively serving in this capacity, would be less draconian than the policy he originally seemed to be recommending (namely, the policy of never publishing work by any of the “main editors” of the journal). But I think I just differ from him about the degree to which operationalizing even this more targeted policy would involve sacrifices, both for the individual editors and for the journal. It isn’t merely that there are seven people in any given year whose work wouldn’t be published by the journal. Each of those people is an active and prominent scholar in mid-career, working centrally in the areas that the journal targets. When we are thinking about inviting them to join the editorial team, and trying to persuade them to sign on to a very heavy commitment that will last for many years (up to a decade or even longer in some cases), it has seemed to us that it would involve significant sacrifices for both sides to insist that those who agree must never submit to the journal while they are serving as Associate Editors. This is for reasons that do not necessarily generalize to other journals, having to do with the small number of prominent venues that publish work in normative moral and political philosophy, the comparatively small cohort of people doing cutting-edge work in these areas, and the extensive commitment that is undertaken by Associate Editors at PAPA (in virtue of its distinctive practice of in-house anonymous review).

To say that there would be sacrifices to the more restrictive policy Tom proposes, however, is not to say that it would be unreasonable to impose them (both on the journal and on those who make the huge commitment that is involved in agreeing to serve as Associate Editors). All members of the editorial team at PAPA are of course “concerned with its reputation and want to improve it”, and we may well revisit this and other aspects of our editorial practices in our ongoing effort to ensure their consistency and integrity. But there are reasons for allowing submissions by current Associate Editors that should not simply be dismissed out of hand, and special procedures are already in place to address the legitimate concerns about favoritism to which Tom and others have given expression.

Anonymous
Anonymous
9 years ago

I wanted to raise two issues for consideration.

1. On Phil Studies. Like #16, I found the quick statement encouraging. Initially, at least. Now I have lots of questions that I think won’t be answered as everyone seems ready to move on because of the statement mentioned above. How extensive was the practice of subverting blind review? Who benefitted from this? Has anyone asked about how extensive this was and how many people benefitted from this? Are there others at Arche or Arizona who had similar stories? Most of us find #21’s story shocking. My initial impression was that the Phil Studies story wasn’t anything like this one, but now I don’t know why we’d think that. We know nothing about how rigorous the review process was for people who weren’t put through blind review and had their work accepted. Will someone be looking into this or are we simply satisfied that the editor has stated publicly that he will now adhere to the guidelines that the journal has claimed were in place for years? (Also, will Phil Studies mark the papers that were accepted without proper blind review as such? Perhaps a mark to indicate that they were neither invited nor properly vetted? Might I recommend using the question mark as the mark?)

2. I’m worried about other journals. I haven’t looked to see how many Michigan grads are publishing in Phil Imprint, but it doesn’t look to me like Cornell students benefit from their association with Phil Review or that Columbia students are filling the pages of J Phil. It is interesting to see how often students of the editor of PPR, Nous, and Phil Issues publish in these places. I started to look into this because of some rumors that I’ve heard over the years. (Feel free to dismiss that as a mere rumor of rumors, but am I the only one who has heard such things?) Do we expect to find people who publish above 20% of their journal articles in these journals? In one case, it’s 40% of publications listed on a website and some of the listed publications were invited.) I don’t know many people like that. (Interestingly, many of the people who do very well in these journals also publish a surprisingly high percentage of this work in these journals or things edited by former classmates. In one case, 38% of total listed publications.) There are explanations of this that would make it all look totally innocent. If it turns out that these numbers are standard for people at similar stages of their career in similar research positions, maybe nothing is amiss. If these people just submit much more work to these venues than other people do, then perhaps this is to be expected. (It doesn’t account for what seem to be surprisingly high rates of invited pieces to these journals.) When the numbers look strange, though, and we hear tales like #21’s, shouldn’t we start to get curious about how all the top journals run? Can we compare acceptance rates for people with connections to editors (e.g., colleagues and former students) to people who don’t have such connections? If it turns out that former students of editors have abnormally high rates of acceptance, that’s a sign that something has gone wrong.

Maybe we should have an audit and require journals to share their records with some independent body (e.g., the APA). Going forward, maybe journals should be required to keep and submit documentation to verify that accepted papers have been through a proper review process. And maybe we should move to a system where acceptance is done more by committee so that single actors don’t have the power to decide what gets in and what doesn’t.

Bharath
Bharath
9 years ago

Some key words from Jay Wallace’s comment at 99: the Associate editors are “prominent” scholars, there are a small number of “prominent venues that publish work in normative moral and political philosophy”, “the comparatively small cohort of people doing cutting-edge work in these areas” (this last one is particularly amazing. Really?).

Some info I was able to google about the editor and associate editors at Philosophy and Public Affairs:
Alan Patten: chaired professor at Princeton, DPhil Oxford
Barbara Herman: chaired professor at UCLA, PhD Harvard
Seana Shiffrin: chaired professor at UCLA, DPhil Oxford
Niko Kolodny: tenured professor at Berkeley, PhD Berkeley
R. Jay Wallace: chaired professor at Berkeley, PhD Princeton
Arthur Ripstein: tenured professor at Toronto, PhD Pittsburgh
Stuart White: associate professor at Oxford, PhD Princeton

I think neither of Wallace’s two reasons works for why associate editors can publish at PPA. First, they are prominent thinkers in the field. Granted. But as their positions show, they have no material need to publish, at least not in this one particular journal. They are already at or near the top of the academic ladder. They can publish in lots of other places. Second, being an associate editor is a heavy commitment. No doubt. But doing this without the possibility of publishing in the journal would be a simple instance of giving back to the profession, and making the circle of influence wider. Otherwise it looks as if one is willing to take on the time commitment in order to be part of the in crowd.

One question is: is PPA only trying to understand justice, or does it aim to enact justice and create better institutional structures, starting with academic philosophy itself? If the former, I doubt justice can be understood in such a purely theoretical way. And if the latter, it is puzzling that Patten and Wallace say that PPA’s procedures are fine, as if its structures are already basically just.

Cat
Cat
9 years ago

Maybe this topic has been covered here earlier or on other blogs . . . but am I right to assume that if a paper has been rejected at PPR or at Nous, one ought not to bother submitting it to the other? Isn’t there something wrong with having two major journals with the same editor? Or has anyone had the experience of having a particular paper rejected by one and accepted at the other?

Schnee
Schnee
9 years ago

Re: Phil Review — I know a number of folks at Cornell, and non-tenured faculty and grad students are, in fact, discouraged from submitting to the journal out of concern that it might look inappropriate or otherwise undermine the credibility of the author’s work. Which I think is admirable. (This doesn’t mean, of course, that no Cornell folks have ever published in Phil Review. They obviously have.)

Karen Bennett
Karen Bennett
9 years ago

@103: yes; see my comment above (#19). Anyway, that’s how it’s worked during the 2 or 3 years for which I’ve got detailed knowledge.

Ben
Ben
9 years ago

Cat (#102): I recall reading a comment from the editor (Ernest Sosa) saying that submissions to PPR are handled independently of submissions to Nous, and that papers rejected from one journal are more than welcome to be submitted to the other.

Cat
Cat
9 years ago

Thanks to Ben (#105) for the reply about PPR and Nous. That’s somewhat reassuring. Still, I can’t see how papers can be handled independently at the two journals in all cases. If the editor decides not to send a paper out to referees at PPR, I hardly think it’s worth the effort to send it to Nous, when that journal has the same gatekeeper. And many of us have had the experience of getting a rejection with no comments at one journal, only to have the same paper eventually accepted at a different, good journal. Isn’t the decision whether or not to have a paper refereed made by the editor?

Neil
Neil
9 years ago

Cat, PPR and Nous have associate editors – different ones – that handle submissions. Further, what look like desk rejections at both are probably summary judgments without comments by referees. Both journals ask referees for a very quick judgment without comments or a substantive review.

Ernest Sosa
Ernest Sosa
9 years ago

A Daily Nous post update of 1/20/15 directs the reader to the Nous and PPR websites, each of which contains a statement that the journal has adopted a policy of triple-blind review. Nous and PPR thus operate independently. Also, Nous now has a total of six editors, and PPR seven, each editor with discretion on the papers that they handle. (Incidentally, at least two prominent journals publish more words per year each than do Nous and PPR combined, and this remains true even counting Philosophical Issues as well.)

Cat
Cat
9 years ago

Thanks for all the information and thanks to Neil as well. It’s much appreciated.

Anonymous
Anonymous
9 years ago

I have a completely different question. Is it necessary for journals to insist that authors send their work only to that journal and no others for consideration? This is not how it works when submitting papers to law reviews, by contrast. What would the pros and cons for everyone in the discipline be, if anyone below the rank of tenured professor could send out papers to any journals they chose at the same time? Is this a policy worth considering? What if just a few journals decided to make it their policy? The reason I ask is that I’ve never been fully convinced of the value of this policy and I have heard a large number of people complain that their pre-tenure years were made enormously stressful by the long wait times they experienced at different journals. A few rejections and a few revise and resubmit responses later, half one’s tenure clock may have run out.

Walto
Walto
9 years ago

I don’t really understand what is appropriate and what isn’t at this point. I sent a paper to the Journal of the History of Analytic Philosophy a couple of years ago. It was assigned to two reviewers. One thought it should be published, the other thought it needed significant revisions. I made significant revisions: including, I believe, everything that reviewer asked for. But the editor sent me a letter indicating that he didn’t think he wanted to publish it anyways. I believe the only reason he gave was that he didn’t think it was sufficiently analytical or something equally vague. I guess I’m not sure why editors bother with reviewers at all, blind or not, if they will make the decision in the end anyhow. The guise of “independent referees” simply adds months or even years to the process. In my case, IIRC, the process took over a year and a half. (I took no more than a couple of months making the requested revisions.)

M
M
9 years ago

Law reviews are refereed primarily by law students. Therefore, etc.

Jonathan Ichikawa
9 years ago

In response to Walto #112, I think it’s perfectly appropriate for editors to make their own decisions, taking the advice of the referees into account. Referees vary in expertise and quality a lot. The reason we have editors with considerable experience and expertise is because somebody has to make these decisions. I don’t think it would be an improvement if all the editor did was send the manuscript in the direction of referees who made the final decisions themselves.

But all of this underscores just how important it is to have editors who exercise good judgment. Short of a really significant overhaul of the journal system—something I’d love to have, but don’t really see how to get—there’s no getting around the fact that editors play an important role, and we have to trust them to play it well.

That’s why it’s so distressing to learn of cases in which editors don’t follow the appropriate procedures. Unfortunately, editing, like refereeing, is a rather thankless task. (And unlike refereeing, it takes up a ton of someone’s time.) I am sure that, just as it is hard to find good referees, it is also hard to find good editors. This is part of the problem, I think. Those editors who who follow proper procedures and employ good judgment do so out of their own sense of ethics and professional responsibility. There’s very little external pressure on journal editors to be good editors. Opinions expressed in comments threads on blogs can generate a little. But unless there are lots of qualified people who want to be journal editors, the pressure will always be pretty limited.

Walto
Walto
Reply to  Jonathan Ichikawa
9 years ago

I haven’t said that my treatment was inappropriate, but I don’t see the point of carefully ensuring the blindness of the referees if the editor knows who the author is the person who will make the decision on publication regardless of whether or not the referees like the paper. It seems like a waste of time and trouble.

Mavan
Mavan
9 years ago

I’m not a philosopher, and as a result I found reading this thread fascinating.

In my field, psychology, very few papers are blinded to reviewers, I’ve never heard of a paper being blinded to an editor, and many reviewers sign their reviews. The Frontiers journals publish the names of reviewers along with the paper and PeerJ even publishes the full text of of the reviews and response letters.

I think making everything open is a much better way of dealing with bias and poor behaviour.

Bill
Bill
9 years ago

In response to Marvan: it’s worth noting that there are some philosophy journals that insist that recipients of referees’ reports keep those reports confidential (even when they are anonymous) and also some that have a policy of not making referees’ reports available to other referees.

It strikes me that if we want more transparency in refereeing, trying to change this would be a better place to start than getting people to sign their reviews or abandoning triply anonymous review (both of which, whatever their advantages, have known downsides.)

nobody important
nobody important
6 years ago

I just went to submit a paper to Phil Studies and their current editorial procedure says this:

Double-blind review procedure
The journal follows a double-blind reviewing procedure. Authors are therefore requested to place their name and affiliation on a separate page. Self-identifying citations and references in the article text should either be avoided or left blank when manuscripts are first submitted. Authors are responsible for reinserting self-identifying citations and references when manuscripts are prepared for final submission.

But I remembered this thread and felt that this didn’t seem to mesh very well with the editor’s comments in the post.

Postdoc
Postdoc
6 years ago

As ‘nobody important’ resurrected this dead discussion, I thought I’d add to it too.

I submitted a paper to Philosophical Studies in 2016/2017. After two months or so the status still read ‘editor assigned.’ As this is unusual in my experience, I decided to email asking why it had not been sent for review. I got no response. I emailed a few more times. No response. So, I withdrew the paper and sent it elsewhere.

The status at Phil. Stud. stayed ‘editor assigned,’ then it changed to ‘reviewers assigned’ after some more time passed (maybe a month, maybe two). The status never changed to ‘under review.’ After a few more months, the paper was rejected without comments.

Did the editor ever get any of my emails? Was the paper ever reviewed? Was the paper ever even read? There’s no telling.

But whatever happened, or didn’t happen, it took 6 months!

Thankfully I had submitted it elsewhere months earlier after sending my withdraw into the void that is Philosophical Studies.