A Desk Rejection Scorecard (guest post by Antti Kauppinen)


The following is a guest post* by Antti Kauppinen, currently an Academy of Finland Research Fellow at the Department of Philosophy at the University of Tampere, and soon to be (as of 2018) Professor of Social and Moral Philosophy at the University of Helsinki. It’s about improving desk rejection: the practice of editors at academic journals rejecting papers without first sending them out to peer reviewers.


A Desk Rejection Scorecard
by Antti Kauppinen

Sometimes my papers get desk rejected by prestigious journals. (I know—crazy, isn’t it?) Apparently, I’m not the only one to whom this happens. And it seems that I’m not the only one who is left not only with bandaged pride but also a great deal of puzzlement: What was wrong with it? How could I improve it, and is it even worth trying? This is because desk rejections standardly come with no feedback whatsoever, since editors don’t have time to write comments on everything. This makes perfect sense, but I’m going to suggest a small tweak that might provide potentially significant benefit for authors at very little cost to editors.

The proposal is simple: journals should have a desk rejection scorecard containing a list of most common reasons for rejection, and editors should tick all the boxes that apply. Here’s an initial sketch:

Thank you for your interest in publishing in the Journal of Prestigious Studies. Unfortunately, our editor or associate editor has decided against sending your paper for review. Because of the great volume of submissions to the journal, we cannot give comments on all the papers. However, the boxes ticked on the following standardized feedback form give some idea of the grounds for our decision.

We have decided against sending the paper for review for the following reasons (all applicable ones are ticked):

[ ] Not original enough
[ ] Not clear enough or stylistically poor
[ ] Doesn’t sufficiently engage with the literature
[ ] Doesn’t sufficiently advance the debate
[ ] Topic not important or interesting enough
[ ] Fails to satisfactorily address objections
[ ] Not a good fit for this journal
[ ] Other

This is just a first pass. I’ve derived these from some virtues of a good philosophy paper—originality, clear and engaging style, responsible scholarship, making a significant contribution to an interesting debate, and anticipating objections. I’m sure there are more, and people who actually have editorial experience could likely come up with a better list.

I think the benefits of the proposal for authors are obvious. If I got a rejection with only the “Not original enough” box ticked, I’d do my best to highlight what I think is new about my paper. (Note that not ticking a box can also convey a lot of information—for example, I can infer that the paper is clear enough from the fact that the box “Not clear enough” isn’t ticked.) If I got a rejection with both “Not original enough” and “Doesn’t sufficiently engage with the literature” ticked, I’d think “I need to check out what that goddamn Dale Dorsey has published this time”. If the rejection said “Doesn’t sufficiently advance the debate”, I might think “OK, I’m making a minor new point—maybe I’ll try a specialist journal”. And so on.

I should say that this proposal isn’t in any way radically revisionary, since some journals already give feedback along these lines, at least on an ad hoc basis. For example, I recently had a paper desk rejected by the Journal of Political Philosophy (I use the name, because I’m about to praise them). It took them only a few days, and they said basically “It’s an interesting paper, but the political angle isn’t all that developed. You should try an ethics journal or a generalist one.” That was very useful (and fair enough—I had only sent it to JPP because a friend recommended doing so), and the paper is now under review elsewhere, unchanged. Using the scorecard wouldn’t have conveyed all of that information, but having only the “Not fit for this journal” box would have done much the same.

So, let’s say this is good for authors. If there’s an objection to the proposal, it would have to be that it imposes an additional burden on already overburdened editors. But does it? Presumably, editors who reject a paper read it first, or at least read some of it, and have some grounds for their decision beyond “My gut tells me this isn’t good enough”. For example, I can imagine editors reading some of my (luckily, now published) papers and having thoughts along the following lines: “Great, another fucking paper about how empathy isn’t so bad for morality—if only you could feel my pain reading this shit”; “Narrow scope, wide scope, why not fucking periscope… Louise, wake me when this paper is over!”; “Oh, it’s all about narrative, is it? Well, here’s a sad fucking story for you: you sent a paper to a journal, and then it got rejected.” These are the kind of thoughts that occur unbidden when one is reading a paper one doesn’t like. Each would easily translate to checking one or more of the boxes above.

So what’s not to like?

 

Use innovative tools to teach clear and courageous thinking
Subscribe
Notify of
guest

43 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Fritz Warfield
Fritz Warfield
6 years ago

Fails to satisfactorily address objections.
Not original enough.
Topic not important enough.
Fails to advance the debate.

Ben
Ben
6 years ago
Rebecca
Rebecca
6 years ago

I’m a journal editor and I tentatively like this proposal. My hesitation is not about increased workload – I agree with you that it would be very little extra work and that it would provide a real service. But as it is, I am inundated with a**holes who want to argue with and insult and threaten me when I reject their paper. It’s exhausting and awful and very gendered. I suspect that giving someone this kind of checklist would often just be taken as an opening to argue the checklist with me point by point.

I don’t think that’s a good enough reason to reject your proposal, but I do get sort of exhausted just imagining the fallout from doing this, even though the initial work seems trivial.

Dale Miller
Reply to  Rebecca
6 years ago

I’m not sure that I’ve ever given feedback on a desk rejection without coming to regret it. That’s not entirely true; quite a few of my desk rejections are on the grounds of lack of fit with the journal, and when I say that I usually don’t get any response. But I’ve stopped giving substantive feedback on papers, since it seems to be read as an invitation to discuss.

Douglas W. Portmore
Reply to  Dale Miller
6 years ago

Exactly. That’s my experience as well.

Fritz Allhoff
Fritz Allhoff
Reply to  Dale Miller
6 years ago

Agreed with Dale. If I try to help, authors use it as a launching point for email arguments. Also “not original enough” people take personally. I’d rather say less than more, like “due to the high number of quality submissions, we are unable to extend an offer of publication at this time.”

Also, while I do read everything I’ve desk rejected, I think it’s overly optimistic to assume that’s standard. Some editors probably look at name/institution, title, abstract, and stop there–for better or worse.

David Velleman
David Velleman
Reply to  Fritz Allhoff
6 years ago

Respectable institutions don’t look at “name/institution” because their editorial process is blind from the start.

Julia
Julia
Reply to  David Velleman
6 years ago

I guess several of the top philosophy journals are not respectable institutions then – http://dailynous.com/2015/02/02/guarding-the-guardians-or-editors/

Sergio Tenenbaum
Sergio Tenenbaum
Reply to  Dale Miller
6 years ago

Same here.

Michael Kremer
Michael Kremer
Reply to  Rebecca
6 years ago

Why not just send out the form along with a letter stating that you will not discuss the decision with the author(s)? And then just stick to it — if they write, send back a one-sentence reminder: we do not discuss desk rejections with authors.

Rebecca
Rebecca
Reply to  Michael Kremer
6 years ago

Michael, that is how I reply when they challenge me, almost always. But I am reluctant to send it out with the original rejection because (a) every once in a while there is something legitimate to discuss – only 95% of the time are they just being an a**hole, and (b) it feels so hostile up front. Maybe I am just a softie.

Dale Dorsey
Dale Dorsey
6 years ago

That goddamn Antti Kauppinen has another good idea.

Douglas W. Portmore
6 years ago

I’m with Rebecca in suspecting that this would significantly increase the amount of fallout that editors have to deal with. And it’s not clear to me that the putative benefits to authors would adequately compensate for this. For I suspect that the benefits to authors would be quite minimal. Consider how often authors reject criticisms of their papers even when those criticisms come with detailed explanations. And it seems likely that a far greater number will reject criticisms that come with no explanation at all. So, even if some authors like yourself would do their best to highlight what they think is new and important about their papers when they get back a rejection with only the “Not original enough” box ticked, I suspect that most will just conclude that the editor is an idiot for not seeing what is obviously original and important about their papers. And I doubt that any will conclude, although some should, that there is nothing original or important about their papers. Lastly, I think that editors are already grossly overburdened with a thankless job. Indeed, it’s hard to recruit good people to be editors these days. So, I think that it’s a bad idea to add to their duties. And, as I see it, editors’ duties primarily lie with only readers and potential reviewers. They have a duty to readers to ensure that only the good gets published and a duty to reviewers to ensure that they not be asked to waste their time reviewing papers that have very little chance of getting through the review process. They don’t, it seems to me, have a duty to help authors improve their papers. Their only duty to authors is to ensure that their submissions are treated fairly and professionally.

Antti
Antti
Reply to  Douglas W. Portmore
6 years ago

Doug, I can certainly see the dangers you mention. One thing that occurred to me after writing the post was emphasizing even more in the form letter that there’s a policy of having no discussion, and heck, having a form letter saying “We said no discussion about the decision!” to send in response to those who insist on arguing back. But I don’t know – as I confess, I haven’t been an editor, and it wouldn’t occur to me to start arguing with one myself.

However, I think you’re underestimating the benefits to the author. I wasn’t assuming that most people would think “Gee, I guess it’s not an original idea after all”. I would probably not think so myself (although I have given up or significantly modified projects when I’ve gotten verbal feedback to that effect). But I thought the benefit would be in terms of knowing what to focus on in rewriting the paper. If I concluded, after reflection, that I was still convinced of the originality of my paper, I’d do more work to emphasize how it’s different from other views already out there. (In fact, I’ve just revised a paper along those lines, and I think spelling out more carefully how it differs from similar proposals made it much better.) But I will of course grant that how useful any negative feedback is to an author depends on their attitude, and the tick might be entirely wasted on some.

Douglas W. Portmore
Reply to  Antti
6 years ago

How do you propose to prevent authors from emailing the editor to express their objections to his or her expressed reasons for desk rejecting their papers? You can have a no-discussion policy. But that itself doesn’t prevent authors from emailing the editor. And I’ll just report that my experience is the same as Dale Miller’s. Most of the time, I’ve ended up regretting providing feedback on desk rejections. So, I rarely do it any more. On the other point, I agree that some authors would benefit somewhat from this proposed policy, but I just don’t think that it would be that beneficial to tick a few boxes without giving any explanation for why those boxes were ticked — especially, given the subjective nature of these judgments. Suppose, for instance, I tick the box saying that your paper doesn’t adequately engage with some of the relevant literature. Won’t it be infuriating not to be told what I think is the relevant literature that you needed to engage with. After all, you presumably didn’t think that there was some relevant literature that you needed to engage with or you would have engaged with it. And won’t many authors be tempted to email the editor to ask. And if I’m the editor and you’re Antti (or someone else I know and respect), won’t I be tempted to respond. But, most importantly, I don’t see why we should see it as the editor’s duty to help authors improve their papers. Don’t they have enough to do already?

Antti
Antti
Reply to  Douglas W. Portmore
6 years ago

Doug, let me first say I’m surprised and sorry to hear that you (and other editors) have been the target of authors’ ire for providing feedback for a rejection. I genuinely didn’t realize it is as common as it seems to be. Good editors like yourself provide a hugely valuable service to the profession, and we should bear that in mind in spite of personal disappointment.

How would you feel about a compromise: having an optional scorecard for when some reason for rejection stands out? You wouldn’t use it if you thought the paper was just too meh to bother referees with, but only if it was clear to you that the paper suffered from some standard shortcoming.

There’s a bit of a collective action problem here, but being perhaps too optimistic about human nature, I can imagine that if enough journals did something like this and stuck with the policy of not responding to queries, people would start to think of ‘form rejection’ in the same way as they now think of comment-free desk rejection, as something for which there is no court of appeal. I’m sure I would myself be curious to know more in the kind of case you mention, but even a few bits of information would beat the fish slap from my perspective. (Maybe I should also say this post was occasioned by someone else’s commentless rejection after several months, not my own experience. I’ve been pretty lucky.)

Douglas W. Portmore
Reply to  Antti
6 years ago

I see no problem with having an optional scorecard for when some reason for rejection stands out.

Michael Kremer
Michael Kremer
Reply to  Antti
6 years ago

I just made this suggestion (of Antti’s) in response to Rebecca, before reading Antti’s reply to Douglas. It seems the right way to go.

Antti
Antti
Reply to  Douglas W. Portmore
6 years ago

Oh, and don’t the haters hate you even if you desk reject without comment? After all, a desk rejection without comment feels just like being smacked in the face with a mid-sized fish, and is about as educational.

Douglas W. Portmore
Reply to  Antti
6 years ago

I think that people will hate editors more for rejecting their papers for reasons that they take to be unwarranted than for rejecting their papers for unknown reasons. But, perhaps, I’m just projecting my own responses on others. I get really pissed when a reviewer recommends rejecting my paper for a bad reason than when an editor rejects my paper for some unknown reason. Besides, I think that you forgot to include on your list one of the most prominent reasons for rejecting a paper: Just not as interesting, important, or good as other submissions. And I doubt that many authors will find this all that helpful unless they are told in detail why their paper isn’t as interesting, important, or good.

Rebecca
Rebecca
Reply to  Douglas W. Portmore
6 years ago

Yeah, my experience leads me to totally agree with Doug here. The more reasons I give, the more they hate and argue.

Antti
Antti
Reply to  Rebecca
6 years ago

Very sorry to hear that, Rebecca! This discussion has revealed to me an aspect of editorial work I didn’t appreciate before. I guess I’d still like some brave journal to give this thing a try (at least in the more moderate form I propose above in response to Doug), and see whether it results in even more assholery.

Bruce
Bruce
6 years ago

[ ] Your supervisor/someone reading your university email in the administration/colleague sent the editor an email saying no one likes you and so we’re not pulbishing anything you send us.
[ ] We don’t like your politics/religion/jokes/face/skin colour/atheism in your Twitter/Facebook/Google/Blog and so we’re doing a little Hitler on you.
[ ] You are the subject of a smear campaign, so, not gonna happen.
[ ] You aren’t a member of our in club/group/lodge/church/business assoication or related to anyone we know on any of these, so tough luck

Kris
Kris
6 years ago

I like the proposal.
I am surprised (and disappointed) that so many editors feel under siege from emails from authors of papers. It would never occur to me to contact an editor to argue against a verdict he or she delivered, since it would almost certainly not achieve the desired goal and would probably generate ill- will from the editor.
I am curious what percentage of the “argue with the editor letters” are written by junior scholars/graduate students vs. senior members of the profession.

Rebecca Kukla
Rebecca Kukla
Reply to  Kris
6 years ago

In my experience they are almost 100% written by men at any stage of career, with a slight skewing towards more senior people doing more arguing. But the gender effect swamps the stage of career effect.

To be clear, by ‘under siege’ I don’t just mean frustrated emails – I mean I’ve been personally insulted and threatened and slurred countless times.

I just realized that the site has only been including my first name on my posts – I wasn’t trying to be anonymous. I am the same Rebecca who posted a few times above.

Kris McDaniel
Kris McDaniel
Reply to  Rebecca Kukla
6 years ago

I’m Kris McDaniel. Wasn’t trying to be either just forgot to add last name. 🙂

Kris McDaniel
Kris McDaniel
Reply to  Kris McDaniel
6 years ago

And I am very sorry to hear about your being insulted or slurred.

Lizard
Lizard
Reply to  Rebecca Kukla
6 years ago

I can entirely believe that the worst comments would be mostly from men, and of those slightly more from more senior men. But I also wish there were a better answer to the problem! It seems like not implementing scorecards or giving reasons for desk rejections would be harming the prospects for e.g. younger female scholars trying to get published all because of the behaviour of some men. As a member of the former group, and one who is likely to get her first desk rejection soon, it’s frustrating. And I don’t know what the right thing to do would be.

Steve
Steve
6 years ago

Every paper I’ve ever had desk rejected was a startling work of utter brilliance, so maybe there should be an extra box along the lines of
() I am an idiot who would reject the B deduction, were I alive in the Eighteenth Century

PS oddly, many of these papers have been made even more startlingly brilliant by feedback from anonymous referees, even when this feedback was part of a rejection letter. Hard to know what to make of that…

R
R
6 years ago

Potentially a good suggestion, that can benefit authors, but only in the presence of triple-blind review (where the editor is unaware of the author’s identity). Otherwise, if there is general journal policy that rejections must come with a checklist explanation, editors will sometimes rationalize why they are rejecting a piece, and sometimes indicate the wrong (motivating) reasons for rejection. This will sometimes serve as bad advice to the author. Otherwise said, editors will occasionally use heuristics (consciously or unconsciously) that they won’t end up articulating. Instead, they will indicate something else, which will either be useless or worse for the author. (Since we are all caretakers of inference here, this point will not be taken as an attack on editors [who largely deserve our appreciation], but as a point about human nature in the midst of institutional structures like our own.) Most of the points here have concerned the partiality of the authors, but we shouldn’t forget the biases of those charged with gatekeeping. Also, triple-blind review would add to the authority of the desk rejection. An author receiving a checklist from an editor who doesn’t know who they are, or what they’re about, is more likely to treat the indications as authoritative.

John Schwenkler
John Schwenkler
6 years ago
Small fish
Small fish
6 years ago

I also like the proposal and am disappointed to see that so many editors reject it for an unconvincing reason. With Kris, I wonder who dares to argue with the editors about their decisions and why the editors are troubled with their arguments. The only reasonable answer seems to be that the argument letters from the Big Names.

Sergio Tenenbaum
Sergio Tenenbaum
6 years ago

I think people are misunderstanding the relevance of the comment about pushback. I don’t care about receiving pushback on my one sentence comment on a desk rejection (If you do it, you just get a form email explaining why we don’t elaborate on desk rejections). It’s presented as evidence that it’s more frustrating than helpful to authors (it’s also illustrated by my reaction to the first, rude, comment on this thread. I thought about replying: “this is totally unhelpful; why don’t you elaborate …” And then it finally dawned on me I was falling for the prank). My main objection to the scorecard is that I don’t see how it could possibly help me, but I can see it slowing down the process. That is, my typing abilities are limited, but I can type a sentence pretty fast. I was also not particularly surprised by Antii’s list of shortcomings; I had a good sense before I saw the list that these were possible shortcomings of a paper. When I think I can helpfully convey in one sentence the reason to reject the paper (even if it would take me two sentences!), I do it. The most common case is the one Antii describes, and it is obviously helpful and pleasing to know that all you need to do is to put a paper in new, better addressed, virtual envelope. But most time I desk reject something, it’s due to a combination of factors; perhaps if the contribution was more significant, I would have overlooked the failure to engage with some literature or the fact that the author missed an important objection, and send it to referees. When deciding whether to send a paper to referees, I’m asking “is it likely enough that a referee will accept this paper and persuade me that it should be accepted?”. A “no” answer is often the result of a number of perceived flaws. But, one may ask, why not tick all the relevant boxes? But the more boxes you tick, the less helpful the feedback is. Moreover, I then start to ask myself: “was the paper really fine with regard to the other criteria”? “Aren’t some of the boxes I ticked really more important than others I ticked?” And even if just one box, such as “engaged with the relevant literature”, is it obvious which literature I thought it was relevant that the author missed? (I once rejected a paper that did not cite anything later than 1990 on a topic that had been all the rage for the last 20 years. That was an easy call, and, of course, I explained that this was the problem. But it’s rarely that straightforward). These are not idle worries; often, revisions make paper less readable, more convoluted, or worse in some other way. So if I had to check boxes, I would want to make sure that I explain to the author why I checked those boxes, and then I’d be writing a full report on the paper.

Tl;dr: I don’t think editors have a duty to send comments, but I do think they have a duty not to send misleading comments. I worry that forcing editors to fill scorecards would amount to ask them to send significant comments on papers desk rejected in light of this duty.

Antti
Antti
Reply to  Sergio Tenenbaum
6 years ago

Hi Sergio, that’s a good point. Perhaps I framed the argument in too strong terms. What I think is that some authors would benefit from the practice some of the time, and I think I would be among them.

Do you think it would help make this palatable if the scorecard, or the possibility of a ‘form rejection’, was optional, as I suggested in my comment to Doug?

Sergio Tenenbaum
Sergio Tenenbaum
Reply to  Antti
6 years ago

Of course, I don’t mind if the journal has some guidelines that say something like “If you think it would be helpful, here are a few examples of one sentence comments you could send”. It wouldn’t help me in particular, because that’s what I do anyway. As long as there’s no instituted practice or expectation that the default is to send back a scorecard, I don’t have anything against it.

Sergio Tenenbaum
Sergio Tenenbaum
6 years ago

Antti, I mispelled your name throughout the comment! I blame autocorrect!

Antti
Antti
Reply to  Sergio Tenenbaum
6 years ago

No worries, Serge!

Bruce
Bruce
6 years ago

[ ] This is a meritocracy dammit, and my besties in the department/lover from a previous conference/someone I am certain is better than you having never met you – hates you and says you have no merit so the quality of your paper is inconsequential to your being sandbagged (Don’t try to tell me that is not how a meritocracy works – I am a Humean about morals, even though I hate Hume)
[ ] See above, and it is your fault that I am ethically challenged
[ ] I have studied Newcombe’s treatment of Peirce, and the academy’s treatment of Hume, and Newcombe and the academy are my heroic role models, because I have exactly the wrong personality type to do this job ethically, but that’s how I got the job in the first place
[ ] I have formed the belief that you have no friends, and that is enough to embolden me

Richard E. Hennessey
Richard E. Hennessey
6 years ago

How about? [ ] Gratuitous use of offensive language.

CW
CW
6 years ago

Desk Rejection Rejection Scorecard (an aid for the desk rejected)

[ ] I do too understand figure/argument X. You don’t understand it.
[ ] Wrong. My argument is nothing like the argument advanced by Professor P.
[ ] Hold on, I said no such thing in my paper!
[ ] The literature in this area is all wrong. My paper is a fresh start.
[ ] I addressed all of the *relevant* literature in this area.
[ ] I did too address that objection.
[ ] But that objection isn’t worth addressing.
[ ] Responding to that objection would require a second paper.
[ ] Are you sure you read my paper?
[ ] You suck.

Tim O'Keefe
Reply to  CW
6 years ago

Reminds of the old “I respectfully reject your rejection of my application, and I will assuming your assistant professor position this Fall” letter. Found a copy of it here:

comment image

CW
CW
Reply to  Tim O'Keefe
6 years ago

I was on the market when I first saw that letter, about 10 years ago. Inspiring!

Pete Lindstrom
Pete Lindstrom
6 years ago

From a non-academic observer who has similar challenges in a related field: The best thing about this entire thread is that everybody is proving everyone else’s point. Really fascinating from a personality and human communication standpoint. I appreciate all the thoughtful comments.