Welcome back to Ought Experiment! Today’s question is from a philosopher reeling from yet another journal rejection, and starting to wonder if publishing is an arbitrary (or even intentionally cruel) ordeal:

Dear Louie,

My favorite paper was just rejected for the 7th time. Let’s see, I’ve had desk rejections, rejections without referee comments, rejections for fairly superficial reasons, rejections because the referee personally disagreed with my position (my bad, guy!), and the most frustrating of all, tonight’s rejection, where the rationale for rejection DIRECTLY contradicted what the previous journal suggested I do! And this isn’t the only paper of mine getting hosed, either.

I know the party line here is “just keep at it,” and that it’s apparently common to go through seven or even more rejections before finally getting a hit. But that just makes me feel like stupid gullible Charlie Brown, and the profession is Lucy yanking away the football at the last second. Oh, except that Lucy is also the ref, and will boot me from the profession if I fail to kick the ball enough times…

So here’s my question: how can I tell the difference between a “normal” rejection spree and a paper that I should give up on? Oh, and the same question about my career, too. Because if everyone agrees that endless rejections are common and journal decisions are often arbitrary but everyone ALSO insists that a lot of high quality publications in a short amount of time is a fair standard to hold people to because publications signal useful information, then I’m not sure I want to play anymore.

Best,
Charlie Brown

Dear Charlie Brown,

Just, err, keep at it.

Look, I get it. I’ve been there, complete with my own brutally fatalistic hair-pulling montage. Whether you’re on the market, trying to publish your way out of a job, or on the tenure track, the clock is ticking loudly, and it can often feel like your fate rests in the hands of one or two uncharitable strangers. String enough of those strangers together in a row, and you’re effectively walled off from the career you want to have. No matter how hard you work, you’re just not in control of your fate. And the people actually in control of your fate don’t seem to be operating according to any systematic rulebook. Tick, tick, tick.

More than a little vexing, that.

Perhaps the best bit of advice I can give you is to avoid seeing your rejections as a pattern. I know it’s tempting. Explanatory narratives are comforting, even when the narratives themselves are rather gloomy. But false patterns also allow debilitating frustration to build, convincing you to despair when all you need is a little more time, and to overcorrect in ways that make your work worse. Patterns can also prevent you from seeing or accepting more tangible solutions.

The reason why it doesn’t seem like your various referees are operating according to a systematic rulebook is because, well, they aren’t. The profession is not an agent with discernable intentions. It has not made any judgments about your quality, nor has it resolved to carry on without you. To paraphrase Salma Hayek, what you’re experiencing as a meaningful pattern is really just the aggregate of localized individuals with different values reaching disconnected verdicts. Lucy may have been a hope-raising, hope-dashing jerk to Charlie, but there’s no Lucy here.

You might be thinking: “Great. You’ve just replaced a concerted conspiracy with indifferent and unfeeling chaos. I’m so much more comforted now. Thanks, Louie!” But I honestly consider chaos a hopeful perspective. Why? Because it means the profession hasn’t rejected you. A dozen or so shmucks have. And that’s nowhere near as decisive. The more your circumstances resemble a coin flip, the less it makes sense to infer that the next flip will also come up tails.

If every one of your many referees had given you the exact same advice, then it wouldn’t be either “a normal rejection spree” or “a paper you should give up on”, but instead a paper that you should change accordingly. You could safely conclude from such consensus that they were tracking some objective feature about the paper (or at least some feature of currently favored work that it might be in your interest to match). But your referees aren’t giving you the exact same advice. You mention that one ref rejected it for superficial reasons. Another rejected it because they substantively disagree with your position. Another rejected it because you failed to do X, and then the next one in line rejected it because you did X. When you get an inconsistent array like that, chances are good that the referees aren’t tracking any fatal defects in the paper, which means you shouldn’t give up on it. They’re just having highly individualized reactions. In which case the only thing you need to change is the pair of eyes evaluating your work. Send it back out. Because the next ref’s highly individualized reaction might be a positive one.

This is why the party line is “just keep at it”. People who care about you and think well of your work aren’t advising you to hang in there because it’s amusing to watch someone fail to learn a lesson, like that old joke about the definition of insanity. They aren’t advising you to bash your head against a wall until you open up a crack in the brick large enough for a paper to fit through. Nor are they telling you to keep at it because this is an endurance challenge or test of attrition where, per prior secret convention, the first seven submissions are always rejected. They’re telling you to just keep at it because each journal submission is a fresh coin flip. This is also why all the people telling you about their own epic rejection streaks are still around. You eventually break through. And it can happen suddenly. But it can only happen if you keep trying.

When my rejections are accompanied by referee reports, I’ll make changes to the paper if a referee spotted an obvious mistake, or if they persuaded me that the paper genuinely would be better if I did or didn’t do some thing. But most of the time, I don’t change anything. I just send the paper, as-is, to the next journal. And wouldn’t you know it: I eventually get an R&R. It would only make sense to faithfully implement all the changes that rejecting referee N wanted to see if the profession were a solitary or coordinated agent with consistent tastes and standards. It’s not. All those revisions won’t do anything to guarantee a favorable reaction from referee N+1 (and may even hinder your chances if referee N’s advice was particularly idiosyncratic or grumpy). Each review proceeds on its own terms.

I get why a rationally ordered process feels better – there are clearly understandable hurdles and benchmarks, and success becomes a matter of dutifully following the maximally transparent steps. There is no luck, and nothing is arbitrary. But imagining that the process is ordered in that way when it isn’t leads us to systematically misinterpret feedback, hacking and stitching our papers into Frankenstein monsters in the hopes of ‘finally’ satisfying ‘the profession’, or giving up when we were possibly one submission away from success because we thought ‘the profession’ had already spoken.

Abstracting away from any one individual’s employment situation, I don’t think many of us would actually prefer that the field bow to a centralized set of judgments or criteria that inevitably stamp out the same research month after month. Localized, uncoordinated referees can introduce a lot of inconsistency into the process, especially when those referees don’t read carefully or consider fairly (ahem), but referees and editors making individual decisions according to their own values is also the only way that weirder, consensus-bucking, boundary-pushing work can get published. An intentional process might guarantee results for everyone that best conforms to the publically determined mold of ‘good work’ or ‘research that makes a contribution’, but that assumes that the only role of publications is to unlock the tower for its next generation of residents. If we want publications to have any chance of expanding the frontiers of truth, then we want hundreds upon hundreds of gatekeepers playing by entirely different rules. Maybe that sentiment marks me as an old romantic, but I genuinely believe that the field is better off without coordination.

Which, to be totally and absolutely clear—because I can already see the torches and pitchforks cresting yonder hill—does not mean that individual referee reports are beyond criticism. Of course some referees make bad calls. Of course there’s something unnecessarily dicey about a situation where the only way to survive is to keep gambling on a succession of slow referees while the clock ticks away. Of course there’s something curious about treating partially arbitrary positional goods as reliable signals of merit. Of course we overvalue landing your paper in the best fifteen journal buckets, when there’s amazing research hanging out in other journal buckets (not to mention rigorous public philosophy that eschews the bucket scramble altogether, and reaches many more readers besides). Maybe we need to rethink how we evaluate dossiers. Maybe practices need to change. These are good discussions to have. (If you’re interested, see: here, here, here, and here, just for starters.)

My sense is that a lot of the heartache involved in time-sensitive publishing can be mitigated if referees and potential referees acted better. Yes, stray reader, I’m talking about you now. It can be as simple as actually declining a request to referee instead of ignoring the email, cutting weeks of wasted time out of the process. It can also mean accepting a request instead of declining it. Yes, it’s onerous and uncompensated labor, but that’s how our field is currently arranged, so you have to chip in even when it’s not in your direct interest to do so. It can also mean completing a referee report some time before the deadline, instead of treating the deadline as your first reminder to read the paper. Yes, we’re all very, very busy, but commitments matter. It can also mean reading the paper with more care than usual, and using standards like “this paper contributes to the debate” instead of “this paper supports my side of the debate”. Not because it’s our job to bestow jobs on people who want them very badly, or because the state of knowledge somehow improves when we let anything through, but because as scholars, false negatives should concern us all.

When the best institutional arrangement involves disconnected referees reaching uncoordinated and highly individualized verdicts, participating individuals have to see refereeing as a responsibility, as something owed. That’s the only way this works.

Now back to you, Charlie Brown. How can you tell the difference between a normal rejection spree and a paper that you should give up on? The inconsistent verdicts you’re getting tell me that it’s probably the former. Ditto the fact that many of us contend with similar streaks. Sharing a common problem rarely indicates that there’s something uniquely wrong with you. But it would be a good idea to get input from people other than journal referees. Ask colleagues. Ask friends. And most importantly, ask yourself. Basing your sense of the paper (let alone your sense of your career prospects) on what a dozen referees have said is some risky thinking. Remember, the profession is not an agent. It does not speak with one voice. So what matters, in the end, is learning to trust your own voice.

Just keep at it.

— Louie Generis

Do you want Louie Generis to tell you what to do? Send your questions to [email protected]! You can also follow Louie on Facebook. And in the meantime, continue the discussion in the comments below.