Flipping the System: One Possible Solution to the Publishing Odyssey (guest post by Felix Bender)
In the following guest post*, Felix Bender (CEU / Amsterdam) surveys some proposed solutions to our current time-consuming, backed-up, overcrowded system of publishing academic articles, as well as some problems with them, before offering up an interesting solution of his own.
[photo by J. Weinberg]
Flipping the System: One Possible Solution to the Publishing Odyssey
by Felix Bender
1. The Problem
We all know the dreadful journey our papers must take until they are not only received positively by an editor, but sent out to review, received (somewhat) positively by the reviewers and the reviews receiving somewhat positive responses from the editor. Often, there is a second iteration of this whole process. Even more often, however, a paper’s journey ends before any of the latter steps: they are simply desk rejected.
In Political Philosophy this seems to be a huge problem. The acceptance rates of some journals are very low[1], making them much lower than the acceptance rates in other academic disciplines. For scholars this often means that they have to go through many, many submission processes until they find a journal that is at least as interested in their paper as they are in publishing in it. Researchers often wait weeks, if not months for even the first editorial decision on a manuscript, and then have to iterate the same procedure for many times for each journal, often being rejected on grounds such as fit or simply the mere preferences of the editors. This results in an ever-ballooning amount of papers existing in the cloud.[2]
2. Some Solutions Others Have Offered
How can this problem be solved? One solution would be to make the pool of papers floating around in the cloud smaller. Several different scholars have debated how this could be achieved. One could simply restrict access for some parts of academic professionals, thereby eliminating many contributions from the get-go. One could, for example, make it inadmissible for PhD students to submit papers before they graduate.[3] While this solution would undoubtedly cut the submissions to journals, it is not as promising as it sounds. Why? It favors the status-quo. Junior researchers, especially PhD students from less well-known universities have (mainly) one way of appearing in the landscape of their prominent peers, getting known and landing a job: publications. If they were barred from publishing papers, the PhD students at famous institutions such as Oxford, Cambridge, and EUI would cement their advantage. For if there is no other standard for judging the quality of a prospective hire, universities will revert mainly back again to the name of the institution and the academic stardom of the referees. Even if the system is not perfect as it is and people at such institutions are already advantaged, the possibility of publishing offers a way to overcome such barriers.
Another proposal would go in the exact opposite direction: restricting publication in peer-reviewed journals to junior academics only. This would entail that older (tenured) academics publish their work in online venues, series of invited volumes, and other outlets other than peer-reviewed journals. These outlets would encourage participation, leaving the possibility for replies. All in all, established scholars would not suffer from such an option. Their names are known, meaning that their work will be found and read—enough people want to know what the argument by a famous scholar in the field is. However, this method is faced with a different problem: it would merely shift the demand for publishing from peer-reviewed journals to publishing in such outlets. It would shift where the “value” in publishing lies, leading journal publications to be seen as less important than publications in which works by the well-known and established appear, and we would then see the exact same pooling, competition, waiting periods, and frustrations, but in this case for different outlets.
Others have proposed solutions that are, perhaps, more actionable but that do not solve the underlying problem at hand, such as introducing scorecards for desk-rejections.[4]
Another possible solution would be to adopt a “publish first, curate second” approach. Such an approach would entail that the authors decide when and what to publish. Peer reviews would also be published (anonymously or not) and curation would occur after publication, not before. While this may work in the life sciences, as Bodo M. Stern and Erin K. O’Shea suggest[5], one must be skeptical with regards to the humanities. The danger here seems to be that we would run into the opposite problem of what we are experiencing right now. We would run into the danger of being swamped by so many publications that we would, inevitably, resort to reading only the established names as a means of “quality assurance”.
3. Flipping the System
What if we could restructure the system in another way? What if we could do something besides banning parts of academia from publishing or asking journals to increase their acceptance rates (which would be great. But how should this be done? Forcing journals to have minimum acceptance rates?)[6]?
How could we, in other words, make the publishing system more efficient both for the authors and the journals?
One possible way to change this would involve the creation and use of a general platform or system to which all academics and all journals have access to. Researchers would upload their paper, title, and abstract to the system and it would appear there for journals to choose among.
In the current system, authors approach editors, seeking a journal in which to publish their work. My proposal flips this around: the editors, through the system, approach authors, seeking content for their journals.
Certain details would need to be worked out. It is imperative, for example, to make the process blind. (There might still be possible problems of some people in the system undermining anonymity, but there’s no reason to think that there will be more of that problem compared to the current system.)
My proposal involves making the system more transparent. One would be able to see which papers are selected for peer-review and which are eventually published.
Additionally, researchers would get rid of the process of searching for a fit with a journal, which can be extraordinarily time-consuming owing to being able to submit one’s paper to only one journal at a time. It would allow journals to select papers they are interested in, and if they want, create special issues on specific topics.
My proposed system would require editorial teams to comb through the database of articles and select ones they see as promising for their journals, but it would not necessarily mean a much greater workload. One possible way to limit the number of articles appearing in the system would be to allow each scholar to submit only (let’s say) two articles at a time. This in itself would promise not only a limitation of the size of articles floating around in the cloud, but also force scholars to further value the quality of their submissions over the quantity.
All in all, such a system would possibly make the publishing process more effective both for researchers and journals. That means: more time for researchers, less time waiting for first decisions by journals, more quality-oriented research articles, a better allocation system regarding the fit between article and journal, and more transparency in the system. This, one might think, is but a researcher’s utopia, but maybe it is a realistic one.
NOTES:
[1] The “Insanely Low Acceptance Rates” of Philosophy Journals
[2] 2,000 Spaces for 10,000 Papers: Why Everything Gets Rejected & Referees Are Exhausted (guest post by Neil Sinhababu)
[3] The Publication Emergency (guest post by J. David Velleman)
[4] A Desk Rejection Scorecard (guest post by Antti Kauppinen)
[5] A proposal for the future of scientific publishing in the life sciences by Bodo M. Stern and Erin K. O’Shea
[6] The “Insanely Low Acceptance Rates” of Philosophy Journals
If I’m not mistaken, basically this system has been in place at law reviews for decades: https://www.bepress.com/products/expresso/
There are some significant differences between the system for law reviews and this proposal. W/ law reviews, authors still send out the papers themselves, but there are multiple submissions, not a single submission to a central pool. (Law review submissions are also typically non-blind – you often send your CV with the paper. The results are predictable.) You can submit to as many journals as you want, or can afford – the systems used are not free. Professors at well-to-do schools usually have the schools pay for the submission via an institutional subscription, but not everywhere does this, and if you don’t have such access, it will add up very fast, especially as you really need to submit to lots of journals to have a decent chance. Most people get super fast rejections at most law reviews, based on “fit” or by looking at their CV. There are other games played – trying to leverage an offer into a better offer, etc. I’m less negative on the system than many people I know, but it has serious disadvantages. The biggest issue, though, in relation to the proposal here is that law reviews have large groups of workers – students – who make this work. (The law review I last published in had about 45 students working for it, for example. Others have more.) Journals, of course, don’t have anywhere close to this amount of labor power, especially not at the early stage considered here. So, it would take a rather massive change to how journals work if they were going to be similar to law reviews in this way. It doesn’t seem very plausible to me, but not, of course, impossible. (Also, w/ law reviews now, there are two “submissions seasons”, with one, mostly in Feb., being much more important. You have to have your paper ready to submit in one or both of these seasons, and just wait otherwise. This could perhaps be avoided, but it’s one more way that law review publishing is very different, and probably not a good model for philosophy.)
I second most of this except rejections from law reviews don’t usually come quickly, and sometimes don’t come at all. Many times they just finish the relevant issue and don’t bother to mass reject the articles they didn’t take. However. you’ll know if you’ve been de facto rejected pretty quickly.
It’s been a few years since I’ve done the “traditional” mass law review submission, but at that time, it was pretty common to get a fair number of rejections w/in a couple of hours – certainly in less than a day – as law reviews decided they were not interested in you or the article. (It is a “nice” feature of the software that the review can just hit “reject” and it’s done.) Some, admittedly, never get back to you, but at least when I last did this, it wasn’t common to get a good number of rejections super fast.
I have never seen a good argument for why it is unacceptable to send one paper out to multiple journals. Such permissibility would give the author a competitive edge and would give the journals motivation to render a verdict promptly before another journal does so. Of course, this would multiply the volume for editors to sift through, but if the current situation is overly burdensome for authors, especially junior scholars, wouldn’t it make sense to shift the overall burden away from the authors at some cost to the editors? Perhaps some journals would permit the practice and others wouldn’t, and authors could submit accordingly.
This doesn’t ‘shift’ the burden, it increases it – more person-hours would be required per article, due to the significant amount of extra (wasted) editor and referee time. I at least wouldn’t referee for a journal without the usual policy, since I wouldn’t want to spend considerable time refereeing a paper only to find that the author had decided to send it somewhere else instead.
We can’t shift the burdens to editors, because editors don’t have any incentive to shoulder the burden. Authors submit because publication is in their interests; editors edit because they feel an obligation. If it gets too burdensome, they won’t do it.
What Jay S describes is a policy of allowing “simultaneous submissions,” and it is the norm in the world of poetry (where there are far more submissions than in philosophy). Some poetry journals do not allow simultaneous submissions, but it is up to each journal to specify that on their submissions page.
Having been the editor of journal that allowed simultaneous submissions, I did not find it annoying or problematic. As long as the editorial team uses a good submission manager (e.g. Submittable), that allows authors to withdraw their submissions as soon as something is accepted elsewhere, there were minimal interruptions to the workflow.
Coming (back) to the world of philosophy publishing from the world of poetry, I find the norm of absolutely-no-simultaneous-submissions to be tortuous, for all the reasons that Felix mentions.
This sounds a lot like the “arXiv overlay journals” that exist in math. All mathematicians already put their papers up on the arXiv (a free, open, online repository, hosted by some combination of Cornell and UC Davis, if I recall correctly). Many journal editors have thus decided that rather than using a for-profit publisher, or even a university press publisher, they could make a journal fully open by keeping the editorial board, and just relying on the arXiv for all the version control and archiving purposes. So the journal just exists as a website with a table of contents for each “issue”, linking to articles on the arXiv.
https://en.wikipedia.org/wiki/Overlay_journal
This is a super interesting conversation. I guess I have two big questions; wonder your thoughts?
1. Why would editors want anything to do with this? I can imagine that you tick boxes when you upload your paper to this platform that say, for example, “ethics” or “meta-ethics” or “cognitivism” or however specific we wanted to go–cf., the PhilPapers architecture–but then what? Especially for a generalist journal (e.g., PhilReview); what do they search for?
2. Why would publishers want anything to do with this? Say there’s 10,000 papers on this thing, and then PhilReview (or whatever) “publishes” 10 of them. Just not sure what that’d mean at that stage; like they get a “tag” saying the name of the journal? And then what, do they stay on the platform for free or get migrated behind a paywall on the publisher’s site? Either way, there’s a lot of fair criticism about the corporate structure of our current journals, but just not sure how this doesn’t presuppose a complete overthrow of all that.
“The problem”, as stated, isn’t the real problem. The real problem is that the number of graduate students in a given department is determined by their personnel needs and, to a certain extent, the prestige involved in supervising dissertations. The system as a whole calls on the shrinking set of qualified reviewers to read an increasing volume of output. They don’t want to do this and aren’t going to do this.
Velleman favors stipulating that journals won’t do it, shifting the burden back onto hiring committees. The hiring committees tend to say that change would lead to increased prestige bias; this is a more polite way of saying that they’re not going to do it either.
We’re now already well past the “narrow technical” articles phase — no one wants to read those either. The requirement is now an abstract sexy enough to avoid reflex desk rejection. Then a couple reviewers will actually read it and it might get published. After that it’s back to the abstract again until a committee is narrowing down their phone screen list, when someone might look beyond page 1 to make sure they don’t embarrass themselves.
As Dr. Easwaran ·has pointed out in a different context what is really needed to break the logjam is conference poster sessions. That way those trusted to evaluate papers can absorb the abstract along with the important information of how the candidate looks and makes them feel. Then that important data, which is going to drive hiring in the end anyway, can feed back into the review process, so that hired candidates can be published candidates. No other solution is likely to be efficient enough.
I would actually appreciate hearing from an editor (David Wallace perhaps?) share more about how journals go about finding reviewers in the first place. A rare number ask you outright to suggest reviewers but most do not. As someone who has published for a while, I’m still surprised at how few reviewer requests I get (at most 1-2 per year). Now, I suppose this could mean that I’m a lousy reviewer but I do try to provide a couple of pages with every review. I’m not necessarily saying that I’d love to review more, but perhaps reviewers are overburdened because the pool of reviewers is artificially low? Are a few people just asked to review things all the time?
Seconded. I’d also love to hear how editors choose referees.
Judging from a handful of philosophers on Facebook who humblebrag about the fact that they receive too many requests to referee to keep up with all of them, I get the impression that referee selection is (like so many other parts of philosophy) an exercise in nepotism.
So, at the BJPS (and I suspect at many other journals) we have a database of names, addresses and key terms that is constantly expanding as we add new potential refs gleaned from trawling through places like Phil People, Stanford Encyclopaedia, other journals etc. Given how busy everyone is, and also how specialised some papers are (especially the ones we get), finding referees can be a difficult and protracted process (its not uncommon to go through 6-7 people before someone agrees – and given that each invitation and decline thereof can take up to a week or more, that all adds to the delay in getting a decision to the author). Not so much nepotism as an exercise in stamina …
As for the proposal, Jon Light asks the right questions – why would anyone agree to do something as time consuming and potentially frustrating as that? And why would publishers agree? How would the society that owns the BJPS get any income from such a process?
As Steven says, there is a lot of looking at sources that give you names of people who also work on the same topic as the topic of the paper you are finding referees for. But the search is often more fine grained than just looking for the same general topic. Papers are specific and you want someone with relatively specific expertise.
Sometimes the perfect person comes to mind and you can skip that step. But if you are practical you don’t just go to the most well-connected person you know who works on a topic (which I gather is what “nepotism” means here) since they are liable to be over-asked and overworked. You want to find people who you trust but who have the time. Maybe you trust them because you know them and have interacted with them in ways that make you trust them, but maybe you read a good paper they wrote and that is why you trust them.
Often no one comes to mind and that is where Phil Papers, SEP, etc., come in to help. But then there is another level of work. You have to look at abstracts for papers that are both relevant to the work you want advice on and which looks to be of good quality. It can take a good bit of work to find the right fit.
People who can’t do it often recommend someone they think can and eventually hopefully you find a decent referee. I think it is important to realize that referees are doing editors a favor by agreeing to advise us on the merits of a paper. We’re not going to referees to curry favor with them.
Associate Editor for BJPS here. My process for finding referees proceeds along several lines.
First, I look at whose work the paper under review is engaging with. Sometimes this means identifying who is the main target of critique, other times it just means seeing who is cited prominently as being important in establishing the terms of the debate. Those people whom the author takes to be the main landmarks relative to which their own views are situated are plausible candidates to approach. This, incidentally, highlights the importance of being reflective about one’s citational practices and the various purposes they can serve.
Second, I check up on who is actively doing work in the paper’s main areas. This usually means an extended search through PhilPapers to compile a list of candidates. I then have to look at each candidate’s total research profile and sometimes read through several of their publications to decide whether I think they would be a reliable judge for the journal. The journal’s referee database can also tell me whether they’ve refereed for us before, how recently, and what sort of job the editor thought they did in that case. I try not to overburden people with too many requests, but with narrow topics this is hard to avoid.
Third, I keep my own lists of people in various areas whose work I’m following so that I don’t have to rely on my own memory, which experience tells me is subject to all sorts of recall biases. Filling in these lists means trying to be aware of almost all of the work being done in the major areas that I’m responsible for (philosophy of psychology and neuroscience primarily) as well as cognate fields (e.g., general philosophy of science and philosophy of mind). Developing this sort of panoramic perspective requires reading extremely widely, but I regard keeping on top of trends, movements, emerging views, and active scholarship across the discipline as one of the main tasks of an editor. It not only helps with finding referees but also in seeing how (or whether) a submission will make a distinctive contribution.
Fourth, I go down the list of people who have been recommended as alternates by those who have declined my initial round of requests. If you must decline, offering a slate of 2-4 alternate names is invaluable, so please take a moment to do so.
As for why some people are approached more than others: when I have leaned too heavily on one or two people it’s almost always because the topic is a narrow or new one that not many have published on. Fortunately during my tenure at BJPS the number of people working on, say, philosophy of neuroscience (to choose just one striking example) has positively exploded, so I now have many more options.
I don’t think any of this is particularly mysterious, but in case you’ve received a request to referee from me, this is the process that eventuated in that form email being sent. And thanks to all who referee for their service!
One option would be reducing the pressure to publish by valuing teaching more in hiring and promotion decisions. It’s far from obvious that we serve best by insisting that everyone publishes regularly.
Another option would be to reduce pressure to publish in academic venues by increasing the weight we give to work published in public venues. It’s far from obvious that we serve best by focussing to heavily on speaking to one another.
Skef, was with you but then surprised the conclusion wasn’t “we need to stop promoting so many PhD students”. The sheer number of people hoping to enter the field as full-time professionals and under pressure to publish a lot and quickly seems to me to be the locus of a lot of problems (see upcoming strikes in U.K. for example – is anyone surprised the universities are taking advantage of massive supply). Merely anecdotal, but I’ve heard of an open junior job this year getting 650 apps. I know I’m under pressure to recruit more PhDs and have been at former institutions too. They are good for a university’s research profile – Carnegie, REF, etc. Might we be spinning our wheels about a symptom?
My observations were about the reality of the situation. There are as many PhDs as there because of how University departments view themselves and because overproduction is ultimately a benefit to the higher-ups: that’s where the super-cheap adjuncts come from, as you note.
Tenured faculty see this dynamic as out of their control and responsibility because of the coordination problems involved in fixing it. And for the most part it creates advantages for them rather than problems anyway. The glaring exception is the expectation that the work all these graduate students produce will be evaluated. No one wants to read this stuff. Journal editors talk about needing to desk reject because of “graduate-seminar-level papers” but insist that judging submissions by an explicit standard is out of the question. They claim this is because of author-hostility but it’s ultimately a capacity problem — to judge submissions by a standard you would have to read them and there’s no time for that.
Look at this this way: As long as the people expected to read this stuff can get away with not doing so, what is the problem *for them*? It boils down to a PR problem. So everyone makes a suggestion that would put the work on someone else’s plate and gets on with their day. There are many, many more people who want to look like they care about graduate student lives than those who would give up even a small personal advantage to improve the situation.
This comment is about multiple submissions, not about the other aspects proposed:
Maybe I’m idiosyncratic, but I find refereeing papers to be time-consuming work. I would be far less inclined to do the work for multiple-submission journals, where I would suspect (i.e., be almost certain) that my work would be superfluous. So my guess is that a shift to allowing multiple-submissions would a) drastically increase the numbers of requests for refereeing and b) somewhat reduce the rate of acceptances.
Law reviews are different, of course, because:
1) they have an endless pool of free labor (look at all the names on the NYU Law Review masthead, https://www.nyulawreview.org/about/mastheads/);
2) refereeing is a serious résumé item for their referees (i.e., to be on law review is to be more or less obligated to referee when asked, and being on law review is very, very helpful; contrast no one is impressed by the list of refereeing roles I’ve filled at the tail end of my cv); and
3) their referees have little if any substantive expertise upon which to evaluate submissions so can evaluate lickety split.
Maybe I misunderstand some bit of the dynamic? But if I’m right, this seems to be a near-on conclusive argument against allowing multiple submissions.