“What do we lose if LLMs kill the journal system? Nothing but some heavy ass chains, and a few weak heuristics for assessing CVs quickly that were never worth the outrageous amounts of time and labour the journal system wasted anyway.”
.
That’s Jonathan Weisberg (Toronto), who knows a little bit more than the average philosopher about the journal system, having co-founded Ergo.
With AI and “a little human guidance,” he says,
you can crank out the kind of papers that make up 95% of what we publish in a few hours. Soon (already?), human guidance may not even be necessary. Do I find this depressing? Just the opposite. Sure, I built my career on this skill, honed it for decades, and now a computer can do it in a hundredth of the time... But this was always a goofy way of making a living…
[I]t’s always been a drag dealing with bafflingly slow journals, maddeningly lazy referees, and typesetters so incompetent it must be a bit. The publication process puts years of frustrated waiting, and dozens of hours of wasted labour, between when I’m actually done understanding an idea, and the point where my dean finally gives me a pat on the head and a sticker that says “good job!”.
Many see the AI onslaught as a net negative for academia, but Weisberg sees it as an opportunity:
just tell the dean, “sorry, LLMs.” They’ll find some other way to count our beans, I have no doubt. But while they work on figuring that out, we have an opportunity to radically revise the whole bean economy.
If the journals crash and burn, you can still read, think, and write about philosophy. You can still share your writing with other philosophers, build on what they’ve done, hope they’ll do the same in turn, etc. None of this requires publishers, typesetters, or even traditional peer review.
There are better things to do with one’s time than “agonizing over how to revise your manuscript so the referees at the fourth journal you send it to won’t be able to miss the reply to their objection buried somewhere in the middle of page 23.”
So what should replace the current system?
Note that it’s not just that the deans (etc.) who would need a new way to “count our beans.” Being able to assess skills and achievements is important in any academic discipline for a variety of reasons (hiring, promotions, grants, collaborations, attention), and it’s generally good to have more than one heuristic for doing that. (We don’t want to rely solely on, say, inherited institutional prestige.)
On x.com, Wes Pegden, a mathematician at Carnegie Mellon, suggests that “deep talks” could be part of “a new system for how we identify ‘awesomeness’ in math” (h/t Henry Shevlin). What is a deep talk?
This is an extended format talk given on a paper (regardless of authorship) accompanied by the claim that the presenter is prepared to reconstruct any desired detail of paper’s proof on the spot, unaided. For example, currently there is a document that contains the solution to the Navier-Stokes problem. No person currently could give a “Deep Talk” on this paper. But certainly, I expect it is within human capability to do so. Someone can be the first! So impressive!
A mathematician would have a repertoire of Deep Talks they currently offer; departments could invite speakers offering talks of new results of interest (sometimes for longer “Deep Talks”, with the detail-probing component, sometimes normal seminar talks). What this system offers is a way to measure technical excellence—impressiveness—in our field. (The present discussion is not about the value to society about having mathematics as an academic field, which I strongly believe in but which is a separate topic).
Pegden adds, “Having such a measure is vital to having a thriving academic discipline. In particular, we need to have a way of measuring excellence in a way roughly aligned with what we recognize as mathematical capabilities.” If we replaced “mathematical” with “philosophical” in that statement (with a suitably pluralistic understanding of what counts as “philosophical capabilities”), I think most philosophers would endorse it. But I’m not sure that they’d agree that “deep talks” would do the trick. What would?
At the moment, to my knowledge, AIs are not yet producing excellent philosophical work, at least not in the form of articles or books. (And it may turn out that articles and books are not the best formats for showcasing the philosophical capabilities of AI. I suspect that complex, customizable dynamic inference mapping will be among their distinctive excellences, and the best format for that may be a tool or application, not an article or book.) Still, many people believe AIs eventually will produce excellent philosophical work. If they do, what then should be the currency of achievement in human philosophy?
Yeah but you could just get fed what to say in a Deep Talk in a wireless mic or displayed on smart glasses where the source is just an LLM feeding this to you. The deep talk would need to occur in a Faraday Cage.
Local AI operating on a nearby laptop within the cage could defeat this. What you really need is to be in a sauna, with the heat cranked so high as to defeat all devices.
95%? Really? I personally look at my publications as achievements even if, in the grand scheme of things, they are nothing special. And perhaps those in the publication factory do indeed publish a paper every second week having honed their skill to do so, I personally don’t. Probably because I am more mediocre but also because I have less time, or just simply like to spend more time on my work. I don’t see why I should welcome the great AI-inspired advance in the publishing system, (This is not to say I like the system but I also don’t see why and how AI will solve its problems, other than simply eliminating it. This seems like a classic case of throwing the baby out with the bathwater,).
And as for the deans and the opportunity given to us in the alleged transition stage, I hazard the guess that the deans, unless they re genuinely into high-quality human research, will just fire us and rehire us teachers only (assuming now our students will still want to be taught by humans, not an obvious assumption). Why bother having us round if AI with some help can do almost all of the research we’ve done so far? This just proves that we were not needed in the first place.
I agree with most of this, especially the Deep Talks idea. The current journal system seems pretty broken in terms of speed, access, reliance on unpaid labour, and obscurity to outsiders. I expect AI to put it under still greater strain over this decade as LLMs become capable of contributing philosophy at a very high level. I realise some will doubt that prospect, but it seems very likely to me – Eric Schwitzgebel and I bet on this back in 2023 – most obviously in fields close to mathematics, like logic and formal decision theory.
That said, I still see a place, at least in the near term, for anthologies and special issues with invited contributors. These can be a very good way to kickstart collaborative research, especially when contributors read and respond to one another’s work. More personally, I’ve found commissioned pieces useful as a form of pre-commitment (having promised someone a paper by a deadline, I’m more likely to actually deliver it rather than sit on it endlessly fussing). And inviting contributors avoids the LLM-slop cannon of 500 mediocre submissions.
The obvious downside is that invitation-based formats favour established researchers and people the editors already know. Adding an open call brings back many of the problems facing regular journals. Perhaps reserving some parts of a special issue for early-career researchers could help, though those opportunities would also need to reach people outside existing networks.
I’ve no idea whether this is how things will play out, but I suspect we’ll see some movement in this direction. If so, my advice to philosophy graduate students would be to become known for a niche topic early, while developing their presentation skills and, to some extent, a personal brand. I realise that’s evergreen advice, but it becomes more pressing if publication record declines in importance or value as a metric. Being able to explain your ideas in engaging and memorable ways, defend them in conversation, and help other people develop theirs may count for more.
On the upside, there are plenty of unexplored questions right now, especially at the intersection of AI, mind, ethics, society, and political theory. There’s also, frankly, a lot of low-hanging fruit in how philosophy is communicated and presented. That’s partly because existing standards are often low, and partly because it’s easier than ever to experiment with new forms, eg interactive video games and short films on philosophical themes.
The following can be true at once:
1. Very little written philosophy is of “history making” importance – even in the small sense of ‘history making’
2. Humans doing philosophy is intrinsically valuable
3. We have yet to see a piece of computer generated philosophy that is ‘history making’ in even the small sense
That being said, if LLMs were really doing philosophy I think Wittgenstein is right that we wouldn’t be able to understand it. Andrew Gakh, a chemist and fan of LLMs in chemistry research, described “large world models” as “essentially forms of “alien intelligence” and are conceptually incompatible with human reasoning.” What we’re asking the current generation of LLMs to do is to play act at doing human philosophy but I think the more interesting idea is to see what sort of philosophy LLMs generate when they do their own philosophy (much as Gakh describes large world models as the representational structure of artificial neural networks unchained from pre-existing human frameworks).
There’s a lot of low-hanging AI fruit and, in my more optimistic moments, I think that maybe we can turn over salami-sliced mediocre philosophical work to LLMs and focus again on big ideas. In other DN topics, I’ve said that the talk might return as the coin of the realm in an age of easy AI-philosophy and that’s probably going to be true – for a while. I have about 15 years before I retire but I predict that we’ll see a radical restructuring of academic humanities labor where ever-advanced AIs can engage students about as well as most of us can in real-time using Socratic methods to do what most of us do in most of our classes (I say most of our classes because most of our teaching, perhaps discounting very advanced undergraduate work and graduate-level teaching). This will make most of our teaching redundant and leave most of our papers lost in a sea that makes the current paper-bloat look quaint. I suspect that there will be fewer and fewer people doing “real” philosophy and even fewer teaching it. It’s not all bad though. I suspect that those who survive will be doing philosophy more freely and creatively than we’re allowed to do now by the pressures of the publish-or-perish system. Those lucky few may be able to do some really neat philosophy…but maybe 1% of will get to do that.