“What do we lose if LLMs kill the journal system? Nothing but some heavy ass chains, and a few weak heuristics for assessing CVs quickly that were never worth the outrageous amounts of time and labour the journal system wasted anyway.”
.
That’s Jonathan Weisberg (Toronto), who knows a little bit more than the average philosopher about the journal system, having co-founded Ergo.
With AI and “a little human guidance,” he says,
you can crank out the kind of papers that make up 95% of what we publish in a few hours. Soon (already?), human guidance may not even be necessary. Do I find this depressing? Just the opposite. Sure, I built my career on this skill, honed it for decades, and now a computer can do it in a hundredth of the time... But this was always a goofy way of making a living…
[I]t’s always been a drag dealing with bafflingly slow journals, maddeningly lazy referees, and typesetters so incompetent it must be a bit. The publication process puts years of frustrated waiting, and dozens of hours of wasted labour, between when I’m actually done understanding an idea, and the point where my dean finally gives me a pat on the head and a sticker that says “good job!”.
Many see the AI onslaught as a net negative for academia, but Weisberg sees it as an opportunity:
just tell the dean, “sorry, LLMs.” They’ll find some other way to count our beans, I have no doubt. But while they work on figuring that out, we have an opportunity to radically revise the whole bean economy.
If the journals crash and burn, you can still read, think, and write about philosophy. You can still share your writing with other philosophers, build on what they’ve done, hope they’ll do the same in turn, etc. None of this requires publishers, typesetters, or even traditional peer review.
There are better things to do with one’s time than “agonizing over how to revise your manuscript so the referees at the fourth journal you send it to won’t be able to miss the reply to their objection buried somewhere in the middle of page 23.”
So what should replace the current system?
Note that it’s not just that the deans (etc.) who would need a new way to “count our beans.” Being able to assess skills and achievements is important in any academic discipline for a variety of reasons (hiring, promotions, grants, collaborations, attention), and it’s generally good to have more than one heuristic for doing that. (We don’t want to rely solely on, say, inherited institutional prestige.)
On x.com, Wes Pegden, a mathematician at Carnegie Mellon, suggests that “deep talks” could be part of “a new system for how we identify ‘awesomeness’ in math” (h/t Henry Shevlin). What is a deep talk?
This is an extended format talk given on a paper (regardless of authorship) accompanied by the claim that the presenter is prepared to reconstruct any desired detail of paper’s proof on the spot, unaided. For example, currently there is a document that contains the solution to the Navier-Stokes problem. No person currently could give a “Deep Talk” on this paper. But certainly, I expect it is within human capability to do so. Someone can be the first! So impressive!
A mathematician would have a repertoire of Deep Talks they currently offer; departments could invite speakers offering talks of new results of interest (sometimes for longer “Deep Talks”, with the detail-probing component, sometimes normal seminar talks). What this system offers is a way to measure technical excellence—impressiveness—in our field. (The present discussion is not about the value to society about having mathematics as an academic field, which I strongly believe in but which is a separate topic).
Pegden adds, “Having such a measure is vital to having a thriving academic discipline. In particular, we need to have a way of measuring excellence in a way roughly aligned with what we recognize as mathematical capabilities.” If we replaced “mathematical” with “philosophical” in that statement (with a suitably pluralistic understanding of what counts as “philosophical capabilities”), I think most philosophers would endorse it. But I’m not sure that they’d agree that “deep talks” would do the trick. What would?
At the moment, to my knowledge, AIs are not yet producing excellent philosophical work, at least not in the form of articles or books. (And it may turn out that articles and books are not the best formats for showcasing the philosophical capabilities of AI. I suspect that complex, customizable dynamic inference mapping will be among their distinctive excellences, and the best format for that may be a tool or application, not an article or book.) Still, many people believe AIs eventually will produce excellent philosophical work. If they do, what then should be the currency of achievement in human philosophy?
Yeah but you could just get fed what to say in a Deep Talk in a wireless mic or displayed on smart glasses where the source is just an LLM feeding this to you. The deep talk would need to occur in a Faraday Cage.
Local AI operating on a nearby laptop within the cage could defeat this. What you really need is to be in a sauna, with the heat cranked so high as to defeat all devices.
That wouldn’t stop me from delivering a neuralink-backed sauna deep talk (sponsored by the Neuralink Corporation (TM)).
If it’s really true that the AI could cue someone without the skills to give a Deep Talk of this sort, then we’re far beyond the situation we’ve been talking about. At that point, it seems plausible that we really *shouldn’t* care who is good at it, since everyone can just get a Deep Talk fed to them whenever they feel like it.
I think the question is, what do we do to identify the kinds of human skills that we actually care about, where those human skills actually provide some value to the community.
95%? Really? I personally look at my publications as achievements even if, in the grand scheme of things, they are nothing special. And perhaps those in the publication factory do indeed publish a paper every second week having honed their skill to do so, I personally don’t. Probably because I am more mediocre but also because I have less time, or just simply like to spend more time on my work. I don’t see why I should welcome the great AI-inspired advance in the publishing system, (This is not to say I like the system but I also don’t see why and how AI will solve its problems, other than simply eliminating it. This seems like a classic case of throwing the baby out with the bathwater,).
And as for the deans and the opportunity given to us in the alleged transition stage, I hazard the guess that the deans, unless they re genuinely into high-quality human research, will just fire us and rehire us teachers only (assuming now our students will still want to be taught by humans, not an obvious assumption). Why bother having us round if AI with some help can do almost all of the research we’ve done so far? This just proves that we were not needed in the first place.
If the dean identifies “research” with “journal publications”, then sure. But most deans are actually academics, and understand that the journal publication is a particular public-facing part of the research, but is not identical to the research itself.
I still think that this prediction (or, shall we say, hope?) is way too optimistic. If universities weren’t run as businesses, I might even believe it, but not in the present state of things.
I also think that many of these more optimistic predictions concerning the effects of AI leave the institutional system in which all this is happening out of the picture. I think this is a major mistake.
Having said this, I very much hope I am wrong.
I agree with most of this, especially the Deep Talks idea. The current journal system seems pretty broken in terms of speed, access, reliance on unpaid labour, and obscurity to outsiders. I expect AI to put it under still greater strain over this decade as LLMs become capable of contributing philosophy at a very high level. I realise some will doubt that prospect, but it seems very likely to me – Eric Schwitzgebel and I bet on this back in 2023 – most obviously in fields close to mathematics, like logic and formal decision theory.
That said, I still see a place, at least in the near term, for anthologies and special issues with invited contributors. These can be a very good way to kickstart collaborative research, especially when contributors read and respond to one another’s work. More personally, I’ve found commissioned pieces useful as a form of pre-commitment (having promised someone a paper by a deadline, I’m more likely to actually deliver it rather than sit on it endlessly fussing). And inviting contributors avoids the LLM-slop cannon of 500 mediocre submissions.
The obvious downside is that invitation-based formats favour established researchers and people the editors already know. Adding an open call brings back many of the problems facing regular journals. Perhaps reserving some parts of a special issue for early-career researchers could help, though those opportunities would also need to reach people outside existing networks.
I’ve no idea whether this is how things will play out, but I suspect we’ll see some movement in this direction. If so, my advice to philosophy graduate students would be to become known for a niche topic early, while developing their presentation skills and, to some extent, a personal brand. I realise that’s evergreen advice, but it becomes more pressing if publication record declines in importance or value as a metric. Being able to explain your ideas in engaging and memorable ways, defend them in conversation, and help other people develop theirs may count for more.
On the upside, there are plenty of unexplored questions right now, especially at the intersection of AI, mind, ethics, society, and political theory. There’s also, frankly, a lot of low-hanging fruit in how philosophy is communicated and presented. That’s partly because existing standards are often low, and partly because it’s easier than ever to experiment with new forms, eg interactive video games and short films on philosophical themes.
Do you have any info or write up on the bet you guys made or would you relate the details? I’m interested.
It was back in May 2023, we were walking in Riverside, and I was making the case that by the end of the decade AI would be able to make significant contributions to philosophy, equivalent to the work of Chalmers, Williamson, Korsgaard, Brandom, etc.. Eric was very sceptical; he thought we wouldn’t see a single AI authored paper of that calibre. So the terms of the bet are something like “not a single paper by LLMs at the level of the very best human philosophy in a typical year”, with donation to a charity of the other’s choice as the prize. Terms obviously a bit loose but we’re reasonable people so we’d defer to a mutual friend if it wasn’t clear!
Oh, and with end-2030 as the specific resolution date!
Interesting and thanks! How are you feeling about your prospects?
Pretty confident! I think it’s likely that AI will dominate in all analytic philosophy domains by 2030, but the terms of the bet are unreasonably charitable to me insofar as I think I’d qualify as winning if AI was producing world class work even just in more technical domains (e.g., formal epistemology, decision theory, logic) where it seems especially well suited. The NUS philosophy writing contest will be a nice dry run.
The following can be true at once:
1. Very little written philosophy is of “history making” importance – even in the small sense of ‘history making’
2. Humans doing philosophy is intrinsically valuable
3. We have yet to see a piece of computer generated philosophy that is ‘history making’ in even the small sense
That being said, if LLMs were really doing philosophy I think Wittgenstein is right that we wouldn’t be able to understand it. Andrew Gakh, a chemist and fan of LLMs in chemistry research, described “large world models” as “essentially forms of “alien intelligence” and are conceptually incompatible with human reasoning.” What we’re asking the current generation of LLMs to do is to play act at doing human philosophy but I think the more interesting idea is to see what sort of philosophy LLMs generate when they do their own philosophy (much as Gakh describes large world models as the representational structure of artificial neural networks unchained from pre-existing human frameworks).
There’s a lot of low-hanging AI fruit and, in my more optimistic moments, I think that maybe we can turn over salami-sliced mediocre philosophical work to LLMs and focus again on big ideas. In other DN topics, I’ve said that the talk might return as the coin of the realm in an age of easy AI-philosophy and that’s probably going to be true – for a while. I have about 15 years before I retire but I predict that we’ll see a radical restructuring of academic humanities labor where ever-advanced AIs can engage students about as well as most of us can in real-time using Socratic methods to do what most of us do in most of our classes (I say most of our classes because most of our teaching, perhaps discounting very advanced undergraduate work and graduate-level teaching). This will make most of our teaching redundant and leave most of our papers lost in a sea that makes the current paper-bloat look quaint. I suspect that there will be fewer and fewer people doing “real” philosophy and even fewer teaching it. It’s not all bad though. I suspect that those who survive will be doing philosophy more freely and creatively than we’re allowed to do now by the pressures of the publish-or-perish system. Those lucky few may be able to do some really neat philosophy…but maybe 1% of will get to do that.
Yes, but assuming now that those who today ‘do’ philosophy, do it because they like it, despite the pressures, the incentives system, the publish and perish, the impossible job market and so on and so forth, it does seem like that the net result will be many thousands of people not getting to do what they like, what, I dare say, gives meaning to their lives. Why is this a good result?
Or is the idea that all those by then jobless philosophers will get the chance to do something they equally like to do, something that equally fills their life with meaning? This would be a rather optimistic prediction (or, rather, hope).
I honestly don’t see how we could come out of this well as a society, or as a profession, for that matter.
Why are 90% of posts about AI as of late?
Yeah, it’s tiresome.
I think because 90% of the events and discussions that are affecting philosophy as of late are related to AI.
Hmm… Sounds like a self-fulfilling prophecy
A computer can autonomously do philosophy now. Is that not a significant thing?
No, it can’t on any serious definition of “do philosophy.”
There is plenty going on in our profession unrelated to tech fads.
Well even if it was significant, we had twenty other posts discussing a variation of this theme in the past week, so maybe we could move on
I imagine a lot of philosophers may wish to analyze the “autonomy” involved, too.
I don’t understand. When it comes to social justice, we can disregard all established human institutions. But, when it comes to technology, we just have to accept what four tech bros decided two days ago? Makes no sense to me.
One makes people rich and the other doesn’t…age-old story. Follow the money.
“Do I find this depressing? Just the opposite. Sure, I built my career on this skill, honed it for decades, and now a computer can do it in a hundredth of the time… But this was always a goofy way of making a living…”
I’m glad that this all worked out for a tenured professor. I’m sure it will work out well for me, a junior untenured scholar whose only means of upward academic mobility has been, and likely would have continued to be, publications. Chapeau!
Don’t worry mate, it’s been a goofy way of making a living anyway!
Just a few CV heuristics, we’re such dummies!
I’m not sure I understand the complaint here.
The one that makes the most sense to me is the complaint that people at a particular career stage – post-PhD but pre-tenure – have been encouraged to practice a particular skill, but a new skill will be relevant going forward, so there is a challenge for this particular cohort if we change the skill that matters. People who are tenured will have protection while they learn the new skill, and people who haven’t yet learned the old skill can just learn the new one, but the people who already learned the old one but don’t have protection yet are at a loss. Is that a correct understanding of the complaint?
We should definitely be doing something to properly support people at this career stage in making whatever transition becomes necessary. But I don’t think we should just accept that peer-reviewed publications in journals should remain the currency of the profession for eternity just because it was for the past few decades and any transition is going to have a generation that gets hit.
I don’t understand your comment. You aptly described the situation I’m lamenting (as sort of “cosmically unfair,” with no one particular to blame). But then you warn against concluding something—publications should remain the currency of our profession forever—which I did not (and would not) recommend.
If anything at all is being recommended by my comment (beyond a mere expression of grief), it’s perhaps that those who will remain mostly unharmed by these changes might consider being a little less casual when dismissing the practices we’re trained to follow (and invest hope in). But I actually wouldn’t even go that far, because these are just words after all. The grief nevertheless remains.
I think I understand the complaint. Some people seem glib and or even jolly about something that, if true, is going to screw a lot of people over. The people being screwed over are complaining about that attitude. And I think they’re right to do so.
I’ll just add that this is true: “We should definitely be doing something to properly support people at this career stage in making whatever transition becomes necessary”
I’m also very pessimistic that “we” are going to do that. “We” don’t offer much support to the folks right now who are struggling transition to permanent positions, and I think that we’d only be adding more to that set of people with this transition. So the people with the support get their heavy chains lifted, but it’s different for other folks.
My thoughts exactly.
And who, pray tell, gets invited to give a “deep talk” in the first place? Since finishing my PhD, I’ve managed to publish a few books (with more coming) and several dozen papers. I have only been asked to give a colloquium talk three times, though, and for a great many people I know, that number is zero. I guess we could blend in conference talks, but… That just doesn’t seem like a relevantly similar sort of research output. I don’t think it’s a desirable direction to pursue.
I have yet to see evidence that a humAIn centaur’s philosophical work is as good as a respectably good human’s. Until then, I just don’t buy the claim in the first place.
Besides which, I have absolutely no interest in reading humAIn work. I read it all the time in emails from students and admin, as well as student work. I see it everywhere online. That’s more than enough for my taste.
The other way to get around this is to have the journals (and the whole field) rethink the criteria of philosophical excellence. At some point, AI can definitely write up sharp critiques of existing views and extend existing theories to some extent. These are the kinds of work that dominate journals including the top ones, since those are hard to nitpick.
But we still have little evidence that AI can ever intelligibly introduce and defend big ideas that are truly original. Perhaps human philosophizing effort should be shifted toward attempting to achieve that; and journals should learn to weigh originality and potential groundbreakingness much more than they do now.
Nothing is going to change. Just more of the same. More publications and less quality work. Standards will go up (now asking for more publications) but they are not going to change (to something more organic or faithful to “real skill”). A lot of “systems” have been broken for a while, supposing they actually worked in the first place, we will just go along pretending they work, like we have been doing until now. But there are indeed new worries worth thinking about, like the fact that AI seems to assess AI driven-work more highly than human produced work (now called “AI bias”). That is so dystopian.
“… AI seems to assess AI driven-work more highly than human produced work (now called “AI bias”).” .Interesting. Can you point to any discussions/measurements of this?
Here you go:
“AI–AI bias: Large language models favor communications generated by large language models”
https://www.pnas.org/doi/10.1073/pnas.2415697122
Thanks!
If AI really does start to do philosophy, then why are we talking about papers? Papers will be obsolete. Philosophical artifacts will be AIs themselves, built and trained by philosophers for the purpose of doing philosophy. Our jobs will be to imbue AIs with philosophical excellence.
Make Oracles Great Again (M.O.G.A.)! 🧘
If you’re not using entrails, you’re doing it wrong.
Ouji boards work too, I heard.
Only if you use entrails.
Deciding what should be the currency of achievement in philosophy requires deciding what the professional point of our work is. We don’t have agreement on this point. I assume that we can at least mostly agree that we do not, in general, provide final answers to philosophical questions.
Anything that radically changes the horrible journal system will probably come up with something better. Good riddance.
I wonder if AI-generated philosophy will NOT produce the kind of ordinary philosophy we humans produce. Almost everything we produce now has major weaknesses somewhere. Maybe future AI work will just skip all that low-quality stuff and coalesce around a small number of ideas and arguments that are quite different and superior. For instance, almost all human philosophy contains arguments that are either invalid or indeterminate in validity because of lack of specificity about quantifiers, domains, modalities, scopes, and so forth (plus unarticulated but dispositionally present antecedents).
Maybe when philosophy gets REALLY good—far better than what we have come up with—that kind of weakness disappears. Asking 100 advanced AI tools to say something about material composition, say, might end up generating 100 essays saying pretty much the same thing–sort of like how asking AI tools now about the hydrogen atom produces pretty much the same stuff every time. Obviously, none of us have much of a clue as to what AI-generated philosophy will be like in 5 years.
What a depressing view of the profession, and I strongly disagree with it. I don’t actually think salami slicing is all that common in philosophy. I take pride in my work and most of the philosophers I know take pride in theirs.
Also, I use the AI models every day and I don’t think it is close to correct that 95% of published work could be done by current AI (with or without a little human guidance).
There are blogosphere posts about LLMs basically every single day, often multiple, and as far as I can tell almost all of them have the unquestioned starting premise that “It is profitable and financially sustainable to provide LLM services of this kind & at this scale, and this situation will only improve.” Why does anyone even believe that to begin with? There is no evidence for it, and lots of evidence against it.
I can certainly see the case that, if AI progress slows or stalls, that the major companies won’t be able to service their debt and will go bankrupt. But the models will still exist and have been trained, and someone will acquire ownership of their weights, and cost is fairly linear with compute. Why wouldn’t there be a market-clearing price for AI provision at that point? It wouldn’t be the first bubble to create lots of usable infrastructure even though the original providers went under. If there’s “lots of evidence” against that, I’d be interested to see it.
The AI companies are taking out lots of debt to build new data centers. But the revenue they are bringing in right now is more than enough to pay for the operations of the existing data centers. If they can’t build more data centers, then they may raise prices to keep demand down to the level they can serve with existing data centers (and make huge profits as a result) but it seems more likely that they’ll have huge profit on capped services (that many of us won’t be able to afford any more) than that they’ll end up being unprofitable.
The revenues being reported are annualized–in other words, pure fantasy. According to Ed Zitron’s reporting, in 2024, OpenAI had $12B in costs and almost $4B in revenue. In 2025, $13B in revenue and $34B in costs, plus almost $21B in losses. There’s a fair bit of circular financing and moving revenues and losses around going on, so it’s not super clear exactly how much the extant infrastructure and its use cost, but it seems pretty clear that this is a business model closer to airlines than anything else–in other words, the base costs are very high and only turn much of a profit when the planes (so to speak) are full and running a few high-demand routes and passengers are squeezed dry.
One worry I have is that this will exacerbate prestige bias (which is already pretty rife). As someone at an institute that does not have great prestige, it’s always been reassuring to me that my ability publish in a good journal has ‘cut through’ – at least if it’s properly blind reviewed. If the metric is ‘brands’, niches, and talks (as has been suggested in the post/ comments), I worry that those of us without a prestige institution behind us will be less able to cut through.
According to OpenAI our smart vacuum cleaners will kill us all in a few months time, so who cares about publications. Let them come! (Remember the lady on the rooftop in Independence Day?)
I disagree with Jonathan Weisberg, though not about his prediction that AI will make journals and the current peer review system unsustainable. What I disagree with is his reassurance that, even so, we can go on doing philosophy much as we do now: reading, thinking, writing, and sharing our work with colleagues.
In one sense, of course, we could still do philosophy there. Suppose AI becomes advanced enough to answer any philosophical question I care about, and reliable enough that deferring to it would be reasonable. I could still read, think, and write about philosophy as a personal pastime. But that would no longer be philosophy as the production of knowledge. It would be philosophy as a hobby, less a serious discipline or form of inquiry than an elaborate sudoku: a very hard problem whose answer already exists, which I work through in my own way anyway and then check against the key.
This isn’t bad in itself. But for some of us, myself included, it would be tragic. Weisberg says he came to philosophy because he loved learning and wanted to understand things. For that, AI may take little away. I like reading, writing, and talking about philosophy too, but what I love even more is exploring, on my own, intellectual territory no one has explored before. I enjoy reading good philosophy papers, but I enjoy being a producer of philosophical knowledge more. AI that can do good philosophy on its own would not only bring down the current journal system; it would also take away our role as producers of knowledge. As with checking a sudoku against the answer key, all I could do would be to work through hard problems that already have answers and compare my results with them. So while part of me would be glad to finally have answers to the philosophical questions I care about, a larger part of me would be sad that seeking them would no longer be an expedition into uncharted intellectual territory.
I love doing philosophy, but I also love being a philosopher by profession. I want to be not merely someone who enjoys philosophy, but someone who lives by it. I’m not saying professional philosophy is inherently more valuable. I simply love, as I think many others do, the life we now have, in which we not only do philosophy but do it professionally (though, as many have pointed out, a good number of current practices make professional philosophy miserable). What saddens me is that AI will, at best, leave me able to do philosophy only as a hobby.
Here’s a quaint notion: philosophy pursued so as to become wiser, where this wisdom cannot be gained by an LLM feeding you a preprocessed “answer” to a question or problem. A philosophy whose main purpose is a deep insight gained through a combination of reflection and discussion with like-minded lovers of wisdom, whose goal is self-betterment, harking back to Classical and (some schools of) Hellenistic philosophy in the Western tradition or large swaths of, e.g., the Indian and the Chinese philosophical traditions. “But then, why pursue a PhD in philosophy?” — to do it seriously and to have a chance to get a job where you get to keep doing it for a living. “But how will we preserve the hierarchy of prestige and reputation?” — Left for your imagination.
“But how will we paying for humans to do philosophy research, if machines can do it better for less money?” – TBD.
(I’m not convinced machines will be able to do it better, but if they can, the question isn’t “would humans still benefit themselves from philosophy research” but “is there any broader public benefit to that research?”.