A pair of philosophers have launched a contest in which participants submit essays that are “primarily AI-generated”. The aim of the competition is to “gauge the philosophical capabilities of AI in a way transparent to the philosophical community”.
The team behind the contest, Elliott Thornley and Zachary Goodsell of National University of Singapore, have assembled a group of philosophers to serve as judges, and arranged funding for prizes ($3000 for first prize, along with other prizes).
Here’s the announcement they sent along:
AI is now exceptionally capable in mathematics and coding, but how good is it at philosophy? We are organising the first AI Philosophy Competition to find out.
Entrants may submit up to three original essays on any philosophical topic. Essays must be primarily AI-generated, though some human guidance is allowed. Entrants must also submit a methodology report explaining how each essay was produced.
The essays will be judged by a committee of philosophers including David Chalmers, John Hawthorne, Matthew Mandelkern, Rachel Sterken, Herman Cappelen, Branden Fitelson, Kenny Easwaran, Daniel Greco, Cian Dorr, Jake Nebel, and Christian Tarsney, with more to be announced.
The prizes are $3,000 for first place, $2,000 for second, and $1,000 for third. There is also a prize pool of $5,000 for creative methodologies, to be distributed as the judge committee sees fit.
The submission deadline is 31 October 2026. Results will be announced at the end of January. The competition is organised by Zachary Goodsell and Elliott Thornley, and is funded by Forethought Research. Full details are available on the competition website.
I understand that some readers may take my sharing of this announcement as some kind of signal that I am in favor of AI-written philosophy. At this point, I am not. While I think some uses of AI at the research stage could be acceptable and perhaps, eventually, advisable, for reasons I will explain some other time, I think there is something quite wrong-headed about primarily AI-authored philosophy.
Nonetheless, I do think it is valuable to know what this technology is capable of, and so I’m sharing news of this competition. I do so even though there’s a risk that events such as this accelerate the arrival of a state of affairs I currently do not favor, for I doubt that withholding news about them would do much, if anything, to slow that down.
Surely a competition in which all judges believe it likely that all work is largely AI-produced is not a good way of determining the quality of AI-produced philosophy. Unlike maths and coding, there are fewer objective and widely accepted criteria for good philosophy. It certainly is not a good way of measuring the quality of AI-produced vs human-produced philosophy, which I would imagine is the more important measure in the vicinity.
I’ll also add that the website notes they may need to use AI screening if submission volume is high. That should ring methodological alarm bells.
Thanks for the feedback Mathieu. On your first point, the judges will tell us their estimation of how the AI work stacks up against human work, which they are of course very familiar with. Of course the standards for philosophy are difficult to operationalise. In fact so are the standards of mathematics (e.g., what is a “significant” mathematical problem?). Still, competent philosophers are generally thought to be competent to judge the quality of philosophy.
On your point about AI screening. We see no reasonable alternative, since undoubtedly many submissions will be of very low quality, and since the barrier for entry is relatively low we expect the submission volume to be high. We are open to suggestions though!
Hello Zach, thanks for responding.
I agree that there are nuances in the mathematical case, and of course competent philosophers are good judges of quality. But if the ranking is going to work via an “estimation of how the AI work stacks up against human work”, then the threat of bias (for or against AI) is surely considerable, at least given the framing of the competition as a step towards a more accurate appraisal of AI’s philosophical abilities.
On the screening issue, an alternative might be to randomly select some submissions. That will not guarantee giving you the very best, but neither will AI screening, and the latter might introduce biases.
The competition says it’s testing how good AI is at philosophy. But the rules let entrants generate as many essays as they like and submit the one they judge best. Picking the best from a pile of two hundred is itself philosophical work, so what gets measured is what a philosopher can get out of a model – worth knowing maybe, but a very different question.
The rules say “Essays must be primarily AI-generated, though some human guidance is allowed. Entrants must also submit a methodology report explaining how each essay was produced.”
The idea isn’t to see what single prompt directly produces the best output – it’s to see how a *human* can use AI most effectively to produce a philosophy paper. We will learn something regardless of whether the winning methodology turns out to be to have a team of agents looking up papers, proposing ideas, shooting them down, and eventually writing up a paper after tens of thousands of pages of arguments and drafts, or whether the winning methodology turns out to be to give thousands of separate agents a single prompt and select the best resulting paper (though presumably some sort of interesting work would need to be done to identify good candidates from that slush pile).
It would certainly be ironic if the winner turns out to have cheated by submitting an entirely human-written essay (with a fake methodology report).
Hi Kenny, more than ironic, it’d be a good and necessary control for the experiment to slip in some human-written essays that would be widely considered to be “good philosophy.” These would have to be unknown to the referees.
Every good experiment should have a control group.
I agree that good experiments should have controls but where does it say it’s an experiment? It’s a competition. Essay competitions don’t have controls. Your control is 99.9% of published philosophy. You know the parameters of this contest. Presumably we’ll learn about different methodologies. We’ll learn something. Exactly what I’m not sure but if the best papers are disappointing, then we’ll have learned that it’s not easy to produce good philosophy with AI.
Right, they don’t say it’s an “experiment” per se, but that doesn’t mean it’s not. It’s a first-of-its-kind contest, right? That makes it experimental, esp. if it hadn’t been thoughtfully designed to avoid or address all the obvious issues raised in comments here.
Experiment also doesn’t need to mean “formal experiment.” And even everyday, casual experiments require a control to be meaningful.
But now that you mention it and as an aside, it’s possible someone could take this as a formal research activity that’s obligated to follow best practices in research design, esp. as it emanates from academics at a university.
If the organizers publish generalizable research from this (e.g., “AI can write good philosophy”) and it may involve human research subjects, then an institutional review board (IRB) may be needed, regardless of whether the proposers call it an “experiment” or whatever.
I don’t know the ins and outs of this contest, and it doesn’t immediately seem to be the case that it involves human subject. But it’s possible that some journal peer-reviewing “experiments” can require IRB approval, and not securing those has ended the academic career of at least one philosopher. If there’s any reasonable uncertainty about this, the best practice is to seek an IRB determination to be safe.
Anyway, this is to say that it doesn’t matter what it’s called. It’s still research of a sort, and good research design should include control groups when they’re relevant. No need to fixate on the word “experiment” here…
Novelty is irrelevant to IRB (or Thornley’s ethics board at Oxford). It’s experimental, you say, in the sense of being untested. But the question is whether the organizers are trying to produce generalizable knowledge (unclear) and whether the research involves human subjects. Neither the judges nor the authors are subjects for IRB purposes. Otherwise any essay contest or conference should be subject to an IRB determination.
In the Boghossian case, reviewers there were the objects of study, were deceived, and never consented. These seem like important disanalogies.
I guess the only wrinkle is this part:
“Members of the general philosophical community will be invited to rate each submission, to make the opinions of the community generally known.”
If the organizers then wrote up a report on the community rating, this collection of data could be subject to IRB determination. But that’s not what you had in mind, was it?
As for experiments, I’m not “fixating” on the word. I’m just using the words you used. Not all research requires control; and research design is largely irrelevant to IRB anyway.
Research design is fully relevant to IRBs. How else can they ensure there’s no unacceptable or undisclosed risks to human subjects?
Anyway, I didn’t mention research design in connection to IRBs. All I said about that was “good research design should include control groups when they’re relevant.”
Do you (1) agree with that and (2) believe that control groups are relevant to the contest? If not, then why do you think my proposals in this thread about control groups don’t work, which were:
Good human-written essays (that are submitted as AI essays) are needed to calibrate the judging; if they’re not judged as good philosophy, then that’s a big problem. And deliberately bad AI-generated essays are needed to test for false positives. And other control groups can do different work. Without these, any conclusion that AI can write good philosophy would be much too premature, which defeats the point of the contest.
And just look at the organizers’ own words about the point of the contest:
This is clearly a study (or even the start of an experiment), not just some prize contest for the best AI art or even an AI article. The organizers could even publish the results as generalizable research, unlike with the vast majority of mere contests.
This is not clearly a study unless it seeks to produce generalizable research. But if it does, then yes, as I indicated.
By research design I mean e.g. having controls, your original issue, not risks, obviously. You didn’t seem to be treating the IRB question and the design question as independent, but if you are, then we agree.
But to answer your questions: (1) yes, good research design requires proper controls; (2) I really don’t think that’s what’s going on here. If I want to know how fast robots can run a mile (or how well they can paint) and organize a robot contest, I don’t need to put them against human runners (or painters) in that particular contest. I’ll just see how fast (or well) they run (or paint). The control is everything humans have ever done in that domain.
Hi Nicolas, a robot race isn’t really a good analogy, esp. given the organizers own words:
There are obvious benchmarks for robot races, e.g., world records held by humans. Lacking a benchmark is the sine qua non for this contest, so a good analogy must preserve that feature.
And you still haven’t explain why the two control groups I proposed aren’t relevant (and there may be more control groups needed). If you deny control groups are needed, you’d need to directly refute those claims.
The IRB issue was a side remark, as I had said, so it’s not a good use of our time arguing about that, esp. as you seem to be misunderstanding me anyway. So, let’s move on from that.
But on your claim that this contest is “not clearly a study”: I’m curious…what do you think the methodology reports are for? They won’t be considered during the judging, so what good are they?
Without the organizers being more clear about things, it’s very reasonable to assume the methodology reports will be used for a study.
Perhaps the organizers think they can develop some benchmarks (if AI can produce good philosophy), or bubble up some best-practices in wrangling AI, or identify process differences between a good use and a bad use of AI, or glean other insights. Any which way, this strongly sounds like a study. If it quacks like a duck and walks like a duck…
You left out the painting analogy, whose very point was to avoid the benchmark problem. But granted, the mile wasn’t a great analogy.
I think it’s fine to introduce controls if organizers are interested. But every one of us knows a good paper when we see it. The methodologies are so we can learn what worked well for the winning papers (no, learning does not make it a study). You’re certainly right that the absence of controls limits how reproducible this all is, but this seems fine to me.
Look, you seem to want to turn this into a research study. In which case, yes, controls are important. I’ve already agreed several times. But I can’t read the organizers’ mind and it’s just not obvious to me that they are doing what you’re saying they’re doing. That’s because I’m watching this as am I the philosopher’s annual. I’m just interested in what judges think are the best papers.
Sorry, Nicolas, I forgot about your painting analogy (because it was parenthetical), but it’s a much better analogy than a robot race.
Still, an AI painting contest could go two ways, one of which is more useful for learning than the other:
The first way is to run it like this AI essay contest. No controls, just see what expert judges rate as good art (however that’s defined). Ok, that can be interesting.
The second way has the kind of controls I’ve proposed. E.g., it seems important to know if those expert judges can correctly identify human-made art as good art, or if they’re somehow biased toward AI art. And it’s important to know if the expert judges can rule out deliberately bad AI art, i.e., eliminate false positives.
Anyway, it’s not my contest, so I don’t really have a stake here. They can do as they please, but again, one way would be much more useful (valuable for learning, etc.) than the other.
I’d also agree that “learning does not make it a study” but with a major caveat. This contest is designed to intentionally learn something about the relationship between AI and philosophy. And that’s what makes it a “study”, not necessarily a formal study but a study nonetheless.
It’s not incidental or accidental learning, e.g., when I interact with someone, I may learn something about them, but that wasn’t a point in the interaction. I agree that kind of learning doesn’t make it into a study. There’s no discovery intention.
As a closing note before I bow outta here, see below where Zach (a contest organizer) states that the contest is an “experiment” in some broad sense. And I’d contend, off the cuff, that all experiments are studies.
Have a nice day.
“But I worry you are using *experiment* in some special loaded sense so I’m not totally sure.” (Zach)
I’ll bow out, too. I intend to learn a lot today. Hopefully, I don’t run afoul of my IRB regulations 😉
Have a nice day, too.
Anyone can submit human-written philosophy if they like, which will be judged without prejudice to the method of generation. Only this won’t be eligible for the prize, and the author would have to consent to their work being published alongside the other AI-gen works. I agree it would be useful to see if human philosophy easily rises to the top.
I wasn’t worried about one-shot prompting. My point was about the permitted uses, especially:
“Human tells AI to generate many essays, and selects the best for submission.”
In that case, the human selection may be the philosophically decisive part, while the judges see only what got selected. A methodology report can tell us that selection happened, but it can’t say what the human judgment contributed. So I think your description is right: the competition tests how effectively a human can use AI. But the site says it gauges AI’s philosophical capabilities. Those are different things.
In some ways it may be worse than that. From the website:
“There are no restrictions on how to generate the work except that it must be primarily AI generated. Minimal human collaboration is allowed.”
This is no doubt intentionally vague; it surely is difficult to enforce. Along with your concern, and many others, having a vague and possibly largely unenforceable standard for being sufficiently AI-generated makes me doubt that the purpose will have it’s advertised effect:
“AI is now exceptionally capable in mathematics and coding. In philosophy, the capabilities of AI are less clear. One reason for this is that there are not any obvious benchmarks by which AI-generated philosophy can be systematically judged. The purpose of this competition is to gauge the philosophical capabilities of AI in a way transparent to the philosophical community.”
In fact there’s a concern in that very passage. Thornley and Goodsell note that there are no obvious benchmarks for judging AI’s philosophical abilities. But they give no indication that they have developed those benchmarks, nor that any benchmarks will be used by the judges, systematically or otherwise.
(I wrote this before seeing Kenny Easwaran’s reply, but I think most of the concerns about what we can expect to find out from this experiment stand.)
Isn’t the methodology report supposed to settle the ambiguity? The website says the methodology report will be published along with the submission.
I assume the idea is to open up discussion among philosophers about the quality of the submissions. There aren’t as many objective criteria of AI’s success as in math, but if philosophers then agree these papers have merit, that’ll count for something. The judges’ comments (which, if I’m understanding correctly, will be made public) will presumably start this conversation.
“Isn’t the methodology report supposed to settle the ambiguity? The website says the methodology report will be published along with the submission.”
Yes, but some submissions might be dishonest, and some methodology reports might be less than competent. A serious study of AI philosophical abilities would make real efforts to minimise these risks.
“There aren’t as many objective criteria of AI’s success as in math, but if philosophers then agree these papers have merit, that’ll count for something.”
There are reasons to think not. Judges might have pro- or anti-AI biases. They might judge the papers as better or worse than they would have without knowledge of its provenance. So agreement or its absence about papers “having merit” or not might not count for much.
I would be extremely skeptical of any journal or contest that based its publication or award conditions on *any* sort of clear benchmarks. That’s how we get US News and World Report rankings, which are not hard to game by allocating money in the ways that are explicitly measured, and declaring certain people to count or not count as students or faculty in order to maximize the relevant ratios. It’s important that we judges will be using our own tacit understandings of what counts as good philosophy, rather than replacing it with a mechanized procedure.
If we did have a good benchmark, then we could all give up doing philosophy and rest easy that AIs could quickly be optimized for it. Without these benchmarks, we all have to keep thinking and re-evaluating what we consider good, and recognize that others will evaluate things differently. We will all continue to disagree about which journals tend to publish the best work, and we will disagree about the results of this competition (just as with any competition or prize).
But to the extent there is some convergence, it is evidence that many of us have similar judgments to each other, even if we can’t make it precise. And to the extent that you think I (and the other judges) have reasonable philosophical judgment, you might find the results of this contest interesting. (And of course, if you’re not a fan of the sort of philosophy I like, which I know many people are not, then I don’t expect you to care so much.)
Many philosophers might have strong pro- or anti-AI biases. This competition is set up in such a way that judges can expect most submissions to be mostly AI-generated. My point was that the attendant risk of biased appraisals might have been mitigated by operationalising at least some of the judging.
I think that these concerns about the epistemic limitations of this setup are reasonable, and compatible with your reservations about gaming some operationalised measures.
I don’t disagree that the results of the contest might be interesting. I genuinely would be somewhat interested to find that Kenny Easwaran and others considered an AI-generated paper to be very good. But I would be far more interested about this finding if delivered via more rigorous mechanisms.
Given limitations in the setup of this competition, I find it difficult to see what wider lessons might be learned from such an outcome. Unfortunately, I expect that people will be disposed to draw wider lessons from top philosophers judging some AI-generated philosophy to be very good or bad. So there’s a concerning disconnect between the reliability and the potential impact of the conclusions that might be drawn from this exercise.
Interesting. A few more things seem needed, such as:
It’s an interesting, even fun idea, though like Justin, I’m skeptical of the value of AI-generated philosophy. Even if it can do a good job, just because it can do x doesn’t mean we should let it do x.
There are many reasons for thinking so. For instance, this isn’t like finding a cure to cancer, where the destination is the only goal, not the journey. Like Justin, I will explain this some other time (very soon, with luck) in another article I’m developing…
These concerns, overlapping with others in this comments section, seem reasonable. They are also quite obvious concerns to raise given the information at hand. (I do not mean that pejoratively.) Since there are such obvious and seemingly serious methodological concerns, framing this competition as a step towards more rigorous assessment of the quality of AI-produced philosophy seems hasty. When one considers the renown of many on panel, that hastiness might morph into epistemic recklessness. And when one considers the many substantive ethical concerns raised about AI research, is it unfair to worry that some ethical recklessness might also be involved?
Agreed, Mathieu! And yes, my concerns were obvious…just some quick reactions while I’m still sleep-deprived and under-caffeinated… 🥱
Another concern is: after the contest is over and winners are declared–and I would bet against a null result–there may be lots of media attention. And most or all of that will tout the results, esp. in dramatic headlines, and not the methodological issues that should make us skeptical of the results.
So that misinformation (or premature information), that it’s been settled that AI can write good philosophy, will be what most people only remember from this.
Maybe one solution is if the contest organizers, in announcing results, make it crystal clear what the limits of their experiments and findings are. But that might be contrary to their goal, if it’s to promote AI in philosophy or academic research…
Yeah, mine were too! Sorry if ‘obvious’ came across wrong. I like obvious. It’s underrated in some places.
I think the suggestion about being very clear about limitations is a good one, especially if heeded by high-profile figures. But even if so, will your friendly local management take that sober message to heart? There clearly are risks here, especially in the current climate, being as anti-intellectual and AI-hype-ridden as it can appear to be…
Haha, no, I got your nuance on “obvious” and also wanted to reply to reassure you of that, given your note about the intention. All good! 👍
I think that if AI-written philosophy were winning *open* competitions, then that would be worthy of dramatic headlines (like when AI-written stories won open fiction competitions a few months ago – though the headlines there weren’t about the AI actually being good). But because this is precisely a competition *only* for AI-written philosophy, it’s results *can’t* support any sort of comparative claim about AI-written philosophy as compared with purely human philosophy.
I agree, Kenny. Yet that won’t stop news editors from writing sensationalist headlines and articles that readers misunderstand. This is an empirical question, so we’ll have to see…
Complicating but not necessarily undermining this response is Zach’s above response: “the judges will tell us their estimation of how the AI work stacks up against human work, which they are of course very familiar with.”
Zach’s statement reinforces to me the anyway reasonable concern that judgements about the quality of AI-produced philosophy will be at least implicitly relativised to human-set standards; and as Patrick notes, that certainly is how many such judgments could be taken, no matter their provenance. So I think it really would have been better, though still in my view far from optimal, to have this as an open competition.
Points 1 and 5 seem very important to me (as does 2, if it becomes relevant). I’m definitely happy to participate as a judge, but I do hope that the organizers will take these very seriously.
(I think points 3, 4, and 6 are less important, for the same reason that it’s not particularly important that other journals and contests do much to mitigate them.)
Fair enough.
I still think point 3 is needed for transparency, if the contest is really committed to that.
Point 4 is needed for a panel of judges who are more representative of philosophers at large, since they (we) also can be reliable judges of good philosophy. Otherwise, if the judges are all pro-AI or AI-curious philosophers, then no one would be surprised much if they declare that AI can generate good philosophy. AI skeptics, who are sufficiently open-minded, could lend more credibility to the contest; and if they disagree with the pro-AI judges, it can be illuminating to find out why.
I can concede point 6, esp. if it’s impractical to do so. It’s still be nice to see and would serve transparency, if that’s a goal.
I’m not sure that I’ve ever seen a writing contest that is committed to transparency of this sort! And again, I won’t be making my judgments by making a binary decision, “is this good philosophy or not”, I will be making my judgments by making a comparative decision, “is this paper better than the other papers in this competition or not”. I don’t want to prejudge whether or not the winner will be a paper that I would say is good philosophy.
Thanks Patrick for the feedback. To address some of your concerns:
I agree it would be terrible if our human philosophy were replaced by AI. Still, I think it would be good to have a better picture of AI capabilities in philosophy.
Thanks for this nice reply, Zach–good to hear all that.
On (4), I assume you have a pretty good idea that your judges are pro-AI or open to it. (Are all of them?) Many seem to be discussants in Daily Nous’ AI threads. And it’s also easy to spot the AI skeptics on the same threads.
But I’m not sure what you mean by this: “However isn’t it obviously silly not to be open to the idea that AI can generate good philosophy?”
Not all anti-AI folks think AI cannot generate good philosophy, either just now or in principle, if that’s what you mean. Many or even most of them seem open to it, but they’re concerned whether it’s a net-good, all things considered.
Or if you mean it’s silly to think it’s logically impossible or otherwise won’t happen (anytime soon or ever) that AI will generate good philosophy, as measured by a single writing product, then ok I might agree with that. AI is just data manipulation, and as the saying goes: torture data enough, and it’ll confess to anything.
But if so, why even run your contest if you already know that AI can generate good philosophy, or if that’s generally accepted, i.e., if it’s obviously wrong not to believe that? All you’d need to do is produce such an AI paper and post it here for an open discussion.
If you’re willing to answer a few more questions (which you’ve already patiently done), perhaps you can clarify the following from other coments:
Thanks again for your responsiveness!
Hi Patrick,
I can’t comment on the judges’ opinions of AI. I suppose you can infer something from their willingness to judge if you want, but for the most part I think they are, like me, curious about how AI fares in philosophy. You don’t have to be pro-AI to recognise that AI is of extreme philosophical significance.
The thing I think is obviously silly is not being *open to the idea* that AI can generate good philosophy, by which I meant being very confident that AI cannot generate good philosophy. There is a reasonable range of opinion on this matter, but this I think excludes very high confidence in a negative answer. This seems unreasonable to me because (a) AI capabilities are in general very hard to gauge, (b) judging philosophy is difficult and time-consuming, (c) AI capabilities have rapidly improved over the last years, giving very little time for gaining knowledge about the latest systems, and most importantly (d) AI capabilities are obviously *extremely high in general*. Current AI systems can do significant and difficult parts of general intellectual work at a superhuman level.
I don’t have high confidence in whether highly capable AI philosophy would be good. On the one hand I happen to think philosophical knowledge matters, so more is better, other things being equal. On the other hand highly capable AI philosophy would be correlated with a bunch of very scary outcomes. In either case surely it is worth knowing the capabilities.
On your new questions:
Thanks, Zach. And, no, I don’t mean “experiment” in any special sense that matters. I’ve long said experimentation with AI could be useful, so I wish you luck with yours.
Patrick is (I think) trying to see if your proposal potentially needs IRB approval. The federal regulations define research as “A systematic investigation, including development, testing, and evaluation, designed to develop or contribute to generalizable knowledge”.
(If you’re thinking: “it would be a ridiculous infringement on academic freedom and First Amendment rights for this competition to need IRB approval”, then you’re right, but unfortunately there is a ridiculous amount of overreach in social-science research by many IRBs.)
Hi David, the IRB thing was a tangential question that came up in previous comments. I had said then that it didn’t look to me (as someone who serves on a major IRB for AI research) that there were human-subjects involved. So, this wasn’t a big concern on my end.
However, you’re right that there can be overreach by some IRBs. So, if there’s a chance that someone could make a plausible argument that human subjects are involved in this study (and I don’t know if there is), then Zach would be well advised to double-check with his institution’s IRB. This is the safe strategy to avoid very serious problems that could wreck his career. What he does is up to him…
> why even run your contest if you already know that AI can generate good philosophy
Because I want to know *how* good the philosophy papers generated by AI can be at this point, and *what method* people use to make it do that!
I’m quite confident that AI *can* (either now or in the next few years) generate good philosophy, but I don’t know whether it actually can yet, and I certainly have no idea how to make it do this. It’ll also be interesting to see if all the best (or least bad) submissions use similar methods, or if there are some very different attempts that do well (either in similar or different ways).
I guess this is an interesting idea, but everyone can already test out the models for themselves to see how good they think the models are for the kind of philosophy they are interested in. One caution I have that I hope the contest organisers take into account is that the models are extremely good at bullshitting and stealing/using ideas from others without proper attribution, so a specialist in, say, normative ethics may not be able to properly evaluate a paper that Claude has written about meta-ethics unless they happen to know the relevant literature very well.*
In the kind of philosophy I do, I find that the models are not good at coming up with novel ideas (and I have definitely noticed that they will steal/use ideas without proper attribution) and they are not good at coming up with their own arguments. They can, however, be good for the purposes of developing an argument or an idea—they can provide useful critical feedback (although they tend to miss the forest for the trees).
*In mathematics, for example, the construction of a non-sofic group got a lot of attention, including among mathematicians, but some specialists in group theory have complained that the construction to a very large extent rests on two prior results (by humans) from a few years ago, and that this was obfuscated/not properly acknowledged by OpenAI/ChatGPT.
Hi Ben, it could also be interesting to submit an AI essay that was deliberately plagiarized, or bullsh*tting, or featuring stolen ideas. Would that make it pass the referees?
Assuming none of those count as good philosophy, that would also be another kind of control group, which this experiment seems sorely lacking…
I’m not sure everyone can test the models for themselves. For one thing, many people don’t want to spend the money for the fanciest models, or for thousands of samples. For another, someone might come up with a very clever system for running teams of agents together to get something better than you could get from just a one-off prompt.
I’m not saying they will do that, but the difference between pushing teams of AI to their limits, and what you see if you just tell Claude “Write me a philosophy paper” is pretty dramatic, and I’m interested in seeing what they come up with.
Sure, I meant “can” conditional on being willing to pay for the best models.
And yes, I grant that it’s possible that someone can come up with some clever philosophy scaffolding that will somehow unlock ability that isn’t there if you use the models more naively. My own experience leads me to think that’s improbable (without heavy human supervision), but I agree that it’s interesting to see what people can come up with.
Agreed, and this makes me wonder why the essays, rather than the systems, are the submissions. If what’s worth seeing is what teams of agents, retrieval, and critique loops can do, entrants could submit the scaffold/workflow. It could then be frozen and run by the organizers on several prompts the entrants haven’t seen. The entrant’s skill would sit in the design of the system, where it is visible and comparable, rather than in inspecting a thousand outputs and choosing the lucky one.
As designed, a first-place finish shows that some human-guided, AI-heavy process produced one essay these judges liked. That’s interesting, but it doesn’t show that the system produces good philosophy reliably or tell us how much of the result came from the person who recognized which output was good.
Hi Ben, I share your experience AI is not currently so great at original philosophy. However I might be using it in the wrong way, or on the wrong topics, etc., so I am not confident at all in my assessment. Hopefully the competition clears up some of that uncertainty.
I trust our judges to see through AI bullshitting. However point taken about AI plagiarism. Of course our judges (and AI screening) will try their hardest to assess the originality of the work, and afterwards all submissions will be published online, which should reveal any plagiarism that slips through.
Yes, I was too dismissive in my original comment. I think this is a valuable experiment.
I have experimented so much with these models that I feel like I have developed a pretty keen intuition for what they can and cannot do. This makes me pretty confident that you can’t get any good philosophy out no matter what scaffolding you use, if you just let the models run completely autonomously without any human supervision. In fact, my own experience is that the models will often produce worse results if you make them think autonomously for too long, because they will overthink, overcomplicate things, and just go off the rails. (Of course I am open to the possibility that someone finds a clever way of avoiding this; I just think it’s unlikely.)
With human guidance, you can certainly do much better, however. So I guess the interesting thing (for me) will be to see how little human guidance is needed to get something good out of it.
The prize money for this competition comes from a non-profit that is heavily connected to the Effective Altruism movement. This shouldn’t be surprising. From the EA perspective the acceleration and/or automation of philosophical research is incredibly exciting: one may quickly resolve all kinds of technical questions that relate to the project of calculating and then producing the best possible states of affairs. And one may finally be rid of recalcitrant problems that have served as irritating obstacles to that righteous project (the ‘repugnant conclusion’ etc.) The machines, ultimately, will both determine not merely the means, but the correct conception of the end itself, and we can finally cease worrying about thinking about anything at all. And who wouldn’t be rid of the task of thinking, since it anyway is just a means to producing some state of affairs? (Unless, of course, the machines conclude that keeping human beings thinking for themselves is part of the best state of affairs – but how plausible is that?)
Obviously noting the provenance of the prize money is not by itself sufficient to undermine potentially sound motivations for holding the competition in the first place. I just think we should note that in the very framing of the enterprise–most importantly in the idea that LLMs could be ‘good at philosophy’–much philosophy has already been done.
Couldn’t effective altruists be concerned that a misaligned AI might produce highly convincing philosophical arguments that go against their preferred visions for the future?
Effective Altruism is philosophically dogmatic in character. The basic edifice is already subject to devastating objections that have never been addressed with sufficient seriousness by its leading proponents. So for EA believer being proven wrong on the fundamental doctine is presumably not much of a serious possibility in the first place.
That’s not really what I meant. Maybe an effective altruist believes that X is the best future. They might think that automating philosophy will likely produce convergence towards the pro-X conclusion, correctly in their view. But they might equally think that it might produce convergence towards an anti-X conclusion. It depends on what they think about where AI capabilities are at, and whether they worry about alignment issues. (We all should.) So I don’t see why an effective altruist would be particularly happy with automating philosophy, at least not with LLMs.
I see, I understand you better now. Well, once the promise of automating philosophy, and hence the path to the glorious future, is on the table, so too is the threat of automating the path to various dystopic outcomes (including through the route you mention). That’s all part of the eschatological vision of EA, the high stakes etc. But the important point is that EA’s conception of the ethical goal to be achieved, as well as what the point/mission of philosophy is given that goal, is highly comptatible with the first outcome being entirely possible. They see a certain promise (and hence a certain danger) where others may simply see folly.
With apologies to others for pursuing a tangent that’s off-topic to the main thread here, would you be willing to point me in the direction of what you take to be the strongest objections to Effective Altruism? I published a paper in 2024 (‘Why Not Effective Altruism?’) that at least attempted to address the most serious objections I could find. As I’m currently in the midst of writing a textbook on the topic, I’d certainly appreciate further pointers if my previous efforts failed.
Your paper distinguishes more carefully than I have been here between EA and utilitarianism. I think most (though not all) of the issues with EA stem from its inherited utilitarianism. And most problematic, in my view, is just the fact that utilitarianism has a bogus conception of human agency and ethical/moral goodness.
It would depend on the particular version of non-utiliarian EA, I suppose, but my suspicion is that it is likely many of the most problematic features of utilitarianism are inherited by putatively non-utilitarian versions of EA. Even if you slap on deontological side-constraints or what have you. It is hard to see an effective altruist as not interpreting what benevolence calls for, for example, along broadly utilitarian lines.
More generally, though, it seems clear that most people attracted to EA as a movement are thinking of morality and ethics along the lines of maximizing good states of affairs in the universe. Again, you needn’t technically be a utilitarian to think that way, but it has obviously helped many.
Most objections (e.g. the famous counterexamples) to utilitarianism depend on its hegemonic character as claiming to exhaust the moral reasons; it seems much rarer to find arguments for the conclusion that the thing utilitarians focus on (promoting aggregate welfare) isn’t even to be counted among the moral reasons.
The main exceptions on my radar are the cluelessness objection and anti-aggregationism (both of which seem to me among the weakest objections to utilitarianism, but YMMV). I’m not sure how most agency-based concerns (demandingness, alienation, special obligations, etc.) would apply once we’re only talking about reasons of beneficence and explicitly leaving open that they may be outweighed by other reasons (e.g. in 90% of your life). But if you have a good reference arguing the contrary, that could be very helpful!
(I’d also welcome emails from anyone with suggestions if they’d rather not comment here.)
P.S. See the ‘Objections to Utilitarianism‘ section of utilitarianism.net for my responses to the most common objections. Surely one needn’t be “philosophically dogmatic” to find these responses convincing? I think I have some pretty devastating objections to non-consequentialism to offer in return, but I’d hardly call someone “dogmatic” just because my objections fail to sway them!
Some people are talking about the new Simulation Argument against effective altruism! Goes something like this.. If we might inhabit a simulation (per Bostrom and others), effective altruism’s expected-value logic should prioritize influencing the simulators…. whose decisions could affect countless copies of us. But then the thought is since this absurdly speculative cause could outweigh proven charities, the framework either looks like its oging to accepts bizarre conclusions or admit that expected-value maximization needs limits.
The other one I had heard (maybe you already know about this one) is the Evangelism Paradox argument against Effective Altruism.. it’s a strange one that says if promoting effective altruism does more good than helping directly, effective altruists should promote it.. (and persuade converts to promote it too), but this then produces an endless chain of advocacy with nobody actually helped. BUT if promotion does less good, they should stop defending the movement. Thus effective altruism seemingly requires either infinite evangelism or its own disappearance
Thanks! There’s been a fair bit of discussion on the EA forum over the years of the latter issue, which they call “the meta trap”. (The standard solution, as you might expect, is to pursue a mixed strategy.)
I’m not an effective altruist but I like consequentialism, so I dispute the argument. Conditional on simulation, I would guess the expected influence of my actions is much less than conditional on no simulation. So a small probability of simulation should have a modest but nonzero impact on what I should do. That seems right!
The evangelising argument is also unsound for reasons that have nothing to do with EA. Replace “promoting EA” with “promoting doing good deeds”.
I think what you say about the Evangelism one is right, not sure about the simulation one. There’s a couple other tricksy knockdowns kicking around, one says once effective altruism identifies whatever counts as the “most effective” cause, donors rush toward it until its marginal value falls… which is supposed to mean that if its recommendation succeeds, it becomes false where as if it remains true, the recommendation has failed, or so the thought goes. Another weird one (like the simulation one) goes like this: EA favors evidence-backed interventions, so an oppressor can reduce concern for victims simply by hiding them and destroying the evidence!! The line is then that the better suffering is concealed, the less confidently it enters the EA calculation (and so this basically turns secrecy into a moral discount (bad!)
Hi “Skeptical Voice”, for what it’s worth I was going to self-fund the competition with much much less prize money before we got external funding, and the framing hasn’t been affected at all.
I also share your worry about a terrible future for philosophy and humanity! That makes me *more* interested to find out how good AI is at philosophy, so I can know where we stand.
as excellent as the judges in their areas of specialty, this is an incredibly small pool of judges for this kind of contest, and their total expertise is not that wide at all. This has huge effect on who would even attempt this competition. Most entries will be at the vicinity of specialty of these judges.
Not the biggest fan of this initiative, but in fairness, they do say that more judges are TBA. On the other hand, if those announcements are not timely, your worry about selection effects remains.
I agree! It would be great to have more judges from a wider range of areas. However it is difficult to convince busy distinguished philosophers to set aside time to judge essays. We encourage submissions from all areas of philosophy and will try our best to have judges competent to assess each submission.
Guys this is a really cool idea that will let us all see what these models are really capable of doing (or not doing), with the best available tools and people who are virtuosos at using them! We’re going to learn a lot whatever the result is. It doesn’t cost you anything not to enter.
That’s what they said about Los Alamos!
“They” said nothing of the kind about Los Alamos. It was a deadly serious project and it was well understood what its specific goal was.
Yes, I am exaggerating a tenuous similarity between two technological projects for comedic effect; a “joke,” if you will. So, to clarify, I do not actually believe that philosophers running a writing competition is comparable to the creation of nuclear weapons (but I hope the mental image of an AI paranoiac crying otherwise might produce in you the humorous effect it produced in me).
It’s a cool idea and it’s helpful to put things in perspective. The prize pool is quite modest and the papers will almost certainly not be eligible for publication in major journals. Good people will read stuff that may or may not be good. We’ll learn something.
I wouldn’t wish evaluating thousands of AI generated symbols upon my worst enemy. Thoughts and prayers for the judges.
I’m curious about the reasoning behind holding the methodology back from the judges? Huge difference between what can be done with a one-shot prompt vs. multi-shot. If I were judging I’d want to know whether it was one-shot, because a one-shot paper vs. a multi-shot paper is way more impressive if the quality is the same. If you want to keep the method mostly blind, you could have a category for one-shot that’s known to the judges as such but the method is otherwise held back?
Hi Sam, the rationale is that we want the judges to exercise their philosophical expertise to judge the quality of the paper, with as small as possible contamination by extraneous factors.
The one-shot idea is interesting! I think however that it is hard to elicit good philosophy even from multiple prompts, and we are more interested in seeing the best philosophy AI can produce. I can imagine the one-shot category would grow in interest along with capabilities, so we might consider implementing it in any future versions of the competition.
This is depressing and it saddens me to see a couple friends of mine on the judges list. It honestly makes me worry about their judgment. There is no way that the results of this will tell us anything interesting. But there is every reason to see it as yet another instance of hype.
Hi! I hope people do not see membership on the judge committee as equivalent to being “pro-AI”. I cannot speak for Elliott or the judges, but I am not pro-AI myself (it seems insanely dangerous).
Still, Elliott and I and the judges are for many reasons interested in the capabilities of AI in philosophy, and we see the competition as a promising initial avenue for understanding that. Are there particular methodological concerns you have?
Why? If they can’t find a single good paper then we’ll have learned something—AI at the very least doesn’t easily produce good philosophy. So maybe it was just hype as you say. You’ll be right! If on the other hand they find good papers, then we’ll have learned something too—that some methodologies can produce some good philosophy. So maybe it wasn’t just hype after all. Either way we’ll learn something.
Chin up! This is just a bit of fun and might uncover something interesting- at any rate it’s not helpful to shame people judging this. This is a lighthearted initiative and should be treated as such
Same.
I think this is a great idea and I’m happy there’s people out there organising and running it. And there’s some helpful advice in these comments. But even if not all of it can be realised in this instance of the competition, I surely hope there will be more.
As is often the case with this topic, (some) people are in complete denial. AI is being used in research (across all disciplines, not just philosophy). All the more reason to bring the topic into the open and have a shared discussion about what the resulting research norms should look like.
If you mean that they’re denying the widespread use of AI in research, then they’re not in denial, because no one is doing that.
If you mean that they’re denying the inference from widespread use to opening a shared discussion for new research norms, then maybe they’re in denial, but it’s far from obvious that their denial is mistaken.
Your comment makes it sound like their denial *is* obviously mistaken, but I think this equivocates on the two meanings above.
I read that as denial about capability – and if it’s widespread , then you have abductive basis to infer capability.
I want push back a little bit on the narrative that AI is “exceptionally capable” in mathematics. It’s exceptionally capable at solving (certain) well-specified math problems, but there is so much more to mathematical research than just that. Mathematics is also full of ideas—derivatives, integrals, the fundamental theorem of calculus (which is profound, but trivial to prove once you have the required concepts), modular arithmetic, soundness and completeness, finite axiomatizability, validity, isomorphism, homeomorphism, epsilon-delta proofs, uniform continuity, Lipschitz continuity, compactness, random walks, ergodicity, and the list goes on and on. As far as I know, AI hasn’t been able to come up with any interesting ideas of this sort yet. (And I don’t think it’s because coming up with ideas is intrinsically more difficult than doing proofs. I think LLMs just have a very uneven ability profile.)
Hi Ben, I agree that there are mathematical activities that AI is not yet superhuman at. But that is a very high bar! Recently, AI has autonomously proved theorems that I could not have proved if I had devoted my entire life to the topic. Or to use my own work as an example, it recently answered problems in higher-order intuitionistic modal logic that on my own I had found intractable (admittedly, using methods from literature I had not discovered). I don’t think “exceptional capability” should be reserved only for the time when human mathematicians are rendered obsolete. Anyway, I think the point stands that there is marked difference in capability between mathematics and philosophy.
I still want to resist the narrative a bit. Don’t get me wrong: AI has gotten really good at solving certain well-specified problems. But there are still several very important respects in which current AI models are far below the level of a typical human mathematician (not just “not superhuman”). I think the following is a pretty standard research loop in mathematics (Lakatos describes this pretty well in Proofs and Refutations):
I grant that AI has gotten really good at 3 (at least for certain types of proofs. My sense is that mathematicians still do not think the proofs AI have come up with have been very “deep,” although they have often been clever) and also 5. But I don’t think the models are at all good at 1, 2, 4, 6, or 7, and those are also very important parts of mathematics research. That’s why I think it’s most accurate to say that AI is exceptionally capable at certain types of mathematical activities, while still being far below humans at others.
Right, but we don’t require that all human mathematicians are good at all these things. If there was a human who was extremely good at finding counterexamples and proofs for ideas conjectured by others, while being weak at the other aspects of math, I think we would be happy to say that they were a good mathematician; indeed, they’d be in high demand as a co-author.
Of course if AI never moves beyond (3) and (5), then most human mathematicians won’t be out of a job, human mathematicians who are only good at (3) and (5) possibly will be out of a job, and we might change our attitude as to what counts as the core of mathematics. As I mentioned in a different thread, something similar happened in theoretical physics even before the advent of LLMs: it used to be an extremely useful and sought-after skill in physics to be able to do very lengthy symbolic calculations without making mistakes, but Mathematica has made it much less useful and sought-after.
Yes, humans have uneven profiles as well, and I agree that technology renders certain skills less valuable. But I think it would be a bizarre experience to meet a mathematician as uneven in its abilities as ChatGPT, just like it would be bizarre to meet a physicist with the ability profile of Mathematica.
Anyway, I don’t think Zach and you and I disagree on anything substantive. I’m fine with calling AI “exceptional at math” as long as it’s recognised that this exceptionality is (at least for the time being) very specialized.
For precisely this reason, I think “exceptional at math” is a better phrase than “good at math”! They are exceptionally good at some things, and exceptionally bad at others, often things that for humans go together.
I have one more remark: I think that in a lot of mathematical research, theorem formulation and proof aren’t neatly separable because they organically co-evolve. A sort-of-correct-but-strictly-speaking-wrong theorem formulation and strictly-speaking-invalid-but-on-the-right track proof idea may come long before the rigorous formulations of either (think of the development of calculus or probability theory for example).
In fact, one thing I’ve been wondering is whether the intense RL’ing to get LLMs to get hyper-attentive to validity and really good at finding counter-examples might sacrifice other parts of their math ability. Professional mathematicians are really good at thinking about things in a way that is strictly speaking invalid (for example because it over-extends some concept or ignores some boundary cases), but still close enough to correct to be useful. That ability strikes me as important for math research. (It strikes me as maybe even important for theoretical research in other sciences, such as physics.)
I think that’s plausible; certainly for physics, probably for math too. (And I agree that the three of us aren’t really disagreeing on the substance.)
I actually think that some of the examples that have been discussed so far precisely are instances of 4 and 6. You don’t find a counterexample to the Jacobian conjecture without spending a bit of time thinking about what it would take to prove it, what the obstacles to that proof method are, and then thinking about how that leads to the construction of an example that makes it false. In fact, I think that what we’ve seen is more 4 and 6 than it is 3. (And I don’t think we’ve seen anything like 1, 2, or 7. And I think, very importantly, we haven’t seen anyone actually treat an AI writeup of any of these proofs as anything like an actual paper – despite the low standards of a lot of mathematical writing, AI isn’t anywhere close to reaching it for novel work.)
I think part of the problem is that I formulated 4 and 6 in such a way that they partly overlapped with 7. I was thinking of 4 and 6 as involving partial theorem repair and 7 as involving a bigger overhaul. In my own experience (mostly using ChatGPT 5.6 Sol), the models are oddly bad at all of that.
But I definitely agree with you that the models definitely, in some sense, have a very deep understanding of the problems they are solving.
Yes, a common criticism from mathematicians seems to be that the models aren’t able to appreciate what the central ideas are in their own proofs, so they waste a lot of space presenting uninteresting background or over-explaining minor steps. That’s pretty of baffling from a human perspective.
I wonder what we’re meaning by ‘philosophy’ here. I think it’s obvious that current AIs can, in principle, do excellent work in philosophy. Here’s my argument for that.
Is it worth having a competition to see whether the conclusion of this argument is right? I’m not sure, unless one is using ‘philosophy’ in a way that rejects premise 1.
I guess. But in philosophy (including philosophical logic), the main difficulty isn’t usually in showing that premises P1, …, Pn entail some conclusion C. It’s much more often finding premises P1, …, Pn that entail some (surprising or interesting) conclusion C, and then trying to show that those premises are plausible (which usually does not involve proving, mathematically, that they are). So it’s still interesting to find out whether AI can do that kind of philosophical work, even if we grant that AI is really good at logical deductions (and we grant that that counts as philosophy as well).
Yes, I agree there are kinds of philosophy which we don’t have evidence that AI is good at. It’s not bad at one-off plausibility judgments, but embedding them into longer papers, that’s an open question how good they are.
Still, being a boring old analytic philosopher, I would (a) like to have as precise a characterisation of what “that kind of philosophical work” (which we’re unsure about/hope AI isn’t used for) covers, and (b) suggest we not to use ‘philosophy’ to mean exclusively it.
Sadly Elliott and I did not first define philosophy before deciding to host the competition. We certainly do not mean to exclude mathematical philosophy. In my mind, merely proving a theorem of antecedently known importance to philosophy is a valuable contribution but is not itself philosophy (rather, it may be good mathematics), so such work might not be judged as highly as its overall contribution to knowledge would merit in a topic-neutral competition. I am interested in your suggestion elsewhere that logic is a “part” of philosophy. I agree there is significant overlap but I wonder why not then include maths as part of philosophy as well?
This isn’t quite the actual history, but imagine that someone had clearly stated the question – is there a consistent system of axioms whose theorems can be listed by an effective procedure (i.e. an algorithm) which is capable of proving all truths about the arithmetic of natural numbers – and then Godel came along and proved there wasn’t. I think the coming along and doing the proof would be doing philosophy. I don’t think that whether Godel counts as a philosopher turns on just what contributions he made to the definitions or importance of the result; the result itself is enough.
Take Plato’s contrast between dianoia and noesis. Dianoia is just calculation, syntactic symbol-manipulation according to rules of inference. AI can probably do dianoia pretty well. Lots of its recent math results look mostly like dianoia. The question is whether AI can do noesis.
Even in formal logic, the philosophical work is noesis rather than dianoia. It’s the non-logical work of figuring out the logic of some topic. Consider Graham Priest’s work on meontology: it’s the non-logical work of figuring out the logic of non-being. That’s noesis. Work on the axioms of set theory isn’t dianoia, it’s noesis. Of course, you have to be good at dianoia to make any progress in noesis. But noesis is a much deeper and harder kind of cognition.
I just don’t agree that the philosophical work is only noesis. That’s the excluding logic from philosophy view that I’m rejecting.
In this paper, we were largely doing proofs, i.e., symbol manipulation. And I think it was still a totally worthwhile philosophy paper.
Notes on Some Ideas in Lloyd Humberstone’s Philosophical Applications of Modal Logic
Just from a brief look, it seems to me you do far more than just symbol manipulation in that paper. I may be wrong, but as far as I can tell, you define new concepts, formulate novel theorems and even axioms, and (yes) prove interesting results. There is no way ChatGPT or Claude could have written that paper (although I’m sure they could have proven the results after you’d formulated them properly).
I certainly didn’t mean to imply that philosphical work is only noesis. Noesis requires lots of dianoia. And lots of ordinary philosophical day labor is mostly dianoia – e.g. stress-testing theories, figuring out the consequences of some position, etc.
Hi Brian, your conclusion is surely right that AIs can do excellent work in philosophy. However this does not answer many natural questions about their capabilities, e.g., how good they are at philosophy (similarly, an underglaze painter might do excellent work in pottery without being a very good potter). The question of what the capabilities of AI in philosophy are is the significant one, which we hope the competition will start to uncover.
What’s the difference between doing excellent work in philosophy and being good at philosophy? Would you say someone (or something) that could write a logic paper at the level of a good Journal of Symbolic Logic paper was good at philosophy?
I was thinking: doing excellent work in field X = doing something in field X that is excellent. Being a good X-er often requires a specific other activity, or a general expertise.
I generally think of JSL as publishing mostly non-philosophical material. So no, an excellent paper in JSL would not make me think its author must be a good philosopher. I am curious how the authors of those papers think of themselves, if you have any insight.
I think when I write papers like that I’m writing philosophy! Like I think this paper was a philosophy paper, even though it was pretty much entirely coming up with counterexamples.
Deference and Infinite Frames
Would that even have been eligible for the competition, had an AI written it?
Or to move to something I’m working on, there are a lot of open questions at the end of this paper
Possibility Frames and Forcing for Modal Logic
I’ve solved one of them, though I don’t know the status of the others. (I did it without AI, though I’ve since got an AI to come up with an independent proof.) I thought I was doing philosophy when I did that. It sounds like a paper that solved some of the others simply wouldn’t be considered for the competition.
Sure, I would think these solutions would be eligible for the competition.
Proving novel results that are publishable in JSL is very different from writing a logic paper that can be published in JSL!
Current AIs can definitely do some excellent work in math – but I haven’t seen anyone claim they can write a decent mathematical paper. In fact, I’ve even heard people from OpenAI who have been trialing these systems say that they’ve gotten AI to produce dozens of novel proofs of results they were interested in, only a few of which were worth reading (and none of which were written like a decent paper). It’s rare for a human to produce a novel proof of a result you were interested in that isn’t worth reading, but the AI has managed to do it repeatedly!
I hope I’m not diminishing any of the consideration-worthy points being made on these various AI threads when I observe: it’s remarkable how obviously the disagreements come down to disagreements about (i) the degree to which the cultural artifacts containing philosophical argumentation are similar to the cultural artifacts containing natural- or social-scientific argumentation and (ii) how important that degree of similarity is.
Ben Mitchell-Yellin and David Wallace (and others I can’t remember) have noted this, so I realize I’m not making an original point. But maybe my formulation is more explicit about what’s at issue.
Thanks Zach! I still have major concerns though:
So far most capable philosophers are not trying hard on producing good AI-written papers, for various reasons. This contest would give the incentive to try harder. And if it succeeds, what’s followup? Help make our profession look obsolete? AI is only going to get even better.
You might think, if AI is good, then we better know, rather than burying head into sand. But still if the above attempt succeeds, it still accelerates our demise. Do you really want that?
On the flip side, if the attempt does not succeed, namely that there is no particularly good paper, what could we learn from it? Have a false sense of reassurance? Have another contest in a few months, and a richer, more prestigious one?
What is the end game? Excuse my slightly alarmist tone to make my point.
Hi Alice, I am very sympathetic with your concerns. I certainly don’t want to be made obsolete or to accelerate our demise. However I think there are two ways in which we avoid obsolescence, conditional on extremely high AI capabilities. One way is for us to adapt to the technologies of the day and find a path where we still have a role to play. Great! The other is to stubbornly permit only “human-made” philosophy to see the light of day (even when AI could do it much better). In the latter possibility I think we have avoided obsolescence in name only: our field will quickly become obsolete, even if we are not obsolete within the field. Already, I find it a bit ridiculous that parts of the profession think AI-assisted philosophy is somehow inherently illegitimate.
I suppose the competition might very slightly boost the probability of the former “AI-collaboration” possibility and very slightly decrease the probability of the latter “AI-ostracising” possibility. Perhaps there is some difference in the magnitude of probability-change so overall the competition increases the total probability of obsolescence, but since the former possibility is much more valuable in relation to obsolescence I guess the competition could have positive EV. We have vague plans to hold competitions in the future too, where I would think the calculation to be more-or-less the same. But please let me know if you think the calculation is flawed.
Unfortunately for us, whether philosophy becomes obsolete is largely driven by external factors, e.g., the increase in general AI capabilities, which may outstrip ours so thoroughly that we have no chance of contributing anything. I don’t think it’s wrong to be an alarmist about that. We should be alarmed. We should not forget: a majority of relevant experts find it plausible that AI WILL SOON KILL US ALL! Less dramatically, it seems obvious that increasing AI capabilities sufficient to make philosophy obsolete will at some point lead to enormous societal upheavals. I personally am quite pessimistic about how that will turn out but while we wait to find out it would be fun to do some cool philosophy.
This could be an excellent resource. As a journal editor, I’d like to see these papers and how they compare to what I hypothesize to be AI-generated submissions to the journal. What these look like changes every few months, so its interest from this perspective will be temporary.
I’ve been gathering a stack of submissions that I’m fairly confident are AI-generated. There’s actually a good amount of variety in them (though there are some similarities I’ve observed, that I hope to write up).
would be very interested in reading this if you do!
I think the pessimists are wrong about AI’s philosophical capacities. Frontier models can already produce good philosophy when used well, and in five years or so the claim that they can’t may look absurd… But the fact that AI can do good philosophy doesn’t (on the view I want to suggest) settle whether journals should publish AI-generated work.
The debate, such as it is on DN in the comments threads, has so far focused on two functions of publication. One is epistemic. Journals disseminate new and important truths. The other is credentialing, where the line here is that publications provide evidence of a researcher’s abilities, evidence we need. There is, however, a third function: sustaining the community in which philosophical inquiry is possible as a livelihood.
That function is epistemic too. A reliable research culture requires trained philosophers who can do all sorts of things including identify and certify whatever novel truths the bots find. Maintaining that expertise requires institutions that give people both the opportunity and the incentive to make careers in philosophy. If AI-generated research begins to substitute for human research in journals, it will weaken the publication-based economy on which those careers presently depend. The likely endpoint is an awkward hybrid system, where philosophy is pursued by independently wealthy amateurs and a shrinking number of patron-supported professionals, alongside machines producing work that fewer humans remain qualified to evaluate.
This gives us a diachronic reason to restrict AI-generated philosophy in peer-reviewed journals… the thought is that even if such work produces genuine truths now, normalizing it may erode the human expertise needed to distinguish genuine discoveries down the road from trivial or uninteresting results. AI can’t solve this problem by certifying its own work without a kind of circularity. So the policy that maximizes the production of truths in the short term (namely, the policy of letting Claude 5.1 or if you like Claude 15.0 in several years, write the papers) may undermine the institutions required to turn future outputs into knowledge we’re in a position to identify as such and use. The case against publishing substantially AI-generated philosophy is therefore, as I’m suggesting, not that AI can’t do good philosophy but rather that allowing AI to replace human researchers is a recipe for consuming the epistemic infrastructure on which trustworthy philosophy, including trustworthy AI philosophy, depends.
In the scenario you’re describing, we might still need human philosophers, but we would need many fewer of them; presumably there are ways we could reorganize academia to manage that. (Not ways that would be good for our personal livelihoods, to be sure.)
This is an interesting argument I’ve not come across before.
Definitely an important issue. I guess one relevant question though is whether we should seek to maintain the journal-publication economy and restrict journal publication in the way that makes this useful (which means that publication of work often moves elsewhere), or try to replace this with a different economy that sustains the community while the journals continue to function as venues for publishing work.
AI-generated writing routinely steals ideas from human work in the training data. So it seems very plausible that the “best” submissions will actually plagiarize human work. For that reason, I worry that this contest will give a very misleading picture of what AI is actually capable of.
I thought people had given up beating that old drum ?
It may be an old drum, but it’s still true. I’ve seen it happen several times myself including with very recent models (Claude Fable for example): the model will present some idea or argument that is known in a sub-literature without referencing the relevant articles. To be fair if you then ask, “did you come up with this yourself or did you get it from somewhere?” the recent models are good at finding relevant literature. But they often won’t give you those references in their first reply.
This is an important point, I’ve noticed the same
To be fair, I think we’re now reaching the point where taking an idea a *human* claims as original and asking Fable or Sol whether it’s really original or has antecedents will end up finding lots of places where someone failed to remember having heard of something and thought it was their own idea.
Indeed. Humans are plagiarism meat machines.
This does happen but it’s largely fixable. First by being conscientious. Ask for sources and citations. When prompted, models (at least Claude which I’m more familiar with) will almost infallibly reveal their sources (if any). Second by learning how to work with the AI. In Claude, for instance, you can create skills. It’s not very difficult to create a skill that will be automatically deployed for research tasks which forces the model to provide citations for its claims. Indeed, as Kenny notes, the AI will humble you by showing you how unoriginal you are. People have been ‘borrowing’ ideas forever. Who hasn’t had the (admittedly unpleasant) experience of realizing you’ve been scooped, or your original idea was in fact old grist, or some paper omitted to cite you?
None of this is meant to deny that training models involved massive copyright infringement. There’s no doubt about that. But the problem of users committing plagiarism by using AI is largely moot.
Humans do this too. If I’m asked in conversation to explain some philosophical idea, I don’t automatically add citations. A chat bot response to a question isn’t an academic paper.
Not in the same way and not to the same extent. Ai will reiterate ideas in much greater detail than a person who is simply unwittingly plagiarizing (or independently discovering) an idea. And, moreover, ai will do this even of you explicitly ask it to come up with something novel.
Chatgpt 6 Astra came out recently and I was playing around with it today. I asked it to come up with a novel idea in Bayesian epistemology and it returned to me basically the idea known as “resilience” in the literature, but under a new name.
I then asked it (in a different chat window) to formulate a novel theorem statement. After 10 minutes, it came back with something. I then asked if it was already in the literature somewhere. After 8 minutes it found it (or a slight generalization of it) in a 2018 arxiv paper. By its own lights, the similarity was so great that it judged that it probably got its own theorem from that 2018 paper, even though it couldn’t be sure.
What I find supremely disturbing about the use of GenAI in these sorts of social/institutional ways is the complete eschewal of the ethical concerns that should be, at least to my mind, at the fore of every discussion about this technology.
Ultimately commercial LLMs are giant plagiarism machines. The corporations that operate them took the entirety of the scholarly commons, ground it up, and are serving it to us without any attribution, compensation, or concern about the individuals who worked entire lives to produce that grist.
Events like this indicate we, as a scholarly community, have completely conceded defeat on having any control at all over our own intellectual work. Someone please tell me what I’m missing here?
I think I agree with all of this, except I don’t think you are missing anything. You say “Events like this indicate we, as a scholarly community, have completely conceded defeat on having any control at all over our own intellectual work. Someone please tell me what I’m missing here?” I think we have conceded defeat, in the sense of: we have to pick our battles, and shouting about LLMs being trained on our work is not something we have any chance of winning and people seem to have recognised this (and those who haven’t recognised it should). Going forward, just assume all your published work is getting fed in to LLMs.
I don’t understand the plagiarism concern here. If I ask ChatGPT to explain the many-worlds interpretation, its answer clearly indicates that it read my book. Great: I like my ideas being disseminated. If I then ask it for the sources of those ideas, it gives an accurate answer. How is that different from asking a human philosopher of physics about the many-worlds interpretation?
Good point.
The difference is that it will often do this even if you ask it to come up with an original thesis or argument.
Another difference is that the human philosopher is not using your work, without consent or compensation, to make money.
People in fact ask me questions about philosophy of physics all the time; it’s part of my job, and in that sense absolutely I make money from it. Of course I’ll discuss sources of ideas as and when they’re relevant, but I don’t ask consent from the originators of the ideas, nor do I compensate them.
I suppose it’s a good thing that commercial LLMs are now able to correlate their output to a set of articles that are related/say roughly the same things as the output.
But I wasn’t thinking about the output of a prompt, I was thinking about the training of the programs. As Someone put it below, these companies are using our work — all of our work, all of *everyone’s* work ever — without consent or compensation to make money. It’s gotten to the point where these companies are physically destroying books as grist for their mill.
If the issue is just copyright violation, then sure: it looks as if there might be a case to answer. Certainly a bunch of class action lawsuits are pursuing this. But if someone writes a bestselling novel or popular science book, and they reference and discuss my writing, they don’t need my consent and don’t owe me compensation. I’m unsure why this case would be different, beyond the copyright issues (which would also apply if a human author read a pirated copy of my work).
I guess my point was the copyright issue. I don’t see why the “just” though. I didn’t agree to my work being used in this way. Still it’s being used in this way. Regarding the human author reading pirated copies: here I at least would have agreed and don’t mind. Not so in the case of AI.
There is also a substantial difference between individual action and corporate action — differences in power, intent, ethics, etc.
They can, but that doesn’t mean they will.
OpenAI came out with a solution to the Navier-Stokes problem yesterday, and the initial document, although 165 pages long, contained only 16 references, most of which were not from the recent relevant literature.
Does that mean ChatGPT came up with its proof ex nihilo, without building on recent work? I’m in no position to judge and I guess we will see how things play out. But I asked Astra about it, and it claimed that the proof in fact builds heavily on recent work, none of which was cited*. It’s also noteworthy to me that almost all the references are at the beginning of the manuscript, so it’s highly unclear how most of the proof steps relate (if at all) to previous literature.
*At least in the initially published manuscript. More citations have been added since yesterday, but it still has only 22 references in total.
I was just about to post on this. It seems people from OpenAI even threatened the career of Tristan Buckmaster (a mathematician who was working on the problems, and who came up with related solutions that he asserts OpenAI may have cribbed from; Buckmaster had human collaborators, but also used LLMs extensively) when he refused to go along with their version of events.
Article about the controversy here.
Buckmaster’s statement on the same here.
It looks very plausible that OpenAI (the human organization who takes authorship responsibility for the paper) are showing scholarly irresponsibility, at the least, and possibly plagiarizing. If so, that’s definitely bad (very bad if the claims about threatening someone’s career are true), and to some extent points to an incentive problem with AI companies doing pure mathematics: they’re not that incentivized to respect scholarly norms.
But I’m not sure what that has to do with AI, per se. Failure to give appropriate credit is a longstanding issue in scholarship, and any tool can be misused. OpenAI could have taken a day or two to properly cite and reference, and the ultimate result would still have been new and extremely significant.
I mean, to some extent, this is just a matter of what you find most interesting or important. My own reaction was “My god, people have used AI to solve a Millenium Problem (and, by the way, AI companies need to get less irresponsible in their scholarship if they want to sustainably contribute to academia)”. But if your response is more “The biggest issue in AI is that it enables plagiarism (though, sure, I guess it is also solving Millenium Problems)”, then there’s nothing wrong with that per se.
I have never claimed that this is the biggest issue with AI or that it somehow overshadows the importance of the proof of the Navier-Stokes problem. My claim has just been that this kind of unintentional plagiarism is a significant problem—and it can be hard to detect. A few days ago, I reviewed an obviously AI-generated paper that basically regurgitated and synthesised three several decades-old articles that I happened to be very familiar with (without citing any of them). The other referee didn’t notice this (but still recommended rejection, because it wasn’t a very good paper in any case), nor did the author, I guess.
I think AIs often genuinely don’t know where they get their own ideas from. They mix freely from many different sources without necessarily knowing or caring how those sources are used. If you ask them for sources later, they may or may not find the right ones. Yes, something similar can happen with human scholarship, but it’s not nearly a problem on the same level.
Going through the whole 165 pages of the OpenAI manuscript to see where, exactly, various proof ideas may have come from and how various insights in the manuscript relate to the preceding literature would probably require significant specialized expertise and be (I would guess) a very non-trivial exercise that may not be doable in just a couple days.
One more thing: yes, ChatGPT solved the Navier-Stokes problem, but the proof (to my understanding) significantly builds on recent human work. The proof strategy pursued by ChatGPT is (according to Astra at least) due to Cordoba and Martinez-Zoroa. If that’s true (and I don’t know if it is), then they may deserve the lion’s share of the credit for the solution, but they were not even cited at all in OpenAI’s initial manuscript.
“I think AIs often genuinely don’t know where they get their own ideas from. They mix freely from many different sources without necessarily knowing or caring how those sources are used. If you ask them for sources later, they may or may not find the right ones. Yes, something similar can happen with human scholarship, but it’s not nearly a problem on the same level.”
I think it’s pretty common, and problematic, in human scholarship, as the literature expands, given that scholarly credit requires temporal priority but not causal priority. (That is: it doesn’t matter if you came up with the idea yourself, if someone else came up with it first, you have to cite them.) AI might be worse at it than humans, but AI also provides good tools for ameliorating it (as your own use of Astra here demonstrates.)
The draft AI policy for the journal I edit (Philosophy of Physics) includes this paragraph: “the nature of large language models means that the risk of reproducing an existing idea is greatly increased if you interact with generative AI in developing the ideas in your paper. If you do so, it is imperative that you conduct a careful and thorough literature search to ensure that you are not reinventing the wheel and that you give proper credit to the person who first came up with the idea.” So I’m certainly not trying to deny that there are real issues here.
“Going through the whole 165 pages of the OpenAI manuscript to see where, exactly, various proof ideas may have come from and how various insights in the manuscript relate to the preceding literature would probably require significant specialized expertise and be (I would guess) a very non-trivial exercise that may not be doable in just a couple days.”
Fair enough: maybe a couple of days is an underestimate. And to be clear: I’m assuming you need a human expert as well as AI assistance. I would expect that right in the middle of my area of expertise, and with AI help, I could do an equivalent task in roughly that length of time. But perhaps mathematics is different here from even very technical philosophy.
The last paragraph here is important. More significant here is that a bloody Millenium problem has just dropped, and we now have a much deeper understanding of how fluid behaves, and by extension, of a key part of the physical world. Let’s zoom back to the early 2000s when the most recent Millenium problem was solved (Poincare). Back then, there was also a big brouhaha and drama around who gets the credit for something this monumental. This was on two fronts. Perelman himself felt he shouldn’t get a Field’s medal because Hamilton deserved it at least as much. While, Shing-Tung Lau’s team tried to get credit for producing the first ‘full’ proof (this would be analogous to whomever put the OpenAI teams’ proof into Lean claiming credit as having the first full proof). Anyway, the point is — back in the early 2000s, the true bombshell in terms of significance, wasn’t the squabbling and the blame game in connection with ‘who gets the credit’ for Poincare – it’s just that, bloody hell, Poincare FELL, and that’s a huge breakthrough for humanity’s understanding of topology. As I see it, it’s more or less the same kind of thing going on now, albeit with some computers thrown in, and what deserves the real attention in terms of salience is that Navier-Stokes FELL.
Sadly, that is not necessarily the case. I urge you to read Terence Tao’s thoughts on how these sorts of AI proofs can actually be detrimental to mathematical progress (he wrote this several days before OpenAI’s proof):
https://mathstodon.xyz/@tao/117207849921390904
Note that this is a hypothetical about how a bad AI proof of N-S might harm progress; I haven’t yet seen Tao comment on whether the recent proof fits that framework.
Which is not at all to deny that Tao raises serious concerns both here and elsewhere. I thought this comment was insightful, and speaks to some of the issues we are discussing upthread:
“What we are seeing here is a consequence of some very deliberate choices to abandon any pretense of gaining human understanding – for instance by not working with domain experts, or writing up papers to submit to peer review – and using AI tools for the sole purpose of achieving a benchmark. Such a focus on benchmarks was relatively benign back in, say, 2025, when the tools were not capable of solving entire mathematical problems unassisted; however, in 2026 they have become negatively aligned with mathematical progress as a whole.”
Again, I don’t think that’s an issue with AI tools per se, but with their (mis)use.
Yes, it’s certainly possible that things will go well. AI seems on the whole to have had a positive influence on chess and on people’s understanding of chess. One can hope that the same will happen in the case of AI and math. But, as Tao has also been pointing out, whether it will may depend on the course of action taken by both the AI labs and the mathematics community. It’s not automatic.
I folllow Tao closely and the above is not representative of his wider positions about AI in math, in which he is a pioneering optimist , see here https://www.quantamagazine.org/how-terry-tao-became-an-evangelist-for-ai-in-math-20260608/
I was pointing out that your claim that we now have a “much better understanding of fluid mechanics” may not be correct. There are now several examples of AI generated proofs that have produced surprisingly little insight. Tao has pointed that out as well. Whether this is “representative” of his overall take on whether AI will have a positive impact on math is beside the point in my view. (It seems to me he has a nuanced view, which is what I think reasonable people should have, although lately he’s mostly been concerned with pointing out the (neglected) downsides.)
I’m a little flabbergasted by this attitude. It seems quite casual given the issues at stake. But maybe I’m misinterpreting you. In any case, here’s this: https://mathandai.org/
Plagiarism is a pretty specific kind of act (copying without credit). Merely being inspired without giving credit isn’t plagiarism (though it might be an uncharitable thing to do). OpenAI does credit Alpoge and Buckmaster (here). And OpenAI claims that they only resolved a different problem (forced Euler). OpenAI states:
Still, the exact relations between OpenAI and Alpoge and Buckmaster probably won’t ever be clarified.
I think I was fairly careful with when I did and didn’t say “plagiarism”, which I agree is quite a specific thing. But if you prove a result and intermediate steps you take have already appeared in print, you have a scholarly obligation to cite the originators (whether or not you came up with it independently). Whether that happened here, I don’t know for sure: the news reporting is too chaotic for now and I’m not nearly familiar enough with this corner of academia to check for myself.
From my own experience of using AI, it is highly plausible that AI is falling short of its scholarly obligation. Astra Pro has written a paper for me with virtually no citation (while I have explicitly instructed it to make sure that it be a high quality paper and even set an example to beat). It is their tendency somehow.
AI doesn’t have any scholarly obligations; you do.
I thought it was clear we are talking about Ai as the author. As in “ChatGPT solved…”. I am nowhere involved.
Maybe “for me” is misleading. I merely meant by it as “requested by me”.
My point is that – at least for now – AI is not a person in the morally thick sense relevant for (e.g.) incurring scholarly obligations. If you ask AI to generate some material, in itself that’s just your private use of a software tool. If you then choose to submit that material to a journal or other public forum, it’s on you – and not on “AI” – to make sure that scholarly obligations are met.