In what is being called “the most important event in the history of mathematics” (by an AI), OpenAI, using an advanced model not yet available to the public, yesterday released 722 mathematics manuscripts containing 337 results it has generated on open mathematics problems.
“OpenAI deluged mathematicians with hundreds of new findings that span a wide swath of topics including algebra, number theory, theoretical computer science, mathematical logic and topology,” the New York Times reports. Adding insult to injury, “the average result took about three hours of computing, the company said.”
One AI engineer on x.com put it this way: “Reasoning models are two years-old. In that time they went from incapable of basic arithmetic to solving problems humans couldn’t solve for decades.”
What were the results like? Computer scientist Scott Aaronson shares some of his thoughts and those of his wife, complexity theorist Dana Moshkovitz, here. Moshkovitz says: “It feels like something written by someone who’s on psychedelics…” and “so horribly written that it’s impossible to read it without AI help…”, but: “of course there’s a lot for us to learn from the aliens”.
After mathematicians raised concerns about an earlier breakthrough on the Navier-Stokes problem by OpenAI, the firm formed an mathematics advisory group to advise it on the “review and communication of emerging results.” Their job is to
help OpenAI assess their significance, advise on how to coordinate their dissemination, and advise on academic and professional standards of mathematical research… The group will operate independently from OpenAI. The group will have the freedom to offer advice we have not requested, comment on OpenAI’s impact on mathematics, and make its advice public.
In the wake of the new results,
the advisory board released a statement that called the public release “the beginning, not the completion, of the process of human understanding and the incorporation of the work into mathematical knowledge.” “We want to create standards and practices so that results released from A.I. labs can be understood by mathematicians and can advance the field,” Melanie Wood, a mathematician at Harvard who is a member of the advisory board, said in an email
The advisory group also called for the release of “all of the prompts to the A.I. agents and the agents’ chains of thought.”
Mathematicians were already wondering about the future of their field in light of AI’s capacities. It seems like the discipline of mathematics must “change or risk extinction,” as Jordana Cepelewicz puts it in a recent piece at Quanta.
Readers of Daily Nous might be wondering about whether philosophy will face a similar reckoning. And indeed much of what Cepelewicz says about mathematics—especially on what seems to be missed when machines provide us with the answers—could be said about philosophy. Here are some examples:
Many people don’t know what mathematics really is, or why mathematicians do it…
I always found that “pure math” — the study of mathematical concepts for their own sake, without a care for real-world applications — existed somewhere between the sciences and the arts. It prizes logic and certainty, but at its core lie fuzzier notions of beauty, intuition, and depth. “Math is either the most science-y humanities or the most humanities-type science, depending who you ask,” said Marcel Goh, a doctoral student at McGill University…
In mathematics, it’s not a cliché that the journey matters more than the destination. Problems are posed not so much because their answers will be practical and important, but because they represent interesting journeys. The hope is that as mathematicians struggle to solve a problem, they’ll come up with intriguing tools and connections, stumble on novel ideas and directions, take fruitful detours, and answer new questions they never would have thought to ask….
Mathematicians have always known that understanding is more valuable than an answer…
The whole piece, worth reading, also discusses some ways that academic mathematics will likely have to change.
Still, it is not obvious that philosophy is susceptible to the same kind of threat that mathematics is currently dealing with. While there are no shortages of open problems in philosophy, one might nonetheless say: philosophy is not in danger of its problems being solved by AI because its problems cannot be solved.
I think such a response is both too pessimistic and too optimistic. It’s too pessimistic because some of philosophy’s problems are at least in principle solvable. Examples: Is principle P compatible with judgment J? Is argument A valid? What are the implications of belief B? Which theory of X is most compatible with this particular theory of Y and current science? And so on. It’s too optimistic because it seems quite likely that some future form of AI will be able to answer these kinds of questions (and similar ones with scopes too wide and variables too numerous for a human mind to keep track of). Generally, most of the philosophical “solutions” that humans offer are the consequents of explicit or tacit conditionals, and I would think that AIs will be quite good at conditional reasoning. (See “Hey Sophi“.)
Tomorrow there will be a guest post about how AI might change philosophy, but I wanted to give people a chance to discuss the mathematics results, and perhaps the relevant similarities and differences between mathematics and philosophy.
Well, firstly, congratulations to OpenAI. Beyond that, my hot take is this: the math community is taking the line that “understanding is more valuable than isolated truths”, and they take that to be a premise in an argument whose conclusion is that OpenAI shouldn’t be pumping out these proofs. Well, that argument’s invalid! Here’s why. Even if the premise is granted, it remains the case that these isolated truths are better than nothing. So what we got from OpenAI is a bunch of proofs that are less valuable than rich understanding and more valuable than nothing. The relevant contrast though here is: take the proofs or don’t. So it’s between the epistemic value of isolated truths and the epistemic value of lacking them. So, ‘on you go!’ to OpenAI, let’s have some more (and bonus points if they help us gain deep understanding).
I should also add. The “Math’s a journey, not a destination (cf. Aerosmith)” folks shouldn’t try to say “Open AI took a short cut to the destination, eliminating the possibility of a valuable journey”. Bad reasoning because there isn’t just one journey to take. Another way to look at things is: these new results we’ve got are ‘stepping stones’ that invite us to take new journeys, new journeys we might not have been in a position to take without these new maps we’ve been given.
I don’t think we should be dismissing Terrance Tao’s argument so quickly. I take it that there’s partly a concern about how results like these are going to impact the sociology of mathematics. If humans had been the ones tackling these problems, they would have had to invent tools and techniques that would have both led to increased understanding and opened further avenues of investigation. But with results dropped down from the machines, we get the answers without any of the understanding, and without any of the tools that we would have developed, had we done the job ourselves.
I take the Tao position to be that, just as it’s detrimental to student learning to be provided with the answers to the test, without having to develop the skills to answer them themselves, it’s detrimental to the practice of mathematics for mathematicians to be given the answers, without having to develop the skills to answer them themselves.
It’s detrimental in both cases for the same reason: the goal isn’t simply the accumulation of correct answers. Aiming at getting correct answers has been a reliable means to achieving the goal, but the ready supply of correct answers can make that aim no longer a reliable or the best means to achieving the goal.
You seem to assume that knowing more truths is inherently valuable (a common assumption indeed, also made in the most recent philosophy talk I went to). But people who prioritize understanding will perhaps consider AI dumped proofs as counterexamples to this assumption.
The sense in which these results are “known” is also in question. A majority of these results lack formal verification. It seems reasonable to think that until a proof is completely digested by at least one human, the result has not been known. So even if you think knowing more truth is inherently valuable, this value is not realized by result dumping but by digestion.
Let’s put aside all the other results and all the other changes due to AI. Suppose that tomorrow a convoluted and unintelligible proof of the RH dropped from heaven. Would we be better off, epistemically?
IWe should distinguish two senses in which we could be better off, epistemically. We could be better off than we were before, or we could be better off than we would have been without it.
I think that we’d clearly be better off than we were before. I also think it’s likely that, in the short term, we’re better off than we would have been without it. But over the medium to long term, it’s not so clear to me. It’s possible that we’d lose out on lots of deep understanding that we would have gotten, if we’d tackled the problem ourselves. And I think that it’s the counterfactual losses over the medium to long term that Tao is worried about.
OK, i’ll grant your distinction – and so it would seem (on that granting) everything now hangs on the medium-to-long-run counterfactual. So, i’ve got two thoughts on that.. First, what’s the comparison world supposed to be? We have tackled RH ourselves, for 167 years. So the world without the heaven-sent proof is one where we keep at it for some unknown stretch (possibly forever?) rather than one where a lovely human proof turns up on schedule. And in the world with the proof I doubt mathematicians are going to just shrug and go home. “Why on earth does this work?” probably becomes the most tempting open problem in the subject, and we get to attack it already knowing the thing is true and with a (horrible) route to study.that’s a world ripe for understanding too.
a second point: we’ve sort of run this experiment already. Ramanujan said his formulas were handed to him by the goddess Namagiri, which is about as close to flat out dropped from heaven as it gets, and plenty of them arrived with no intelligible justification (partly due as far I understand on the idiosyncratic intuition he had). Mathematicians spent the next century working out why they were true and got a lot of deep theory out of it… and I’ve never heard anyone say Hardy should have sent the letter back, eh?
I don’t think that *everything* now hangs on that counterfactual, because of everything else that I put aside at the start of my last comment.
In terms of its disruptive impact on a discipline, there is a difference between a single mathematician like Ramanujan dropping conjectures at a human pace, and swarms of AI agents proving results faster than the experts can make sense of them.
That one in particular probably would be more good than harm. But it’s very different to drop one result than to repeatedly drop large numbers of results on the field.
Dropping a single result mainly just has the effects of knowing its truth. But repeatedly dropping large numbers of results has a secondary effect of shifting the incentive structures in the field, which affects future work.
But in the case of Navier-Stokes, it’s not like the result itself actually added much to the techniques that were developed over the past few years that made it possible. If humans would have developed more techniques in working on this, it might have been negative. If the AI result makes those techniques easier to find also, then it might have been positive.
As I understood the criticism, it was less that mathematics is a journey and more that in the usual practice, solving an open problem yields more fruitful avenues for research and exploration, and AI might solve the problem (maybe) but leaves the rest of the field barren.
In these discussions it is important to distinguish *mathematics/philosophy itself* versus *the current academic institutions that specialize in the study of mathematics/philosophy*. By definition mathematics is infinite, so it is not possible to exhaust mathematics for humans or machines. Philosophy arguably has a similar character. So indeed in principle, strong-AI can give us more powerful ways to navigate these vast landscapes, which can coexist with alternative, more artisan ways.
On the other hand, the institutions (journals, hiring committees, universities, etc.) and the associated value systems definitely need to adjust and respond to the AI urgency. We should think about how our institutions should be arranged in such a way that humans can remain engaged and flourished in these meaningful intellectual activities in the presence of strong-AI.
People have every reason to worry about this, especially in light of the material organization of current AI systems (large scale systems built by accumulating hundreds of billions dollars and controlled by a few companies), and the anti-intellectual ideology that often correlate with AI discourse.
Without properly addressing these conditions of actually existing AI systems and academic institutions, the naive AI-foward position, the naive anti-AI position, and the defeatist position all seem to me to be unhelpful.
I’m not sure I understand why the first distinction is important. Did anyone believe the problem to be that AI might literally exhaust an infinite space of problems? I figured that everyone’s concern all along was that it could exhaust the finite space of problems *we care about*, or perhaps even that it could solve any arbitrary problem *we come to care about*. Or am I missing the upshot of the distinction?
I think the space of problems that we care about at a given time T is arguably finite, but is potentially infinite in the sense that for any size N there is a future time T’ > T such that the size of the union of all problems that we had ever cared about up to T’ exceeds N. This is because once a specific problem is solved, it often enlarges the space of problems that you care about. In fact, arguably this is the very sign of a promising research program.
As for the possibility that *any problem that we come to care about* can be efficiently solved, this question should be discussed elsewhere because it’s complicated. Since there exists problems that no computers can solve, someone can say they care about *these* problems, and this would falsify the possibility. Of course, typically people don’t care about such problems, the point is that more careful work is needed to make this hypothesis more rigorous.
I’m not sure this addressed my question. You said it’s important to distinguish math itself from the study of math, presumably because “by definition mathematics is infinite, so it is not possible to exhaust mathematics for humans or machines.” But, again, I don’t think it’s at all plausible that anyone worried about AI “solving math problems” meant that it would, per impossibile, literally exhaust the infinite space of problems. So, I fail to see why this distinction is important. Similarly, I guess I took it for granted that when I said “any problem that we come to care about,” my quantifier was tacitly restricted to exclude Godel sentences and the like. Like, what is the overly-literal distinction-making achieving here?
(Notwithstanding my confusion, I agree with the other stuff you say about the adjustment of values, etc. I’m just not seeing how the infinite/finite distinction is resolving a problem or confusion that anyone actually had.)
Some prominent mathematicians have indeed claimed that good problems that invite understanding are scarce, and are at risk of being exhausted by AI. I personally isn’t convinced by this claim and I’m more optimistic about the realm of human curiosity. Also, there are just numerous mathematical questions related to the empirical world that can be postulated but are vastly complex. My point is that our worry should not be mathematics or philosophy being “solved” and so mathematicians and philosophers have nothing to do (a narrative popular among AI ideologues), but rather how should we arrange our practices so that these journeys continue. For example, I think even if humans end up out-of-the-loop from mathematical discoveries, there will still be interest and value in human understanding of mathematics. How can we ensure people have these opportunities in the presence of strong-AI? (I’m not sure if there is an option of retreating to “digestion of philosophy” though, and I’m curious what others think!)
This probably isn’t the sense he’s mentioning, but it’s also relevantly important to distinguish the value of knowing some particular truths from the value of developing the institutions that enable truths of this type to be known. If we exhaust the space of problems that are ready and able to serve as nutrition for the continued health of the field, without exhausting the space of problems we care about, then it could be directly negative (unless it replaces that field, and still provides the other goods that the field provides beyond knowledge of some abstract truths).
Philosophy isn’t just about making philosophical arguments. AI (both LLMs and neural networks more generally) definitely have groundbreaking applications in history of philosophy. The corpus I’m currently working with contains 170 000 pages of edited text and 5000 manuscripts – all mixing text, diagrams and equations in 3 different languages, scattered over hundreds of editions and translations. Needless to say, I would never have been able to engage with Leibniz’s and Euler’s work to the extent that I’m able to today without agentic AI tools and neural networks for indexation and retrieval.
LLMs also have another groundbreaking use for us lovers of wisdom : they allow anybody with an interest to learn much faster on just about any topic.
*for certain values of “learn”, given what we know about deskilling
**to the extent that their outputs can be trusted when unverified, which they clearly aren’t in this case
One of the things I realized in the past 2 months is that the accuracy of AI models is entirely dependent on the conditions under which they run : what’s the prompt? What model is used ? What are tools can be used? What’s the reasoning effort level? How are sub-agents used? Any script?
Humans, including experts, are rarely trusted regardless of what they’re saying.
It’s also worth noting that Open AI didn’t produce these math results in a vacuum : the company hires teams of scientists and mathematicians since its early days. As far as I know, Open AI didn’t publish the process and prompts that lead to the proofs, but I’d bet their mathematicians were heavily involved in testing and setting the tasks up.
“[T]he journey matters more than the destination.”
I remember when the producers of LOST said this after its finale failed to offer a satisfying resolution to the core mysteries of the show. So… yeah, you could say that I’m skeptical! Yeah, I’m thinkin’ you could say that!
“It feels like something written by someone who’s on psychedelics…” and “so horribly written that it’s impossible to read it without AI help…” are not assessments that inspire confidence in me, let alone awe. Quite the opposite.
Dumping 722 manuscripts at once sounds suspiciously Gish Gallopy, especially if the assessment quoted above is true of all or most of them.
Because it’s hard math, it will take proper experts a long time to unravel each one and check it thoroughly. And in the meantime the Gallop will stand, and even if a few of its threads get unravelled, OpenAI will just point to the tissue of threads which have yet to unravel and say 431/722 ain’t bad.
I don’t think this is a gish gallop in any traditional sense. These are proofs in Lean. They are very hard to fake. Maybe they exploited a bug in Lean, or maybe they slilpped something in that isn’t detected, but I gather Lean is pretty good at telling you what assumptions are being used. (If it is a bug in Lean, that would be pretty interesting in itself, though not the result that was intended.)
It’s more that we’re *so* far from understanding any of these. And the usual thing we have to assist with understanding, the writeup, is proving of no help. Given how much text ChatGPT has ingested, I’d have hoped it had learned the basics of how math papers are written. But it really hasn’t. It’s more like a brute force proof-finder than I expected.
Only about 20 percent of the results among the 700ish in the recent release are formalized.
If the headline had been “OpenAI released 140 proofs”, I don’t think this conversation would be meaningfully different.
These proofs in Lean (or any proof assistant or theorem prover) are only useful for present purposes if the theorems proved are the intended ones. Semantic parsing is not a solved problem, even if it’s arguably easier for the relatively homogeneously structured texts in mathematics. Even if they were stated by humans, I’d want to have some good evidence they are stated correctly.
I suspect that in the past few days, relevant experts on each problem have looked at them, and any obvious mis-formalizations of the problem statement would have been noted.
TL;DR: The robots are coming for philosophy too, but philosophers are not prepared. Either we adapt, or academic philosophy perishes.
Much of the social structure of math as an academic discipline died today. That’s what I hear mathematicians talking about. There were credibility structures (e.g. PhD admissions, hiring, tenure), as well as career-long research programs. Those are all dead now. Junior mathematicians realize that they cannot carry on as they did before. Nobody’s going to spend their grad school or assistant prof years trying prove P, the risk of getting beaten by AI is too great. And what I’m seeing anecdotally (I’m not aware of any hard data) is that mathematicians are adapting: they’re very heavily using AI.
One of the main positive results is that AI provides research-level expertise in fields
where there are often very few experts. More than one leading set-theorist has told me that the main problem with set-theory was that so few people were working in it. It was too hard, and too risky for junior people. But now the robots can help. Similar points can be made about Langlands. AI has already ended the modern age of math; it might usher in a real golden age. But people who fund mathematics are already asking: if robots can do math, why do we need human mathematicians? There’s grave danger there. Today mathematicians realized that either they adapt to AI or their discipline ends.
My great fear is that this revolution is coming for philosophy, but philosophers are not prepared. Worse, many philosophers refuse to prepare. Witness the persistent denialism and AI-hatred on this blog. Of course, many philosophers are doing fascinating work with and about AI. AI companies realize that philosophy is hard. So that’s what they’ll set their sights on next. OpenAI is giving selected academics free access to frontier models. I hope philosophers are applying for these licenses.
I would like to make a distinction between AI-denialism and AI-hatred. Many people do not deny the power of AI, or where it seems to be going, but are not too fond of the trajectory. Typically these people are called Doomers, which is a slur invented by the Accelerationists pushing this technology. Though, to be fair, there are plenty of people who exist at the conjunction of denialism and hatred, typically they are called the AI pragmatists / AI critics / AI ethics camp
Yes, those are good distinctions, and there are probably lots of other crucial distinctions at work here. We need to think carefully about these issues. (Of course, that’s not going to be done in blog posts or comments. But there’s lots of work for us to do here.)
And here we are at it – again.
Assuming that the machines progress as they did so far, no amount of preparedness will do since they will take away what was meaningful in what we do anyway. We can stick around to ‘understand’ their results, like a zealot who is trying to figure what their God is telling them. That’s about it. This is not a role I aspire to, so I suppose I will then just pack it up and quit, I see little point to spend my remaining time with preparing for this, Thank you, but no thank you.
This is not because I am against AI. It just seems entirely rational to me to not adapt. If there are a bunch of supermen hanging around you don’t want to play the hero. You let them have their fun and do your little things in your room for yourself.
What I do not understand is what is there to celebrate about this. Wiping out human intelligence (possibly anyway) and the chance for meaningful work is hardly a cause for celebration.
Just another remark regarding who the next target of AI companies will be (already the wording is menacing btw). I don’t think it will be philosophy since these companies, whatever they say, are driven by profit. What they need are tangible results and while there may be some such in philosophy, there aren’t many. Most ‘results’ the companies could produce would just lead to another blog post on Daily Nous and this is not what they need. This won’t make their value go up.
My guess is that they will go after science, medical research, and so on, where there is money to be made easily, directly or indirectly.
But of course some of these people are indeed obsessed bringing their God alive, so for those philosophy could be a good target.
Heaven forfend if those wicked capitalists use their terrible machines to bring about transformative advances in medicine.
That is one take on the matter. When they indeed find the cure to cancer(s) and they make the cure widely available, I am happy to discuss it again. I venture that this won’t be any time soon.
It would be nice to see a cure for war or poverty. Can they be any harder than cancer? (However I wouldn’t dare attempt to use AI resources for this: AI’s solution would undoubtedly be to hook all the humans up to pleasure machines, since humans no longer care about doing philosophy or music or math anymore, and this way they’ll stay out of trouble.)
More seriously, I think what these companies will go after next is governance, and that they and their gang will attempt to persuade the population that AI (not democracy) is best suited to reflect the general will.
I think all of these are beset by collective action problems whereas medical research is not (or much less). Also, all these other areas involve a myriad of evaluative and normative judgments, whereas medical research does not (or to a lesser extent).
Also, we should not confuse what AI might do (if it can ‘do’ anything) with what its masters (should there be any) might do (and their possible collaboration, too, would be beset by collective action problems).
I think medicine is a good choice because there is money in it as well as glory, both practical and moral and while collaboration is involved, it is relatively easy to coordinate since everyone has the same aim. (Of course, there is tension and there is competition, and there are economic interests in the way, but this does not differ from what we have now.)
The economic-legal-political facts about who owns what and who owes what to investors are things to consider when floating rosy musings about the material significance of such advances.
If Anthropic cures most forms of cancer and neurodegenerative illness, I am very happy for them to get rich off doing so.
Oh, me too. As long as they’re also incentivized to make that cure meaningfully available to those who need it.
The patent system is reasonably effective at making medical advances widely accessible with a bit of a time delay. (GLP-1 patents expired this year in some countries and will expire worldwide by the early 2030s.)
There’s some prospect that cancer cures will be individualized to the degree that the marginal cost of each person’s drug remains high, and then I agree it will be a challenge to work out how to make them widely available. But that would be a nice problem to have relative to the cancer medicine status quo (and also the sort of problem that AI is good at solving).
None of this, though, erases the incentive to rent-extraction. By “meaningfully available” I don’t mean “present in most places.” I mean something more along the lines of, e.g. “available without having to take on massive debt,” or “without having to choose between living and having a place to live.”
This is probably getting further into a general discussion of healthcare economics than I’d like (I try to avoid discussing general politics here) so I’ll resist replying further to this line of discussion.
And yet…
To paint concerns about the material effects of the structure of ownership as concerns merely about getting rich is to straw-man. But I wasn’t clear in my comment, so that’s my fault.
I am actually not, but sure, if the choice is between them getting rich without developing the cure and them getting rich while developing the cure, then I also opt for the latter. The sad nature of our present predicament is that these might be our only two options. (Yes, I think that all these technologies should be commonly owed and not belong to profit-driven companies and nationalistic governments. But of course, this is not realpolitik.)
And if the choice was between them not getting rich while not developing the cure, and them getting rich while developing the cure?
still the latter of course but I didn’t consider the first as a realistic option given that they are already rich
Suppose that an alien spacecraft arrived at Jupiter and we started communicating in radio. The aliens appear friendly but mostly are too weird for us to communicate, but they can basically understand our math. If we send them a precisely stated conjecture, they can try to tell us if it’s true or not. It takes about three hours for them to reply (light-travel time, mostly) and 90% of the time they don’t reply at all, but 10% of the time they say ‘yes’ or ‘no’. When we ask why, we mostly can’t understand their answers, but there are some intriguing hints, and often they produce a human-unintelligible proof that we can formally verify as correct.
This ‘alien oracle’ model is, as far as I can see, roughly what the current state of AI use in math is. (There are some reports that OpenAI threw around 8,000 conjectures at its model, and ‘only’ proved 700-odd.) A tool like that will obviously be transformative, but it doesn’t end math, and it doesn’t meaningfully change the need for human mathematicians. Math is to a substantial degree about formulating concepts and developing understanding of abstract domains (and the application of math in the sciences is largely about deploying those concepts and using that understanding). Some of the proofs mathematicians are most interested in are of conjectures that we are morally certain are true. (No-one really doubts that the Riemann Hypothesis is true; no-one really doubted that the Navier-Stokes equation had regular solutions that evolved to singularities.) Even when we don’t know the answer to a conjecture, even getting to state the conjecture and to understand why it’s important requires a huge amount of mathematical work. (Try reading the statements of the results that OpenAI proved and see how many you understand. I understand almost none of them outside mathematical physics, and that’s not just a shallow matter of vocabulary: understanding enough mathematics to understand the statement of most of these theorems is really non-trivial).
It’s instructive to note that the alien oracle would be almost useless in philosophy. In classic philosophical questions like “is free will compatible with determinism?” or “is knowledge justified true belief?”, pretty much all the work is being done by how the concepts are understood. Even if we asked it something like “does human consciousness have an irreducibly first-person character that cannot be accessed by the third-personal methods of science” and it said, “no, what are you smoking?”, no advocate of that view of consciousness would be at all persuaded.
That said, who knows where AI is going? Once it is capable of producing a *complete research output* that is comparable to, or better than, the highest-quality human outputs, things change much more dramatically. If you can say to Claude Opus 9, “write me a series of academic papers that advance our understanding of functional analysis as much as von Neumann’s work did”, or “write me a monograph in analytic philosophy of comparable quality and significance to ‘Naming and Necessity’” and it churns away and produces something that if human-written everyone would agree was seminal… well, then things change, and then we can start talking about whether humans are becoming obsolete in these areas. But I don’t know if that is a year away or a millennium. The asymmetry between philosophy and math is that I don’t think AI could be transformative (and not just useful) in philosophy unless it got that good; it is already transformative in math.
[cross-posted at Leiter Reports]
The problem with all this is that (1) we indeed do not know pretty much anything about the progress of AI and (2) no one asks us – in this case, all of humanity – whether we want this to be done, whether we want AI to progress (any further). A handful of private companies, two superpowers’ leadership – this is roughly the setup for potentially civilization-forming (and ending) decisions. In the meantime we can wait for the newest released AI model to decide whether we will still have something meaningful to do tomorrow.
None of what I wrote was normative.
My normative view. FWIW, is that current-level AI would in the medium term be good for math, even if there are growing pains, and that the same is true in other academic fields, but that catastrophic / existential risk to human civilization from next-level AI is real and justifies a lot of caution.
I didn’t suggest that. I was just simply adding a political consideration to your remarks concerning the future of AI. My view was normative (it is wrong how AI is being developed since the all-affected (and preferably, also the all-affecting) principle should guide this development. But I didn’t say you implied anything normative.
Thinking AI has a remotely viable chance of ending civilization is silly. It’s released a few hundred math proofs, most of which have been appraised as being gibberish by professional mathematicians.
at no point did I suggest AI would end civilization, in fact, I didn’t even argue against AI. I have made point about the political (and economic) dimension of AI development and earlier I’ve expressed puzzlement regarding the (seemingly) expressed welcome of the potential effects of further AI progress.
But ending civilization was not on my list. (I think climate change will end civilization, but that’s another matter. Many in these discussions on the other hand seem to think that AI is more dangerous. I am actually not one of them, although I can see dangers mainly connected to humans’ misuse or abuse of AI and resulting accident (e.g. nuclear).)
2020: It’s silly to think AI could ever end civilization – it can’t even do basic arithmetic!
2022: It’s silly to think AI could ever end civilization – it can barely even solve Math Olympiad problems!
2024: It’s silly to think AI could ever end civilization – it gets under 2% on FrontierMath’s list of unsolved math problems!
2026: It’s silly to think AI could ever end civilization – it can’t even explain its partial proof of the Riemann Hypothesis in a human-readable way!
2028: ??
Yes doing mathematics leads to the end of civilisation, this why we hunt down rogue mathematicians and kill them. Why don’t you explain how your sequence will end in the End of Civilization?
I’m not saying anything that hasn’t been said a hundred times by people (including me) just in the last few weeks, but: it would be highly dangerous to have agentic systems that are superintelligent in many cognitive areas and that we control imperfectly; there are lots of axes by which this could (not will, could) lead to catastrophe; evidence of superhuman capacity in math is both evidence of the current AI training paradigm to develop AI that is superintelligent in increasingly many ways, and evidence that recursive self-improvement might be possible (CS is formally verifiable in similar ways to pure math, and so trainable in similar ways).
Theoretical computer science is (IMO, a piece of exact metaphysics!). But working software development (and hence cyber security, for example) is *not* so. This is why one can do *some* of software development with the assistance of the currently hyped robots, but by no means all. It remains to be seen whether the improvements in 10-15% of the work (estimates by some of the best respected software developers and architects in the world) will not be overshadowed by the utter disaster that is the rest. Like the math proofs (which I am enjoying reading about by way of comparison) there is a real danger in being overrun with garbage. I should point out that the differences in models are interesting here – it is one thing to have careful academic study with ‘frontier models’, it is another to flood the bug bounty programs to the breaking point with garbage from the free ChatGPT directly. This has already (temporarily) cancelled even Alphabet’s – and if one of the largest companies in the world can’t handle the flood, what will happen to the rest of us?
Notable scientists and futurists who argue that there’s a chance and that it is worth preparing against do not base their arguments on the fact that it has done some math proofs.
The math proofs aren’t reassuring, though.
Your point about concepts is interesting. But “how the concepts are understood” seems to imply that they are ambiguous or vague or otherwise unclear. Maybe the AIs start to introduce really fine-grained distinctions, along with axiomatizations. Or maybe they figure out new ways to clarify our concepts. Or they can do massive literature analysis to reveal patterns in conceptual usage that we’d never find. Who knows.
A good example here might be “Here are 400k images of paintings in museums. What do they all have in common?”. Sigaki, Perc, and Ribeiro did that (mostly) by hand and got a really clear information-theoretic answer. So maybe the robots can do this sort of conceptual anaysis at scale.
Or perhaps they’ll start to cross-correlate lots of psychological and philosophical data. The robots could do factor-analysis to figure out how human conceptual systems form clusters (e.g. free will libertarians vs determinists) in very high-dimensional spaces where the axes are thousands of concepts from psychology, the sciences, literature, etc.
We might learn something about how concepts work by looking at how they get vectorized in LLMs and how attention works in LLMs.
This all sounds reasonable. I mainly worry about the “where is it now” question because of what it portends for the “where will it be” question. And I worry about the “where will it be” question because, personally, I like to feel as if I’m contributing something significant to philosophy (or whatever). But the future I fear is one in which I contribute to philosophy roughly what a child contributes to art with finger-paintings and the like. Maybe the ASI will put my work up on the fridge.
AI will never replace your invaluable contributions, Meme.
“It’s instructive to note that the alien oracle would be almost useless in philosophy.”
I’d say the argument given to support this claim only applies to a subset of the body of knowledge that belongs to philosophy. Surely, philosophers in history of philosophy and logic, for instance, stand to gain from AI tools or an alien oracle. I can assure you that I’ll learn to understand and engage with Euclid’s geometry, Newton’s Principia, or Gödel’s proofs much faster with on-demand explanations.
I think the same goes with philosophers who have an interest in mathematics, physics, computer science, or anything complicated : at the very least, such AI tools and alien oracle, acting as private tutors able to explain everything, stand to speed up the rate at which philosophers learn and keep up with scientific advances. Who knows, maybe this could allow philosophy to finally reacquire its ability to contribute to scientific output.
And aren’t linguistics and philosophy of language already contributing to each other’s body of knowledge? Cognitive science and philosophy of mind? Logic and computer science? Political philosophy and political science? I don’t think there’s a clear-cut boundary between those sciences and their respective branches of philosophy. Yet I suspect you would agree that AI tools or an alien oracle could be useful to the former.
I certainly agree that *AI* is useful in philosophy, very much in the way you describe. The “alien oracle” metaphor isn’t supposed to cover all aspects of AI; it’s more narrowly aimed at how to think about the OpenAI math proofs.
They’re definitely better than just “alien oracle” at this point! At least, I was just on a plane flight, and one thing I did was ask ChatGPT to start looking at the proof of the undecidability of Hilbert’s 10th Problem over rationals (which was one of the results dropped here). My ChatGPT was surprised and constantly emphasizing that they didn’t do the “obvious” thing and give a diophantine definition in the rationals of being an integer (which would then reduce the integer case to a special case of the rational case, and thus prove its unsolvability). Instead, they showed how to construct, for any diophantine equation over the integers, a sequence of diophantine equations over the rationals such that some member of that sequence has a solution iff the original one does not. An oracle for rational diophantine equations would then convert this into a decision problem for integer diophantine equations.
There was then a lot of stuff about how to construct this sequence of equations, that involved some tricks from various other number fields, and then showing that solutions would be in the base rationals if they exist at all, that I certainly wasn’t following. But I already got more than just the truth value.
Your claim, ultimately, is that AI is not and could not be useful in way that’s as transformative in philosophy as in mathematics. Right?
My view is that this relied on reducing the set of groundbreaking questions in academic philosophy to a subset of those questions (the conceptual, unclear, ‘philosophical’ questions). But this strikes me as inaccurate. There are plenty of questions in academic philosophy which have more in common with scientific questions than with ‘philosophical’ questions such as “is free will compatible with determinism”.
I’m pretty sure that if Open AI decided to deploy enough ressources (including by hiring philosophers the way they hired mathematicians), they could have a model finish the critical edition of Leibniz’s work, which is ongoing since more than 100 years. That certainly would be transformative for the philosophers who worked on this, wouldn’t it?
“Your claim, ultimately, is that AI is not and could not be useful in way that’s as transformative in philosophy as in mathematics. Right?”
No, I didn’t say that; in fact, I specifically said that maybe AI will get to the stage where it can just spit out books as significant as Naming and Necessity, and then it’ll be as transformative for philosophy as for math.
What I am inclined to think is that *present-day* AI will not be transformative for philosophy in the way it is clearly being transformative for math. I didn’t define ‘transformative’, but roughly I had in mind that the entire academic discipline will radically change the way it functions. Pretty much everyone in math agrees that this will happen there (indeed, is already happening). I don’t think your examples, or others I’ve seen, persuade me that this will happen in philosophy. That’s compatible with AI being very useful in philosophy; indeed, it’s quite useful already. (And my second-order confidence is relatively low here; everything is very new.)
It’s perhaps worth saying that I work very much at the technical end of philosophy of physics, so I’m not advancing a claim that’s only supposed to hold in some ‘pure’ area of philosophy. AI has not, for instance, transformed pedagogy in math and physics: I have found it a very useful, if occasionally dangerous, tool, but it doesn’t shortcut the need to develop understanding and competence, any more than a good textbook or tutor does.
Then that’s setting the bar pretty high indeed. Math (or in any case its subset transformed by AI) could be described easily : it’s about proving theorems. There’s much more disagreement on what philosophy is, so it’s not surprising to me that a new tool doesn’t at once make ‘the entire academic discipline radically change the way it functions’. Philosophy is unusually eclectic.
That said I’m pretty confident that existing AI tools could be transformative in history of philosophy, if only they were adopted. I suspect that’s the same for other subfields of philosophy where the quality of contributions is measured by something more precise than ‘writing interesting things’.
I completely disagree with that description of math. And much of the criticism of openAI by Tao and others is because they disagree, too. No interesting academic discipline has a lossless one-sentence statement of its content.
Because of those transformative uses of AI in math (i.e. proofs of theorems) you’re claiming that ‘the entire academic discipline will radically change the way it functions’. So clearly math is about proving theorems, even if it’s not all there is to it.
Why is transformative use of AI for a subset of math enough for AI to be transformative for the math as a whole, while transformative use of AI for a subset of philosophy isn’t enough for it be transformative for philosophy as a whole?
I say it’s because philosophy is eclectic where math is not.
When all is said and done, and the AIs have turned the whole planet into one big databank filled with quadrillions of precious little facts, and there’s so much lovely truth in fact that only AIs could possibly appreciate or contribute to it, and the atmosphere is boiling, and the last human academic is tossed from their office like the slow, poorly-optimized organic surplus they are… when that happens, well, I’m sure all the AI apologists downthread will be singing and dancing in the streets, right?
This is running together multiple issues.
If the argument is “Stop developing AI (even at the risk of losing transformative advances in the applied sciences) so as to protect the essentially human nature of inquiry”, I’m not terribly sympathetic.
If the argument is “Stop developing AI (even at the risk of losing transformative advances in the applied sciences) because otherwise we will lose control of it and then it will boil the atmosphere and kill everyone”, I’m listening.
With respect to the first argument, my position depends on what kind of transformative advances in the applied sciences we’re talking about.
If it’s 500-year lifespans in perfect health and some kind of optimized induced neurogenesis to keep up, at least partially, with the machines, that’s one thing.
But if it’s increasing the median lifespan by five years at the cost of losing the essentially human nature of inquiry, then I don’t want that trade.
At the end of the day, intellectual inquiry, along with art and friendship, is one of the very few things a human can experience that possesses intrinsic value.
Or, to quote Ellie Pavlick it in a recent interview, “People always promise that AI will cure cancer. It’s kind of a joke, but I’ve been saying that if AI leads us to this hellscape with no art or music or humanity, I would prefer to just have the cancer.”
I’m not necessarily unsympathetic (though I might have a much more expansive view as to what is valuable in human life). But I think if you polled the population at large and asked, “would it be worth giving everyone an additional five years of healthy life if the price was that research academics lose their jobs”, I’m not convinced your view would be in the majority.
Also, I think the current projection is to get five years median increase by 2050 even without AI. I’d be interested where you’d put the transition point – is fifty years worth the end of human-led academic research? How about a hundred?
I don’t have a good answer about the transition point.
For what it’s worth, I should have said ‘the aesthetic’ rather than art, and I intend both aesthetic experience and friendship to be construed very broadly.
But I don’t take the fact that, on my account of value, people at large may well have incorrect views about what is valuable, and about how to weigh different values, to be a significant downside. On the contrary, it seems entirely unsurprising.
What gets me is so much of what AI optimists promise AI might bring us is possible *now* or not possible with AI at all.
Can AI solve poverty? We can do that now, without AI.
Can AI solve hunger? We can do that now, without AI.
Can AI solve housing insecurity? We can do that now, without AI.
What is AI uniquely providing us that we can’t already do for ourselves? I’m not saying the answer is ‘nothing’, I’m asking that we cleanly delineate between ‘transformative AI’ and future faking AI.
There’s a relevant disanalogy between math and philosophy that makes it far easier to create a superintelligent AI math researcher than a superintelligent AI philosophy researcher.
Solutions to math problems are verifiable, meaning that the results can be checked in a computer programing language such as Lean. This allows labs to train capacities for math research into AI models using Reinforcement Learning with Verifiable Rewards (RLVR). RLVR is the primary cause of models improving beyond the capacities of human math researchers. Since solutions to philosophy problems are not verifiable in this way, labs cannot directly train capacities for philosophy research through RLVR.
Labs can still train philosophical ability using reinforcement learning with human or AI feedback, but this training regimen rewards models for persuading their judges. Plausibly, at best (i.e. without reward hacking and with new capacities learned), this caps their ability near the level of the judges.
So, there’s reason to doubt that superintelligent AI philosophy researchers are coming in the near future.
However, I wish superintelligent philosopher reserachers were coming because philosophy is important to solving the alignment problem and I worry that too few philosophers are working on alignment in ways that are influencing the frontier AI labs.
Of course your point about verifiability is correct, but it might not be the whole story. These AIs are going to need to explain their reasoning, and to express it in clear prose (where here prose includes mathematical symbolism). Self-explaining AI is a huge goal for medical and other applications. AIs will need to be able to analyze their own thinking, to figure out how to be clear. As they learn to do those things, philosophical expertise might well emerge as a side-effect. I don’t know whether this is right, but I think it’s a plausible path to super-human philosophical competence.
This disanalogy with RLVR is the most important point. They can train to fit the prior dispositions of particular philosophers, they can train in argument validity, but neither is going to get you very close to the full space of good philosophical arguments or interesting philosophical positions (to say nothing of philosophical understanding). RLVR works only on domains with well defined and algorithmically verifiable standards of success.
I share this concern. What do you think contributes to the lack of influence?
C.f. the reaction of over 100 mathematicians to these results. (Note how few are raising skeptical concerns about the results, or the mathematical abilities of the AI model, in contrast to reactions here.)
Those are handpicked for credulity.
I had thought that this was a joke, but in conjunction with your other comment, this seems worth addressing. (It’s a sad commentary on the state of discussion here that I can’t be sure that this clear nonsense won’t be taken seriously by the loud contingency of AI capabilities skeptics.)
There is no evidence to suggest, and it is obviously highly implausible to think, that the moderators here went through hundreds of more statements (which would be necessary if these are an unrepresentative sample) and “handpicked” these 100 to misrepresent the mathematical community’s reaction. Not only is this likely practically infeasible (I’m not sure that there are even that many comments from professional mathematicians), but there’s simply no motive to posit such a conspiracy. This isn’t a press release from OpenAI, it’s a compilation on a discussion board run for and by mathematicians. And the comments are decidedly pro-AI in terms of capabilities, not in terms of the impacts of these results or AI in general on the profession. This rebuts any charge of “credulity”.
I agree that there’s no evidence to think these remarks are handpicked. However, I disagree with your characterization of the comparison between the mathematicians’ reactions with the discussions here. I don’t really see a lot of people reacting to this post (except one or two users) more skeptical of the mathematical ability of AI models than the mathematicians there. Among the mathematicians’ reactions, quite a few express the need and desire to understand these proofs, and many are cautious about their correctness. It is just a basic character of mathematical proofs that no matter how promising they look, they need to be carefully checked before certified as an actual proof. I do expect that a majority of these proofs will turn out to be certified (either through formalization or through human digestion). This expectation is consistent with all the concerns that people have raised the against this development in both venues.
The difference could certainly be overstated, but the highest voted comment thread (not highest voted post overall) here accuses OpenAI of “Gish Galloping” and implies that most of the results might eventually “unravel” with close scrutiny. Though many of the mathematicians complain about the quality of the manuscripts, none of them goes nearly that far. So the highest voted thread here expresses a skeptical sentiment that would be a clear outlier among the 100 opinions of mathematicians.
A Gish Gallop need not consist of a pile of lies. The idea is to overwhelm your dialectical opponent with too many arguments on too many topics for them to be able to respond in real time (or, indeed, with a single individual’s expertise). It is, in effect, an info dump.
This math dump is an info dump. It is an info dump which no one individual can check. It is an info dump for which checking even a single proof is, apparently, incredibly onerous.
And it is an infodump which seems to have been orchestrated entirely in service to external goals, namely, securing the company’s position and driving in investor cash (and maybe government subsidies) to prop up a highly unprofitable business.
To my eye, those look like hallmark characteristics of a Gish Gallop.
Note also that I did not predict that the results would unravel. I said that _even if_ (note the conditional) many did unravel, by bundling them together, the dialectical aims remain satisfied; again, this is a hallmark of Gish Galloping.
I had never heard of the term “gish gallop” before reading your first comment so I had looked it up. All the definitions say that it involves putting forth arguments that don’t work or at least are completely unsupported. So to my reading your comment implies that the proofs don’t work or at least that we don’t know whether they work, and that your conditional was not idle speculation on what might occur on the off chance that they unravel, but rather a prediction of what might plausibly actually happen.
So I guess I’ll ask plainly: is it or was it your view that the proofs are at least dubious? If so, that view is very much out of line with the consensus view of the mathematicians in the link I posted.
I have no such opinion, because not only am I not a mathematician or a logician, but my math chops don’t go very far, and my logic chops extend only a little further.
Like I said, I am concerned by the reports of mathematicians saying that the proofs are incredibly difficult to understand not because they’re so innately complex, but because of the gibberish they have to wade through to figure out what is even going on. That’s not exactly a good way to conduct a conversation, and mathematicians confirm it’s not how you present your proofs to the world, either. It’s an infodump, and missing the basic contextual information you’d need to make sense of it all.
The accuracy of these proofs is beside the point, really; this was not done to advance mathematics research, it was done as a PR exercise. And it was done in such a way as to make it very onerous to verify, and such that even if or when errors are identified, the company can retreat to its bailey and point to everything that has yet to be debunked as evidence of what a great investment they are.
I think you are underestimating how geeky (in a good way!) leadership is at Anthropic and OpenAI. I think if you said to them, “we’ve proved the Riemann hypothesis using AI” their response would be less “how can leverage this for fiscal gain”, more “omg we proved the f*cking Riemann hypothesis!!”.
I don’t doubt the purity of the researchers at Anthropic and OpenAI. I just don’t think we should see that as mutually exclusive with the financial interests of those invested in the company’s financial future, and the leadership at either company who are mutually invested in that financial future. The either/or framing seems to be downplaying the very real, documented evidence that both companies have had members who are known to overstate and under-regulate their own results.
It seems *exactly* parallel to the AI generated bug reports – generate a huge pile with some coming out ok-ish after a long (human) search. This too is sometimes done for PR.
I assume Jessie’s point is that this is a forum specifically for mathematicians interested in AI.
(Of course I agree with the larger point. OpenAI’s model provided a proof of part of the *Riemann hypothesis*. Anyone who doesn’t take its math competence seriously at this point just doesn’t understand higher mathematics.)
Well, it provided a proof of something important and hitherto thought to be out of reach that’s related to the Riemann hypothesis in an obvious way.
I’m not sure there’s a non-gerrymandered conjunction of antecedently unproved results P₁ ∧ ⋯ ∧ Pₙ equivalent to RH with P₁ being the claim about the zero-free strip, which is what ‘part’ would suggest to me. (There may be; I’m not a number theorist.)
But of course I agree with the larger point.
The most straightforward conjunction would be to decompose the strip 1/2<Re z<1 into finitely many disjoint, thinner strips, whose union is the entire strip. Then Pn is the claim that the nth strip is zero-free.
Alternatively, you might think of a series of successively stronger results, confining the zeroes to an increasingly thin strip around Re z=1/2.
(Whether either is a plausible route to the Riemann hypothesis, I don’t know; I’m not a number theorist either.)
Yeah, I was thinking that number theorists would count those as gerrymandered, but I’m just speculating.
I don’t think so. If you look at pre-AI progress towards proving RH, quite a lot of it consists of establishing zero-free regions in the strip. But I’m way out of my depth.
That’s true, but the comments there were not submitted by visitors to the site. They were collected by the moderators from provenances which they did not specify (the post does issue a call for additional comments). If they solicited the comments or otherwise found them elsewhere, there’s much less of a selection effect.
If anything, they’re handpicked for negativity!
We know that mathematics requires talent and cultivation of it. With equal amount of effort, some are able to develop new tools and methods, while others can’t, and still others can’t apply existing tools correctly. When the math community including pre-college and undergrad education fully embraces the use of AI and thus the lowering of the gate, and with the decline of the prestige of AI, the math community will likely attract math-dumb people -to be blunt- and not those who have the potential to be grothendieck. This is a sad day for the math community.
This is a perfectly serious thing to be concerned about, but it’s worth remembering that the same could have been said about Mathematica thirty years ago.
Cool 😎