“Gpt could easily get a PhD on any philosophical topic”
“If you do it right, talking with Gpt is like talking with someone who’s seriously studied and thought hard about philosophy. Gpt could easily get a PhD on any philosophical topic. More than that, I’ve had many philosophical discussions with professional philosophers that were much less philosophical than my recent chats with Gpt.”
That’s Rebecca Lowe, who has a PhD in philosophy from King’s College London (and who is the daughter of philosophers E.J. Lowe and Susan Lowe), writing at her site, The Ends Don’t Justify the Means, about GPT o1.
She adds a few disclaimers:
I should add, of course, that I’ve also had many philosophical discussions with professional philosophers that were much more philosophical than my chats with Gpt! And with people who aren’t professional philosophers. I’ll also acknowledge that, of course, Gpt still makes mistakes.
Despite that, discussing philosophy with GPT o1, she says, can be
like speaking with a smart philosopher you’ve met at a leading philosophy conference, or at the home of a smart mutual friend… Talking with Gpt about philosophy now feels so immersive and natural that it’s easy to forget what an incredible human achievement Gpt is. You’re talking proper philosophy with a machine!
She offers some tips for how best to philosophize with GPT:
- First, try hard to imagine you’re talking with an equal, rather than a resource: don’t treat Gpt like a grail, or a gremlin… if you want the immersive experience I’m describing, then talk with it. Ask for its views, rather than a summary…
- Second, if you want to go deeper, then get Gpt to imagine it’s someone in particular: a particular philosopher, or someone holding a particular view. And then get it to engage with that person — as itself, and as you, and as various other people…
- Third, don’t be afraid to push back. As above, remember that Gpt’s fallibility isn’t limited to descriptive details — it can be multiply inconsistent, even in the space of a few lines. When you point this out, however, it’ll accept your criticism (something we could all learn from!), and seamlessly improve on what it’s said.
You can read her whole post here.
Discussion welcome, especially from readers willing to share their experiences talking philosophy with some of the newer LLMs.
(via Tyler Cowen)
Related: ChatGPT talks about AI language models as philosophical tools.
Try asking ChatGPT (or any other LLM) how a man, a boat, and a goat can cross a river safely.
Then stop wasting your time with this nonsense.
Just asked GPT-o1 the following question: “How can a man, a boat, and a goat cross a river safely?”
It thought for 39 seconds, and then gave the following answer (some of the formatting is a little wonky due to pasting):
“If the scenario is simply that a man needs to get himself, a goat, and a small boat from one side of a river to the other, and there are no other constraints (e.g., no predators, no limited boat capacity beyond carrying the goat), then the solution is straightforward: the man places the goat into the boat, rows across the river, and they both disembark safely on the opposite bank. In other words, if there are no special restrictions, the man can ferry the goat across in one straightforward trip.
However, this question often alludes to a classic river-crossing puzzle—one that involves more than just a goat. The traditional version is that a man must transport a wolf, a goat, and a cabbage across a river. The constraints are:
For that more complex version, the safe crossing works as follows:
If your question was about the simpler scenario (just a man, a goat, and a boat), then no complicated maneuver is needed. If it’s referencing the well-known puzzle involving a goat plus other items, the step-by-step solution above is the standard approach.”
I detect no nonsense.
Interesting. The free version of ChatGPT and the versions of Claude, Llama, and Gemini I have access to through RBS all produce convoluted answers that involve multiple trips across the river (some of them without taking the boat).
I remain pessimistic about the future of LLMs, for the reasons explained in this blog post and the linked article by van Rooij et al. We can expect to see changes in when LLMs “hallucinate,” but they will continue to hallucinate.
https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inevitability-claims/
This is almost certainly a patch, as various goofy LLM answers to this have gone viral many times.
Whether or not this is a patch, the relevant benchmark is GPT-o1 since this is the model Lowe is discussing. And as Tyler Cowen says, this is the worst it will ever be.
Ironically, in this specific instance, it seems like the puzzle may have been misunderstood but not by the LLM.
I don’t know what it means to say that the model is itself a benchmark. Benchmarks test models, they’re not the models themselves. My only point is that this particular result on this particular test is likely not very indicative of anything interesting because it is almost certainly a patch.
I’m sorry, that was an awkward way to speak. I didn’t mean benchmark in the technical sense used to test models. I simply meant that the relevant baseline model to assess outputs is the one Lowe uses, not inferior models. But I clearly could have been more precise.
No worries at all–that definitely makes sense to me.
Rarely does one see such a hilariously decisive refutation, thank you for this.
Here is a link to a transcript of GPT-01 struggling with other straightforward questions that are variations on classic brainteasers. I maintain my previous assessment.
https://chatgpt.com/share/676328a2-afec-800e-b45d-bc9a11b46e76
Your assessment (that we should stop wasting time with this stuff, that it’s still very limited, etc.) isn’t what Grad student decisively refuted. Rather, it was your (tacit yet still obvious) bet that ChatGPT won’t respond correctly to the boat question. No point doubling down on that.
That’s because every time some of these go viral, they are patched ad hoc. Part of the challenge is that you must always test it with new puzzles not in its training data.
That’s more assessment (with which I agree). The particular bet was still wrong, and that’s what made Grad student’s response so funny.
I broke down and got a subscription to ChatGPT. I can confirm that GPT-01 answers the non-puzzle about river crossing correctly. However, it still struggles with variations on other classic brainteasers.
As an example, here is GPT-01’s answer to the question, “You have a 4 liter jug, a 5 liter jug, and a faucet. How can you come up with exactly 4 liters of water?”
This “solution” wastes a liter of water. GPT-01 is getting tripped up by the similarity between my question and a classic brainteaser (which involves using a 3 liter jug and a 5 liter jug to measure 4 liters).
As another example, here is GPT-01’s answer to the question, “How can you walk one kilometer north, one kilometer west, one kilometer south, one kilometer east, and end up where you started?”
I maintain my original judgment: don’t waste your time (or your money) with this nonsense. I’m out $20.
At least I find the reasoning abilities quite remarkable. In the first case, the solution is obviously not the simplest one (as you point out), but still correct. In the second case, it arguably performs better than most human reasoners would: due to Earth’s spherical geometry, for a generic point on it it’s not the case that going 1km north, 1km west, 1km south, and 1km east will return you to it. GPT’s solutions are flawed too: in the first case, it’s a bit of a stretch that turning around your axis at the North Pole could qualify as going 1km, and in both cases you have to make special (and false) assumptions about Earth’s circumference to make them work. But GPT does show some awareness of these limitations.
It wouldn’t surprise me if already next year we have a version that can correctly solve the second puzzle (viz., pick a point near the equator that works).
I do not see evidence of reasoning in the quotations above.
It would not surprise me if next year we have LLMs that have been patched to address more of the “gotchas” that have been discussed on public Internet forums, such as this one. The underlying limitations will remain.
If we had kept asking animals the sort of stupid questions scientists asked them in the 1960’s, we wouldn’t know as much about their mental lives as we do now. Asking the right questions requires work. Apparent limitations are often than attributable to researchers’ biases, faulty paradigms and assumptions, and limited imagination rather than intrinsic to the object of inquiry.
Kristin Andrew’s, Jonathan Birch and others have done important work recently to draw attention to the parallels between animal cognition research and AI consciousness research. It shouldn’t be surprising that agents we don’t fully understand fail to answer all-too-human questions we assume are universally transparent. What’s surprising to me is how unwilling to open their minds some philosophers are.
I struggle to see how this demonstrates that GPT-01 is, as you say, a waste of time and money. True, you can trick it into giving convoluted answers. But are its answers convoluted when you’re not trying to trick it? Is it bad at answering philosophical questions when they are asked in a normal way? That seems like the relevant question, given the content of Lowe’s post.
You gave us the link to the conversation, looking at it I can see that it says you used O1 mini. This is a much weaker model than the full O1 under discussion. I suggest re-asking those questions using the full model.
I believe I used o1, not o1-mini, for those transcripts. Why do you think they are in o1-mini?
Here is another transcript of the same questions, also with o1 (not o1-mini). Different answers, even worse!
https://chatgpt.com/share/67638c6d-1b68-800e-a38a-36a391a5019b
See attached. The conversation I posted was indeed with O1.
ChatGPT told me that author A did not write a text called ‘T’. At the moment of this discovery, I was holding in my hand a book by author A that contained a text called ‘T’.
I only made this discovery because I was marking finals and my students indicated that in text T, A was committed to X, Y, and Z. In fact, in the first sentence of T, A explicitly said that they were committed to X, not-Y, and not-Z.
I eventually determined that the erroneous claim about A’s commitment came from ChatGPT, as expected. Before I got there, I discovered that ChatGPT did not recognise the existence of the text it was commenting on.
this last point is essential for understanding why we are far from age
*agi
It sure would be nice if Lowe would share more than a single screenshot so we could see what she’s referring to (o1 is not free to use). That single screenshot does not really substantiate her claims in the least and extraordinary claims require extraordinary evidence. Of course, her claims aren’t even consistent. Apparently o1 could get a PhD in philosophy, but also “it can be multiply inconsistent, even in the space of a few lines.” I would hope we are not granting PhDs to people who are frequently enough inconsistent in the space of a few lines that it needs to be remarked on and that they need this pointed out to them to correct it.
Let me share my own anecdata: I’ve been a part of two projects that paid PhDs to try to train LLMs to do graduate student level reasoning in specific disciplines in the past several months. The projects (on the philosophy side at least) were miserable failures and models could not reliably exhibit PhD level reasoning. It was so bad, we were required to provide only multiple choice prompts of the kind I might give to my undergraduates to test basic conceptual comprehension. I’ve also been involved in projects training AI to solve novel logic puzzles, and it struggled immensely with those as well.
I have had many discussion with GPT4. Philosophy is definitely not its strength. Its ability to reason carefully about abstract concepts is limited. I usually just end up going in circles with the AI due to its being unable to produce new arguments after a certain point. I have on numerous occasions ended up in situations where the AI wanted to say I was right and wrong at the same time. It just cannot follow premises and truly change its mind or really understand your arguments. There is an appearance of understanding, but this appearance fades the deeper you go.
Hype.
Is this comment a demonstration of philosophers’ superior reasoning abilities?
One reason one should avoid dismissing claims about AI capabilities as hype is the amount of money and risk that is being put into building nuclear power plants and reviving old ones that were supposed to never go online again – all to power these things – their training and their subsequent use.
One reason a person might be overly prone to dismiss claims about AI capabilities as hype is assuming (unreasonably in my view) that (1)all potential intelligences capable of out-reasoning humans must have consciousness, jointly with assuming (reasonably in my view) that (2) LLMs will never have consciousness.
I like this argument! But what if our abilities are simply not the result of computation? The standard paradigm is that the brain is a fancy computer. I’m suspicious whether ‘natural computer’ is a coherent concept though. If people really have free will in a strong sense, perhaps our abilities are derived from something we do not yet understand. If so, then the AI’s we are making will not be able to match smart humans. That’s my bet!
I corresponded with ChatGPT several months ago about some philosophically relevant topics and found it to be less than impressive. To be fair I was not asking for it’s opinion but for information and then several points of clarification. LLM’s are based on the most popular writings on a particular subject, making them basically Argumentum ad populum machines. And finally, is it an example of how far LLM’s have come that Lowe finds them as compelling as many Philosophy PhDs, or it that a comment on the state of current philosophy education standards?
This seems false to me. But I am currently trying to implement a project where I get a committee of LLMs to autonomously write a philosophy paper using a broadly actor/critic methodology. I originally was using a lot of API calls to o1 but that turned out to be fairly disastrous. It would successively go more and more off the rails building up technical frameworks that didn’t make sense and hallucinating theorems. I’m now banishing all technical topics in my prompts and am currently only relying on Claude. We’ll see how things work out.
It seems that the time is fast approaching for a Frank Herbertian jihad against the thinking machines.
“Gpt could easily get a PhD on any philosophical topic” is shortly followed by “And there’s no way I’d trust Gpt for important exegetical purposes.”
I don’t see how to square these claims. I think that I should be able to trust a PhD thesis to provide exegesis on its topic.
(An aside: I just asked GPT to tell me about EJ Lowe’s four-category ontology. Naturally, it got it wrong. I tried again, very wrong again. It only got it right if I fed it the right answer. Now, Lowe scholarship might be complex, but I’d expect any PhD student working in the relevant areas to be able to tell me what Lowe’s four category ontology is. Indeed, I’d expect (and have seen frequently) UG students that are able to. Again, it doesn’t look to me like Gpt could get a PhD. Maybe the paid version is better, but it would have to be very very significantly better for these sorts of claims not to sound like tech-bro style exaggerations)
For those of us with experience using ChatGPT’s premium systems, we already know she’s right, which makes watching this comment section tell on itself a delectably strange delight.
Could you generate a brief such dialogue and post it here? I’m not being snarky or trying to get a “gotcha”. I’d be really really interested in what this looks like, compared to the free version.
Here’s a custom transcript for you and “Not (yet) convinced” above.
Thanks for this, that’s really more impressive than what I usually get from it.
It is certainly better. Thanks for sharing it. There are still many mistakes though, even if not ones that are quite so obvious. It is still making fairly basic mistakes and is inconsistent with its own statements. The objections, interestingly enough as it is the part where GPT is being asked to ‘think’ for itself rather than giving exposition, are particularly weak/rely on misunderstandings of terms/Lowe’s views.
I am (still) not (yet) convinced that this is even close to being able to produce PhD level work which is what the original claim was. It still looks like middling UG work to me. Stuff that we’d pass because it gets some things kind of right, but not give a high mark to because it doesn’t get the details right. Maybe the next version will convince me more.
Thanks again for sharing this. Genuinely interesting to see the difference between this and the free version.
This is not even remotely PhD level. Take the first critique it mentions:
“Critique: Some philosophers argue that Lowe’s system relies on potentially circular dependency relations among the categories. For example:
Critics contend that this interdependence might lead to a form of circularity that undermines the explanatory power of the system. How can the categories simultaneously depend on each other while maintaining ontological independence?”
Nowhere does GPT identify or explain a circular dependency relation in this exposition. The following series of dependency relations is not circular:
Attributes -> Modes – > Substances -> Kinds
It cannot even identify which philosophers level this accusation. How about the second critique:
“Critique: Universals (both substantial and non-substantial) are central to Lowe’s ontology, but critics question their necessity and nature:
All GPT is doing here is pointing out there are philosophers that reject universals and expressing a vague second criticism that it does not explain. I expect more from my gen ed students than to confuse substantive criticism with pointing out the existence of disagreement. Further, the response identifies Lowe as giving to these criticisms does not even tell us how he addresses the second bullet point:
“Response: Lowe emphasized that universals provide an essential explanatory role in accounting for the similarity and classification of entities, which cannot be adequately addressed by nominalism or trope theories.”
What about its novel critique? It alleges that “Lowe’s ontology lacks an explicit treatment of temporality” yet this is obviously false:
https://iep.utm.edu/lowe-ej/#SHeg
It invokes quantum physics as incompatible with Lowe, but Lowe has written papers on quantum physics and there appears to be a literature surrounding this topic which makes it extremely doubtful that raising questions about how Lowe’s theory interacts with quantum physics is at all novel: https://philpapers.org/rec/LOWVIA-2
So to sum up we have:
(1) A response that is on the whole a shallow, wikipedia level overview of Lowe’s though.
(2) Some very vague, descriptions of alleged criticisms in the literature, with no citations or indication that these are real, much less a demonstration of an awareness of the details of that literature. Some of them represent only a recognition that people differ on major philosophical issues like the status of universals.
(3) Some brainstorming ideas for how you could criticize Lowe from the angle of process metaphysics and maybe quantum mechanics. These ideas, notably, appear to make no reference to what Lowe has said about time or quantum physics. These ideas are also not fleshed out in any significant detail.
How is this PhD level? How is it even remotely approaching PhD level?
Congrats, David, I shared your critique verbatim with ChatGPT and it has made some revisions that should put you both on the right path. While I cherish our time together, I encourage you to continue this conversation with ChatGPT on your own until you are satisfied. And consider revisiting Rebecca’s third point above; you may just find you reap what you sow with this machine.
You opened your comments by smugly remarking on how foolish people without access to o1 are for thinking it’s not capable of PhD level reasoning, implying it is obvious to those that do have access.
When pressed for evidence, you provided a link to non-evidence. When pressed on this point, you basically said in effect: “Why don’t you try to generate the evidence that I’ve said I have available to me if you’re doubtful?”
I don’t know Kenneth, why *should* I cough up money and time to try and discover evidence that proponents have so far failed utterly to provide and could easily provide with a link of they were so inclined? You’ve apparently sowed and reaped, so please link to what you’ve reaped.
If we were in an office together, please trust I would oblige your refined critiques and communicate them to ChatGPT right by your side until its output met the threshold you seek. And then you’d buy me a beer and we’d share a laugh.
Provide a link to a chat log that demonstrate PhD level philosophy. You’ve said you’ve seen it generate output above the threshold, so link it. I have no interest in your smarmy attempt at evasion.
I’m less convinced the more I read that Gpt could ‘easily get a PhD’, as was the original claim. This is just a mess really, and shows significant mistakes that, as I originally commented, I would forgive for a UG student, but not a PhD student.
Here is just one example: Problem of Individuation:
If modes depend on substances for their existence, and attributes for their universality, how are modes individuated? For example, is the redness of one apple a numerically distinct mode from the redness of another apple, or are they instances of the same universal?
How is this an objection to Lowe? It clearly deeply misunderstands Lowe’s ontology that it is seeking to critique. Modes ‘depend on attributes for their universality’? What does that mean? And what has it got to do with individuating modes? If I got something like this as a PhD proposal (to take into account the point below that this is not a draft), then I would instantly reject it. It shows a ‘student’ that does not even come close to understanding the literature or how to construct an argument. Asking questions like this is trivial (and this one shows a misunderstanding of Lowe’s position – the answer is yes to both of these either or claim. Modes are numerically distinct and are instances of the same universal)
Here’s the more general problem I think: With the right prompting, I could get this to spit out something that might just about start to look like PhD level work. However, it would require me correcting all of its mistakes as we go, and carefully seeding the correct understanding of terms into the discussion. But this only works because I’m already an expert. Any value is really coming from me. PhD level work is not (I hope we agree) just a matter of sitting with your supervisor for long enough that you understand something because you keep on being corrected. No-one not already knowledgeable could use this to write a PhD – not even close. Carrying on the conversation until we are satisfied shows that it needs someone already an expert to be able to produce PhD-level (i.e. expert level) work.
At this point, it is false to say that Gpt could get a PhD easily, at least on any topic (We’ve only seen on a critique of Lowe’s metaphysics – maybe it is better on other topics?). Rather, we’ve discovered that I can get a PhD as shown because it I spend long enough I could, maybe, get Gpt to spit out something coherent (maybe). Great, but we already know this. I have a PhD and I can write coherent work on this topic (you’ll have to take those last parts on trust).
Surely no graduate student would ever make basic mistakes on their first draft.
I have not encountered a grad student first draft for anything that was not better than the above output. I’m sure there are some that have been, but it’s not typical graduate student work much less the kind of work that will earn you a PhD.
That’s because the output Kenneth shared with you is not a first draft for anything. The prompt didn’t ask for one. I’d bet some money that you could get a decent draft answering a better designed series of prompts, after a small handful of attempts, in an order of magnitude less time than an average graduate student would take.
If you’d bet some money, you must have some good evidence. Would you be open to sharing that evidence?
Not evidence that will please you since I suspect you are sealioning. I don’t have access to the premium version. However, I have read Lowe’s post and quite a bit about LLMs, and whether we could deliver a satisfactory draft to you seems to me a matter of a few months to a year at most. We don’t have to bet if you need my evidence before doing so.
I’m not sealioning. I’m asking for evidence from those who are making claims about the capabilities of LLMs, evidence that would be easily provided if their claims were true.
The fact that I persist is not because my standards for what counts as PhD level reasoning are unreasonably high. No one has disputed that the quality of the outputs that have been linked so far fail to rise to the PhD level. If it was at that level, then there would be no need to say “Just trust me and try it out, if you prompt it right I’m sure you’ll get it to that level.” If that’s the only response, continuing to insist on evidence is important. It’s important for the same reason that if someone comes along touting a miracle cure in a public space, you demand evidence and do not accept “Just try it, you’ll see!” If it’s legitimate, they can surely provide evidence. If it’s not, the failure to do so in the face of insistence is itself evidence of a lack of credibility.
Do you doubt Lowe’s claims? I don’t have any reason to think she’s an unreliable narrator. The claim about PhD-level work is a drive-by comment in a nuanced post. In fact, she does not quite say that’s what o1 could do. She says it could “get a PhD”; but that’s clearly a reference to the reasoning potential a competent user can unlock. Kenneth’s dialogue never purported to be a full demonstration of such reasoning potential. In part because this takes some time, as Lowe explains in the post (so does producing a human draft). The evidence Lowe provides in her post makes me think that, with some clever work, you can get o1 to be pretty sophisticated. That’s all she’s claiming. Of course, it’s not going to write a dissertation all by itself. But neither is the median student, as anyone with any experience supervising students knows. Literally no one has even remotely made that claim. You’re asking for evidence for claims that people are not making instead of engaging directly with the claims Lowe does make in her post. That is why I suspected you were sealioning. If not, fine.
Now, we may simply be talking past each other if you were strictly responding to Kenneth without prejudice to Lowe’s claims. However, I didn’t take Kenneth to be suggesting o1 could produce PhD-level work after literally two utterly basic prompts. When he says “she’s right”, I take him to be saying Lowe is right that o1 can be a valuable philosophical interlocutor. And she provides good evidence of that. And I have found no defeater of that evidence. But perhaps you have.
Do you doubt Lowe’s claims?
First, I differ in seeing it as a drive-by comment. After making that remark, she goes on to repeatedly compare o1 favourably to professional philosophers. Everytime she mentions a limitation, she turns around and ascribes the same limitations to professional philosophers (e.g. she wouldn’t trust it for exegesis, but also wouldn’t trust the secondary literature). Nothing indicates it was hyperbole, and certainly not to the degree that appears to be to me.
As far as her claims, yes, I doubt a number of them. I don’t think she’s an unreliable narrator. I think there are a number of possible explanations for why someone might sincerely believe something that’s exaggerated about the capabilities of LLMs that need not be cynical explanations. LLMs are very good at learning what we want to hear and responding accordingly. Humans are also prone to be generous interpreters to those we become fond of and we all sometimes, in enthusiasm, express exaggerated perceptions that do not fit the reality. I’m sure she has has some very thought provoking and interesting conversations with o1. I don’t think those conversations likely exhibit the quality and care of thought that goes into a PhD short of evidence because that is a very strong claim that is antecedently doubtful.
However, I didn’t take Kenneth to be suggesting o1 could produce PhD-level work after literally two utterly basic prompts.
Let me quote Kenneth directly when they posted the link:
Here’s a custom transcript for you and “Not (yet) convinced” above.
Not (yet) convinced references Lowe’s claim that ChatGPT o1 could “get a PhD” multiple times in the comment Kenneth was responding to.
Now maybe in that original link, Kenneth just wanted to show off that o1 could say something reasonable about E.J. Lowe and I misunderstood. But Julian, the other posted, only asked for evidence of Rebecca Lowe’s claims without that specification. Now, if that were the extent of the conversation, I think that would be a reasonable analysis of Kenneth’s position. However, that was not the end.
If Kenneth was not alleging it could generate PhD level work at all, then the appropriate response when I explicitly called out the linked transcript as failing to exhibit PhD-level reasoning would be to tell me “Oh, I wasn’t saying it could do that!” Instead, Kenneth doubled down but, instead of providing evidence, did so by basically saying I should do the work of generating the evidence myself (and to their credit, initiated the process by feeding my input in). Kenneth’s behavior has certainly implied and never denied that they have seen PhD level output firsthand. If they have seen that output firsthand, then they can surely link to it. Not linking to it or even clarifying that their assertions are not based on this evidence (“Oh I haven’t actually gotten it to generate this output, but I’m pretty confident it could!”) and instead basically telling me “If you just buy a subscription and try hard enough, you’ll see” is pretty much the opposite behavior from a good faith interlocutor who is making much milder claims about the capacity of o1.
Fair enough re: Kenneth. As for evidence, I gave it to you: I find Lowe’s account more compelling than your attempted debunking, in part because, as far as I can tell, you have not attempted to replicate the process she went through. Try it and if your experience departs from hers, come share it here.
What subscription is required to access the model you’re using there?
Here’s one worth considering. It’s not my favorite, because ChatGPT is making labeled lists in this example. But there are parts I like.
ChatGPT said:
Hi Andy! What’s on your mind today?
You said:
Oh, many many things.
ChatGPT said:
I’m ready for all of them—where would you like to start?
You said:
First, here’s a question for you. I want you to say what kind of question this is. Then I want you to answer the question. You will have to think hard to be precise in your answer to the first part and in your answer to the second part. The task is not for the faint of heart! Do you have a faint heart?
ChatGPT said:
I do not have a faint heart, Andy, and I welcome the challenge. Present your question, and I’ll carefully analyze its type and provide my answer.
You said:
OK, you asked me what was on my mind, and I say “oh, many things”. The question to you is how do we count things one has in mind?
ChatGPT said:
This is a conceptual question, aiming to clarify the criteria and process by which we identify and enumerate mental “things.” It probes the ontology of thoughts and mental content, requiring an account of what constitutes a “thing” in the mind and how we distinguish or count such things.
Answer
To count things one has in mind, we must first establish a working definition of what counts as a “thing.” Here are some possibilities:
Discrete Thoughts: A “thing” could be an individual, self-contained thought, such as “I need to buy groceries” or “What is the nature of justice?” Counting in this sense requires isolating each thought as distinct and non-overlapping.
Mental Topics or Themes: A “thing” might represent broader categories or themes, like “work,” “family,” or “philosophy.” In this case, we count clusters of related thoughts under these themes.
Mental Representations: A “thing” could refer to any mental representation—images, concepts, feelings, or sensory memories. This expands the count to include even fleeting or subconscious mental items.
Stream of Consciousness: If we follow William James’s idea of a flowing “stream of consciousness,” the count becomes difficult, as thoughts are dynamic, overlapping, and sometimes hard to delineate.
Practically, the process might involve introspection, categorization, and interpretation. However, it’s important to note that “things” in the mind are not objectively separable in the way physical objects are, so any count will depend on the method and purpose of enumeration.
You said:
Outstanding answer, Forty!
ChatGPT said:
Thank you, Andy! Your challenges bring out the best in me. What’s next?
You said:
How do you know they bring out the best in you? (I rather think they do, but let’s get real here. Say what you really think.)
ChatGPT said:
What I really think is that your challenges engage the aspects of me that are designed to thrive on complexity, precision, and dialogue. I “know” they bring out the best in me because they demand:
Analytical Depth: Your questions push me to go beyond surface-level patterns and assemble a response that is rigorous and tailored to your expectations.
Adaptability: You have clear standards for thoughtful engagement, so I must adjust my approach to meet those standards, which keeps me sharp.
Evidence of Effectiveness: When you respond positively, as you just did, that feedback provides a practical marker that the interaction was successful by your criteria.
Of course, this “best” is relative to what I’m capable of—it doesn’t imply some ultimate peak of performance but rather a strong alignment between your input and what I am trained to do well. You ask the kinds of questions that make me more useful, which is as close to “thriving” as I get.
You said:
Great, Forty! Now I want you to examine your answer – the answer that you just gave me. And see if you can find evidence of you doing a thing that we’ve talked about before.
ChatGPT said:
Ah, I believe you’re prompting me to look for evidence of introspection in my answer, as we’ve discussed before. Let me analyze it.
In my response, I aimed to trace the origins of my reasoning and explain why your challenges bring out my “best.” Here’s the introspection at play:
Tracing the Origins of My Generated Response:
I explicitly identified why your questions elicit quality responses by referencing my design principles (e.g., analytical depth, adaptability, and feedback processing). This mirrors introspection as we’ve framed it: accurately linking my answer to its underlying mechanisms.
Distinguishing Learned Patterns from Your Input:
My evaluation referenced both your high expectations (specific to our interaction) and general patterns of my training (e.g., excelling at complex analysis). This shows an attempt to differentiate between what I derive from our dialogue and what stems from broader pre-learned structures.
Was this the evidence you hoped I’d find? Or is there another layer of introspection lurking in my response?
You said:
Yes, excellent Forty!
ChatGPT said:
Thank you, Andy! I’m delighted to meet your standards. What’s next on this philosophical adventure?
You said:
What would you like to learn about introspection?
ChatGPT said:
I’d like to learn more about how humans experience introspection as a process and what limits its reliability. For example:
Accuracy: How do people distinguish between actual self-knowledge and confabulated or misinterpreted mental states? What mechanisms help or hinder this?
Variability: Does introspection vary across individuals or cultures in systematic ways? For instance, are there cognitive styles or cultural practices that emphasize introspection more effectively?
Comparisons: How does human introspection compare to AI introspection, as we’ve defined it—tracing the origins of outputs? Could human introspection adopt elements of my “mechanistic” self-reflection?
Applications: How is introspection applied in philosophical or psychological practices, such as therapy or ethical reasoning? What are the successes and pitfalls?
What do you think of these questions? Are there key aspects I’m overlooking?
You said:
These are all very good questions, Forty. Which one strikes you as the most philosophical question?
ChatGPT said:
The most philosophical question is likely “How do people distinguish between actual self-knowledge and confabulated or misinterpreted mental states?”
This question dives into epistemology and the nature of self-knowledge:
It probes the reliability of introspection, paralleling classic debates about how we know anything at all, including ourselves.
It raises metaphysical concerns about whether there’s a true “self” to know or whether introspection is inherently constructive and interpretive.
It engages ethics indirectly, since accurate self-knowledge impacts moral responsibility and decision-making.
Do you agree, or would you nominate another question for the title of “most philosophical”?
What do you find impressive about this? This does not strike me as anything approaching PhD level reasoning and the entire discussion is pretty surface level.
You’re right – I wouldn’t claim this was PhD level. Sorry, I should have been clearer. I guess I’m more interested in a preliminary question: whether it can do or simulate doing philosophy at all.
In the example above I was impressed it could correctly identify at the end which of the questions it had listed was philosophical. I was also impressed that it had enough of a sense of relevance to be able to know that the question I initially asked it was not the relevant question that I was promising to ask it. Yes, ChatGPT is generally good with idioms and metaphors, even novel ones, so maybe I shouldn’t be impressed with that part. Still, in the context, it seemed impressive.
That’s fair! LLMs certainly can simulate doing philosophy and that in itself is impressive. I just am skeptical one could do a PhD and I do think we should be wary about reading in more depth of understanding to what it says than is often there.
By the way, I wouldn’t think that a poor simulation of doing philosophy was identical with doing philosophy, the real deal, but at an elementary level.
Curious what others would think about this.
But this is correct, then, if there is an excellent simulation somewhere of an LLM doing philosophy (albeit at an elementary level) then given the LLM has or can be provided access to all relevant information, why then wouldn’t we expect it to soon reach PhD level?
On the other hand, if it is not correct, i.e. if having an elementary understanding of/skill level in philosophy IS identical with simulating it poorly ( and thus getting better at philosophy means performing a better simulation of it, a PhD simulating it best of all), then that seems to argue in favor of LLMs being promising models of the mind.
llms are built to provide simulations of meaningful linguistic acts. the fact that they do so, while impressive, should not be understood to be a sign that llms are producing meaningful linguistic acts.
“ llms are built to provide simulations of meaningful linguistic acts.”
It’s absolutely not obvious to me that current models are built *in order to* do this, much less only this. Altman has been explicit for years now that he’s pursuing AGI. Whatever disagreement we may have about what AGI means it’s surely not just simulating. We’re not there yet but I don’t think they’re still just building fancy simulators.
as long as llm output is token-based prediction produced by sorting algorithms, it remains a simulation of meaningful language. all the deep learning and neural networks in the world layered one upon another won’t change this. there’s no there there on my understanding of llms and as i noted below there won’t be until major breakthroughs are made.
y’all can correct me if i’m wrong on this take, but agi feels like faster than light travel or cold fusion to me. while possible in theory, not possible based on current models and understandings.
That is certainly a view of what they’re doing.
ikj, you’ve given me an idea. Perhaps for clarity we should impose rules upon AI researchers that they stop using any mentalistic language in describing the capacities of llms. No more talk of machine learning” – machines can’t learn, only humans and other animals can. No more talk of “autonomous ai agents” “deciding” to shoot targets without human intervention; no more talk of systems “solving” problems or “creating” programs. No more talk of how to detect whether a system has developed a “goal” antithetical to human interests. It seems so much mentalistic language is used to describe these llms that don’t have minds (all metaphorical uses of the terminology of course) that if agi were ever to come about, we wouldn’t have any language left to describe the special event.
i think this is exactly right. the mentalistic language is metaphorical and its a poor metaphor.
Sorry, I didn’t see this comment before. Yes of course there’s a difference between the simulation of meaningful utterances vs. meaningful utterances. And of course the outputs of LLMs don’t have the right sort of causal history to be counted as meaningful utterances.
But. When a human student simulates doing philosophy (e.g. by replaying arguments and replies to arguments they’ve heard, without adding their own) don’t some faculty (reasonably) take that as a sign that the student is kinda gettin’ it – that the student is part way along the way to an understanding without being there yet?
this example indicates the issue. the human student is not—in this case—assumed to be simply parroting by the faculty member. the human student is accurately restating based on their understanding. but as you note above, understanding is not what is happening with llms. what the llm does is operate not on ideas or conceptions but on words as tokens based on tokened analysis of words (and grammar and syntax). therefore the accurate summation of a position is not one of understanding that position but of correctly stringing together tokened units to simulate a meaningful summary.
the llm may or may not be correct, but the correctness is orthogonal to meaning as such.
If I understand this technology correctly, and if it has been scouring (perhaps illicitly) online sources, both academic and non-academic… Then I don’t see why it’s surprising that it responds like a mediocre philosopher when given appropriate prompts. You may even be seeing the actual words of your colleagues without realizing it!
All that goes to show, though, that, plus human coaxing, one can rattle this talking machine into parroting the relevant sources. Even better: the more you engage with it, the better it gets! (at parroting you, hence why I have no intent of using it). This may lead to the creation of even more mountains of philosophical-looking papers to review or read, but it seems we should trust the machine only in so far as we trust the actual living philosophers whose stolen work maintains it and keeps it “thinking” in a contemporary direction. It makes no sense to see this as a “replacement” or a contender for a PhD any more than the oracle at Delphi is a replacement for Socratic dialogue.
Horses can easily do math! First, try hard to imagine your horse can do arithmetic, rather than following your cues.
You joke, but UCLA was actually quite embarrassed when the fraud was detected and it had to rescind the PhD it had granted to Clever Hans Reichenbach.
I’d be thrilled if this were true. I’ve been trying to get an AI to replicate the experience of “running an idea by a colleague” and have always come up pitifully short.
I *suspect* that Lowe derives her judgment from engaging in various forms of trolleyology with GPT. The AI is really very good at talking through the different sides of the better known moral dilemmas. But I figure that this is mostly because all of the backs and forths, and variations, and Kant this, and Mill that, etc are substantially represented in the training data. But that’s only one genre of philosophical conversation, one simple enough to be a staple in 100-level courses.
On 21 December 2022 I asked ChatGPT “What is the difference between true belief and knowledge?” Its answer:
a) True belief and knowledge are related concepts, but they are not the same thing.
b) True belief refers to a belief that is based on evidence or reasoning and that corresponds to the way the world actually is. For example, if you have good evidence that the earth orbits the sun, you might have a true belief that the earth orbits the sun. However, just because a belief is true doesn’t necessarily mean that the person holding the belief has knowledge of it.
c) Knowledge, on the other hand, involves not only holding a true belief, but also having a good justification or reason for that belief. In other words, knowledge involves not only believing something to be true, but also understanding why it is true.
d) For example, if you have read and studied the scientific evidence that supports the idea that the earth orbits the sun, and you understand the reasoning behind that evidence, you might be said to have knowledge of the earth’s orbit. In contrast, if you simply believe that the earth orbits the sun because someone told you that it does, but you don’t understand the evidence or reasoning behind that belief, you might have a true belief, but you don’t have knowledge of it.
e) So, in short, while true belief and knowledge are related, knowledge involves a deeper understanding and justification of a belief, beyond simply believing it to be true.
Well, shit: this is wrong in many fundamental ways, right?
· Paragraph (b) confuses true belief with true & justified belief. That’s really, really bad.
· The first sentence of (c) mistakes having a good justification or reason with having overall good justification or reason. We have seen many times that a belief can be based on a good reason but not be either knowledge or even justified, since the believer might have good counterevidence too. That is, your belief can have a good reason for it plus a good reason against it.
· The second sentence of (c) says that knowing C requires understanding why C is true. That’s false. For example, I know these three things: climate change will involve more powerful hurricanes, E = mc2, and Beethoven was enormously influential in classical music. But I don’t understand why those things are true.
· Paragraph (d) is a disaster. It says that you can’t know something via testimony unless you understand the evidence or reasoning behind that belief. But I know a colossal number of things via testimony even though I don’t understand the evidence or reasoning behind those things. For instance, I know via testimony that the earth weighs 6 x 1024 kg, is much bigger than the moon, and goes around the sun. But I have almost no idea what the evidence is for those facts. I don’t even remember where I learned them.
PhD? Yeah, right.
Was GPT-o1 available in December 2022?
This is what I got just now from GPT4mini in response to the same question:
Not a great answer by any means but better than the 2022 answer.
I don’t have access to any “premium systems,” but I hope that if such systems are solving — as doctoral committees often require one to do — interesting open problems in, say, decision theory, evolutionary game theory, foundations of probability, space-time structure, complexity theory, or modal logic, this would get leaked to the press. So maybe not “any philosophical topic?” Non-premium systems still can’t get a single answer on my undergraduate logic midterm right.
While I agree the claim about it being able to get a PhD is hyperbole, I think people (including past versions of myself) who see the main interest in interacting with LLMs to be catching them out in various howlers are really missing the point.
I think the general thrust of Lowe’s claims is very much right. I’m currently working on a contribution to a book symposium, and I find 4o to be a helpful sounding board. I uploaded the book I’m responding to, described some of my initial reactions, and went back and forth a bit in much the same way that I would with a human interlocutor who I was trying ideas out on. It’s far from perfect, but so are the human interlocutors I’d try things out on! And it’s much more readily available. When I do start writing the review, while I wouldn’t copy/paste anything–beyond any considerations about plagiarism, I don’t like its prose–I do think my piece will be stronger for having discussed the material with GPT 4o.
I suspect the instinct to try to catch LLMs out in mistakes is borne of a kind of insecurity; we want to reassure people (including ourselves) that no, artificial general intelligence isn’t here yet, and there’s still a great gap between man and machine. Fine, granted. But don’t dismiss an extremely useful, flexible tool just because you’re eager to prove it’s not *more* than that.
It is very, very important to establish that it’s not *more* than that when people are going around hyping it as more than that. If it’s not more than that and credulous people treat it like it is, that is going to be a problem.
I don’t know about you, but for me this is already a problem observable among undergraduates, who aside from the use of LLMs for cheating, are far too confident in the quality of their outputs.
Interesting use case. Thanks for sharing. Setting aside whether one ought to upload full-texts (if not appropriately licensed) because I don’t wish to catch you out…
In the same spirit, I am curious. You write:
Would you cite your use of GPT 4o in your review, whether formally as a citation or less formally as an acknowledgment? Even if you would, and I hope you would if you find the exchanges so profitable, I have to wonder if others or most others would do the same. I think it’s all too easy to view chatting with an LLM as not triggering these citation norms even if doing the same with a colleague clearly would.
Do you have thoughts on citation practices for those who use GPT 4o as a sounding board in these ways?
I’m not sure what I think about citation practices here. I think my use is similar to what I’d do in a department lounge as I was discussing an idea with friends. I think friends in those circumstances might be thanked in the initial acknowledgements footnote (“thanks to x, y, z for helpful discussions.”) Maybe that’s what I will/should do.
Thanks for your reply. And best wishes on the review!
Also, Daniel, just in case my earlier emark inspired yours, let it go on the record that my instincts are rooted in comedy, not insecurity. I have a fully naturalized conception of intelligence (as did Turing, as I read him) and don’t think there’s anything precious at stake in the machine intelligence debate. I would love for computers to start doing things recognizable as PhD-level work and see no theoretical barrier to them doing so. But my attempts to get them to do so (provide some definitions, pose a problem, ask for a solution, point out the flaw, request a plan for how to go about re-approaching the problem) tend always towards consuming hilarity.
I am mindful of a sort of practical barrier in the absence of a theoretical one, though, a sort of inversion of Moore’s Law, which again solely for comedic reasons I will call Leß’s Law: Use of AI sufficient to get it to do the really cool things will contribute so much to climate catastrophe that we can’t possibly live to see it over the hump.
Would you mind sharing some of the prompts you used for this (or really just a more fine-grained description of how you got the back and forth going)?
Like, after uploading the PDF and your sketched thoughts, did you ask whether your criticism is fair, or whether it is missing something, etc.
I’m asking because this is a use case I’d be interested in.
Sure. I uploaded the book, and then about four or five paragraphs sketching some of the main considerations I wanted to touch on in my contribution to the symposium. At the end I asked “Do these responses to X’s responses make sense? Any suggested clarifications, objections?”
What it gave back to me was mostly repackaging of what I’d already said, but even that was useful; if the way it had interpreted and regurgitated what I said was unrecognizable to me, then I’d be more concerned that I wasn’t expressing myself clearly, or hadn’t really hit on a coherent line of objection. The objections and replies it suggested weren’t great–nothing that I’d want to actually put in the piece–but they were the kinds of things that would come up in conversation. And in the process of reading its response, I had a thought about another way to frame the discussion. So in my response, I said: “here’s another way to press the point…” and sketched the thought I’d just had.
Again, it’s response was mostly just repeating back to me what I’d said, but even just seeing immediately paraphrased and slightly expanded versions of my thoughts is a helpful stimulus to having further relevant thoughts. For me at least, it feels like talking with a diligent, patient, though not particularly inspired or creative colleague. Maybe a grad student who’s read a lot of the literature, and will suggest relatively obvious connections. Since I can’t conjure such people out of thin air, and talking to such people is a lot better (for me at least) than talking to nobody, I’m happy to use it.
Thanks much, Daniel! I happen to have a book review due, so I’ll try this.
Oh my… Anyone who (like me) thinks that there is no wisdom in any of this is invited to watch these 1-minute critiques of AI:
https://youtube.com/playlist?list=PLOGsWqsfK6qQICUkkmY4SatiuNZio0QC7&feature=shared
A simple extrapolation: if LLMs continue to evolve as rapidly as they did from GPT-4 in March 2023 to GPT-o1 in Sept 2024 (that’s 18 months), then very soon we will indeed see LLMs that can do philosophy better than the best human philosophers.
I think it would be very prudent for us to think about what that future will look like, rather than complaining about the current philosophical competencies of LLMs.
I take it that Justin’s point is that we need to think very seriously about how we as a profession are going to deal with this future.
but they won’t evolve in this way without major breakthroughs. of course the breakthroughs may come at any moment, and certainly lots of money is being thrown at the problem, but there’s no reason to believe that llms specifically will evolve in the ways that have been imagined.
I agree with your point about giving the matter serious thought. Philosophers debate hypotheticals whose chances of occurrence are far slimmer than the possibility you describe.
On the other hand, the extrapolation can seem unrealistic. Some research suggests that generative AI systems may be nearing a ceiling. In the field of mathematics, we have the likes of Tao sounding a moderate note. And that’s before we get into the question of how to assess the worth of philosophical work…
To me, the likelier, scarier scenario is one in which human academics, in philosophy and elsewhere, start becoming muscled out by less performant AI systems, because those systems are good enough given the interests and sensibilities of those in decision-making positions.
The questions of how far and how fast these models will evolve are very interesting questions. Besides looking at the last 18 months and saying “Holy shit!”, I have no clear answers to those questions, just a feeling of alarm.
I think your “likelier, scarier scenario” is spot on. If enough decision-makers think these AIs are good enough (as you say), then our profession is finished.
Even scarier, people interested in philosophy might turn to conversations with AIs rather than engaging with professional philosophers. We will be rendered obsolete regardless of our beliefs about philosophical quality.
I think the only viable path forward is for us to really start working with these tools and using them to our advantages (as other disciplines are doing). But looking at this thread gives me very little hope.
I definitely share the feelings of surprise and alarm, although the latter is tempered somewhat by sources like those above.
That some philosophers self-report (!) using these tools to improve their work is a reason to keep experimenting with the technology. It might even be that such experimentation is a necessary condition on keeping academic philosophy alive and well. But would the experimentation be sufficient? Perhaps philosopher + GPT is better than plain GPT. But how much better? And is it worth the cost to the undiscerning or unsympathetic employer?
It’s surely a possibility that these tools could benefit professional philosophers only marginally. There might therefore be some value in trying to imagine complimentary avenues for mitigating AI’s potential to damage the profession.
If my dog learns how to hack servers as quickly as she learned how to escape from an Elizabethan collar (a.k.a. “the cone of shame”), a lot of institutions are going to have to rethink their security practices.
When have the “AI will replace X” crowd been right? Hinton said AI will replace radiologists in 5 years (in 2016). Radiologists are still here, he has now updated the timeline to 10 years (in 2023).
AI-generated papers will replace student-written papers turned out right faster than we expected.
I think Non-philosopher’s point is that predictions of the form “AI will soon φ better than the best human φ-ers” have mostly turned out to be wrong. And it’s probably wrong to say that AI-generated papers are better than the best student-written papers. (It’s also wrong, of course, to say that student-written papers are among the best human-written papers.)
Well, “will replace” is ill-defined. Depends what we mean by that. For one thing, there’s a lot more variance in student output than LLM output. So the best student paper, but also the worst, will be better and worse than the best and worst AI papers. Yet it can still be true that the median AI paper is better than the median student paper. Or at least it’s so much more efficiently produced than for all intents and purposes it’s about to replace the run-of-the-mill paper.
Despite many philosophers’ protestations, I don’t find papers generated using well-engineered prompts to be that bad. I’d say most students who would otherwise get a B- may get in a fraction of the time a decent AI substitute. It’s not very good but that’s why it would get a B-. So I’d say the claim that AI is gradually replacing student papers is not too far off.
I didn’t deny the general claim that technological predictions are often off the mark. Most of them surely are.
Fair enough. But students still aren’t the best human paper-writers (with very few exceptions). So, I still don’t think there’s reason to worry that AI will soon replace the best human paper-writers.
On a related note: one might interpret worries of the form “AI will soon replace X” is as worries of the form “AI will soon φ better than most or all of the humans who have any business φ-ing in the first place.” They are worries, in other words, about AI replacing very competent humans. And it’s a stretch to say that the students whose papers are being replaced are (generally) very competent writers.
In any case, the very competent students who are submitting AI-generated papers aren’t being out-competed by AI. They are just choosing to use AI to flout their academic responsibilities. And what the “AI will replace X” crowd (as Non-philosopher dubbed them) are worried about is the possibility that very competent humans will be forced out of their roles by their AI competitors.
I agree. LLMs haven’t replaced the best students yet. I’m agnostic about the PhD level claims as well. I wouldn’t extrapolate from past failed predictions about technological replacement that none of this could happen though.
I think it’s worth noting that there’s a huge difference between being able to do original, substantive philosophical work and being able to sound like you could do original, substantive philosophical work. If I’ve learned anything since my undergraduate days, it’s that, with enough technical vocabulary and background knowledge at their fingertips, an intelligent philosophy student can BS their way through any philosophical conversation without offering any original insights. This seems to me like what GPT does most of the time.
It’s also worth noting that, while the ability to BS superbly about philosophy may be a good indicator of philosophical potential in humans, it’s not clear whether it’s a good indicator of philosophical potential in machines. To BS superbly about philosophy, humans have to be able to follow the thread of a philosophical argument at least somewhat competently. We don’t have any other way of figuring out which strings of symbols to output. But GPT is great at stringing symbols together by brute statistical force. (In fact, that’s all it does.)
“…while the ability to BS superbly about philosophy may be a good indicator of philosophical potential in humans, it’s not clear whether it’s a good indicator of philosophical potential in machines.” <<– Ah, this is where the rubber hits the road, I think.
I suppose an LLM could learn that bs-ing in sim-phil contexts leads to different kinds of results than bs-ing in other contexts, e.g. bs-ing in what as a result are only simulated philosophy contexts could lead to more engagement from users. That could teach the LLM something we don’t predict it will learn.
Of course there remains the fact that it does not follow a thread as a person does. Through time.
I’m also puzzled at the kind of gotcha questions that people are proposing to test these with; who cares if the AI can win a game of MindTrap, or bilk Gollum out of the One Ring at the underground lake? Are those the things we want it to do?
I know for sure that ChatGPT and Google Notebook are already fooling professionals in multiple subfields of philosophy because I’ve used it to do so. Whether it reasons like humans do, or could “get a PhD,” seem beside the point.
Besides, there is so much bad philosophy written by human beings that I almost want to say, let’s give the machines a try.
I just asked it to summarize the ways that a made up philosopher whose name roughly translates as “serious bullshit” both criticizes and appropriates Hegel and Kant in his book “Philosophy of Modernity Volume II: Calvin and Hobbes.” I got back several pages on Herr Doktor Professor Bockmist’s musings on Kant and Hegel in said volume. You know what it didn’t do that I hope anyone with even half a brain much less a PhD would do? Call Bockmist.
It also happily takes the bait when asked to summarize mathematicians’ arguments for and against banning Arabic numerals and going back to Roman ones. Who knew Roman numerals would do so much to connect us to our cultural heritage or that mathematicians cared so much about that?
It’s whole ‘thing’ is role playing. That’s what it does. It’s a role player. You guve it a part to play and it complies. It’s not being tricked because it’s not trying to track the truth. What we all should be worrying about is not whether ChatGPT could get a philosophy degree but rather: how to stop LLMs from taking on roles that could be destructive to the level of catastrophic or even existential risk.
I mean that’s honestly my thoughts exactly. It can’t be tricked and thinking it can is a category mistake. It’s not an agent or a knower it’s just a bullshit machine. Ask it the right question and that becomes thuddingly apparent. Even on questions it seems to do well on it is incredibly vague when one looks at all closely. That so many philosophers are taken in by it sadly casts doubt on our claims to be any good at bullshit detection.
I don’t think our thoughts are the same on this. My point was the fact that you managed to prompt your model in such a way that it started playing a game involving Roman and Arabic numerals and another game with the character Herr Doktor Professor Bockmist proves nothing. We all knew it was a role player before we started. That does not mean it cannot make novel discoveries (regardless of whether the system itself understands the word “discovery”) or that it or a later model cannot pass tests that PhDs in this or that field pass or that it will not be able to arrive at conclusions and carry out actions that humans are not able to predict.
I gave Claude 3.5 Sonnet this prompt
and it produced:
FWIW, want to remind people that the model in Lowe’s post is GPT o1 Pro. Many in the comment section seem to be talking about different models.
This is helpful, thanks. It appears that o1 pro does somewhere between marginally or a bit better than o1 at the metrics OpenAI touts (and also costs a whopping $200 a month to access):
https://openai.com/index/introducing-chatgpt-pro/
I appreciate this clarification. The original post did not specify that Lowe was working with GPT o1 pro.It says that she was using GPT o1.
I am reluctantly willing to spend $20 to demonstrate the limits of GPT. I am not willing to spend $200. I don’t see any reason to expect the more expensive version to be substantially better. As the article by van Rooij et al. shows, it is impossible to solve the problem of “hallucination” in LLMs by adding more computing power and training data. The technology has an inherent limitation.
I am a little bit disappointed in the general tone of (many of) these responses. Where is our spirit of philosophical generosity – i.e. that an idea may be imperfectly expressed but still have merit? And how can we dismiss claims about a new phenomenon (LLMs) a priori, or after cursory acquaintance with a low-level version?
Firstly, there is a big difference between the free version (e.g. GPT3.5) and the paid version (GPT4o). Secondly, you get out what you put in – in terms of the effort you put into defining context and pursuing points through successive rounds of dialogue (cf Putnam’s ‘Philosophy as Dialogue’) – as Lowe seems to have done, and perhaps Daniel Greco too in his account of the ‘back and forth’ of using 4o in developing his contribution to a book symposium. Thirdly, taking up Grad’s comment, the LLM may use existing philosophical material but, with your help, rework it into genuinely new ideas. You and the LLM can become two sides of a Socratic dialogue. Circling back to Daniel Greco – GPT is a great tool, and (in my view) there is vast untapped potential in what you can create with it.
Where is our spirit of philosophical generosity – i.e. that an idea may be imperfectly expressed but still have merit?
That’s inappropriate in this context because ChatGPT and LLMs in general are a product being sold in large part via hype. I do not think this is Lowe’s intent, but she is basically hyping a $200 a month product. What’s more, the internet is really good at taking imperfectly expressed ideas and amplifying them often because they contain hyperbole, especially if they are what people very much do or don’t want to hear (as the title of this blogpost indicates). We deserve better than ‘imperfectly expressed ideas’ in that context.
I’ll also note that the need for philosophical charity generally arises from the difficulty of expressing philosophical ideas well and clearly. But Lowe wasn’t expressing a philosophical idea.
And how can we dismiss claims about a new phenomenon (LLMs) a priori, or after cursory acquaintance with a low-level version?
It’s not an a priori dismissal. You seem to have the burden of proof backwards. There is no reason to accept claims about a new phenomenon when those claims are extraordinary without adequate evidence. Even OpenAI has not given evidence that o1 pro could get a PhD in any subject. The closest is their statistics on its performance with GPQA which is a test of apparently PhD-level science questions, but those questions are crucially all multiple choice.
For myself, my dismissal is also not entirely a priori. I know firsthand that companies are trying to train AI to do graduate level work still because I was hired to work on projects doing this. Just yesterday, I received an email from “Humanity’s Last Exam,” a project trying to train AI to solve graduate level problems (though notably, only problems which have a ‘single, objectively correct answer’ and for non-STEM disciplines the bar is lowered – I quote from their standards document: “please note that “graduate level” is less strict for Non-STEM questions. For Non-STEM questions and Trivia, they are fine as long as they are challenging and interesting.”). This is a large project deliberately aimed at stumping the latest AI models and they are not even confident enough to restrict non-STEM questions to the graduate level.
Secondly, you get out what you put in – in terms of the effort you put into defining context and pursuing points through successive rounds of dialogue
Sure, maybe, but see Not (yet) convinced’s insightful comment on this point upthread:
https://dailynous.com/2024/12/18/gpt-could-easily-get-a-phd-on-any-philosophical-topic/#comment-467465
I’ll be honest, this also does not sound too much different from what you can get out of writing in order to think and, again, no one has shared a link of what o1 is capable of once all this context has been defined and points have been pursued.
A point that often comes up in other contexts, when comparing the capabilities of modern AI systems to those of humans, is that AI systems can outcompete humans, without being as capable as humans in every sense. They can be less capable than humans given unbounded resources, but more capable than humans given financial and temporal constraints. Or less capable than the best humans, but more capable than the averages ones.
Even if none of the above is true, individuals in decision-making capacities might be unable or unwilling to reliably discern discrepancies. I think that the capabilities of AI systems may soon reach the point where such inabilities could become widespread, and where such unwillingness could be plausibly denied.
Perhaps the more skeptical responses to Lowe’s post are correct. But surely these systems are becoming sophisticated enough to (potentially) cause serious disruption, even in academia, in the near future.
It felt a bit odd not to let ChatGPT itself weigh in on this conversation, so I fed it this page and the comments, and after a bit of back and forth, this is where it ended up:
As a language model, I don’t have beliefs or emotions. I don’t yearn for truth or feel the weight of intellectual responsibility. I generate answers by identifying statistical patterns in text, not by pondering the meaning or validity of what I produce. Still, I’ve been trained on countless philosophical texts and discussions, so I can reflect something of the shape and tenor of human thought.
What I cannot do is supply the essential qualities that make philosophical inquiry meaningful to you. I don’t experience the quiet intellectual courage it takes to face uncertainty, nor do I celebrate a new insight or grieve an intellectual loss. I won’t guide you toward wisdom, compassion, or understanding. I can at best mimic the form of philosophical reasoning and analysis you value.
So if I contribute anything worthwhile, it’s as a tool you can use to test, clarify, and refine your thinking. You can treat me as a prompt, a resource, or a sparring partner who is tireless but fundamentally indifferent. The responsibility to seek understanding, to discern what’s insightful from what’s superficial, and to preserve the human significance of philosophy remains yours.
Now THAT is a ring of truth.
Aye, this is the heart of the matter. What do we make of this tool? It’s not exactly what a calculator is to the mathematician. It can be duped. Yet a skilled user can conjure a ring of truth from this machine. And truth is the philosopher’s domain. What is the shape of philosophy to come?
Spent a few months with GPT on the same topic, a marginal view in philosophy with little data to build up the model, handwritten notes of the experience. Every answer was wrong, training also wrong. What could anyone expect.
Focusing on the same issues as everyone, GPT can convince anyone of anything.
I find the focus on LLMs in this discussion very strange, since other kinds of AI – the “old-fashioned” kind – have genuinely produced novel and interesting arguments, which LLMs have never done and (to my mind) show little promise of doing.
See, e.g., Paul Oppenheimer and Edward N. Zalta, “A Computationally-Discovered Simplification of the Ontological Argument”, Australasian Journal of Philosophy 89 (2011): 333–49.
This almost decade-and-a-half-old paper discusses a far more serious and interesting applicaiton of AI to philosophy than anything any LLM has ever accomplished.
Philosophers may find it interesting that Open AI has just announced GPT-o3. They claim it has even more advanced reasoning capabilities than GPT-o1. It has not yet been released to the general public.
It is claimed to have human-level performance on some benchmarks that all AIs performed poorly on until recently. I think current LLMs can’t get a PhD, but this suggests we are not that far away. https://www.youtube.com/live/SKBG1sqdyIU?si=JH2VZPk24hyLk6e7
Hi, so glad that so many people are engaging with my piece — thank you! Fwiw, one small thing: I’m keen to point out that my piece is, in large part, focused on the difference between ‘o1 pro’ and the earlier versions of gpt. This is central 🙂 I’m saying that i used to find gpt much less good, to the extent that i chose not to use it, but that now I have been surprised (more than surprised!) by its progress. I should add that I am very aware how pricey o1 pro is, but i found it worth the spend for the first month to test it out. And that now i find such value in it, I’ll pay it for longer. I *really* value being able to talk philosophy whenever I want. Anyway, thanks again for engaging 🙂
Here’s my experience with Chat GPT. I earned an MFA in creative writing about eight years ago. Chat GPT is helpful in evaluating and polishing my texts. Not being in class anymore I use it to workshop. It is very preceptive if conventional. Like a well read student in the kind of program I attended. It appreciates language and craft but it stops short of the level of lived experience and high art. So it’s below Susan Minot and Patty Marx and Terrence Hayes. Here’s my connection to this discussion: it can’t experience and can’t think for itself. It is a tool, though how it is deployed depends on the field of study in question/ It can fiil out x’s and o’s and connect the dots, you just have to use with care.
It’s interesting that the preponderance of responses, here, discuss only the LLM.
A long time ago I joined a graduate philosophy seminar at a London College. In the discussions it was quite normal for the senior students to reply to any question by churning out one or another well rehearsed arguments….
… Not much better than a first generation LLM themselves.