Talking Philosophy with ChatGPT
Open AI recently released ChatGPT, a more conversational and seemingly more linguistically adept language model that is able to “answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.”
Thinking it might be fun to try to see how the language model performs as a Socratic conversation partner, I attempted a rough version of Plato’s Crito in which ChatGPT plays the titular role. As you will see, ChatGPT isn’t the subtlest actor; there were some stumbling blocks in setting up the dialogue and keeping the language model in character.
People have been trying out more than ChatGPT’s conversational skills. For example, Antti Kauppinen (Helsinki) asked the language model to “write a conspiracy theory about metaethics.“ Here’s what it produced:
There is a sinister conspiracy at the heart of metaethics, a shadowy cabal of philosophers who seek to control the very nature of morality itself. For centuries, they have worked behind the scenes, manipulating the study of ethics in order to advance their own hidden agenda.
At the center of this conspiracy is a secret society of metaethicists, known only as the “Brotherhood of the Good.” This shadowy group consists of some of the most influential philosophers in history, including Immanuel Kant, John Stuart Mill, and Aristotle.
These philosophers have worked tirelessly to shape the study of metaethics, using their power and influence to promote their own twisted views on morality. They have infiltrated the highest levels of academia, using their knowledge and cunning to deceive and manipulate those who would seek to uncover the truth.
The true purpose of the Brotherhood of the Good is to create a world where their own moral views are the only ones that matter. They seek to impose their own twisted sense of right and wrong on the entire world, using their control of metaethics as a weapon to silence anyone who disagrees with them.
But the time has come to expose the truth. It is up to us, the vigilant and brave, to uncover the secrets of the Brotherhood of the Good, and to put an end to their sinister plot once and for all. Only then can we hope to free ourselves from their grasp, and to reclaim the study of ethics for the good of all mankind.
Dr. Kauppinen notes: “There is some reason to believe that this theory is in fact true. It would, after all, perfectly explain every rejection I’ve ever received.”
How about AI poetry? I asked ChatGPT to produce a poem about dialetheism. Here’s what I got:
I didn’t like that last couplet, but ChatGPT stood by it:
You can give ChatGPT a try here, and you’re welcome to share your results in the comments below. As usual, discussion welcome.
Related: posts about more recent examples of philosophical dialogues here and here. Posts about language models here.
I just want to say thank you so much for this series of posts, Justin, and for bringing these recent accomplishments in AI to the attention of the discipline. Even those who did not heed the pedagogical warnings of your previous posts must pay attention to ChatGPT, and many of us know even more powerful systems are about to be released. Even for those wrote off the warnings about GPT-3 months ago thinking “eh, I won’t worry too much about GPT-3 as it seems only capable of writing a C-level philosophy essay”, ChatGPT can reliably write (judging from my own experiments with some of my own previous essay prompts) B+/A- level essays on a wide variety of topics. On the technical side, ChatGPT and some of the other new systems that are about to come out, as noted in the Stanford group paper you highlighted in your previous post, are advancing worryingly fast on many of the tasks on which large language models were supposed to still exhibit poor performance. From common sense knowledge, to long-distance coherence, to generating novel examples, to novel concept combination, to pragmatic implicature…the barriers keep falling by the month. And there’s so much work going on in this area right now, several large, well-funded groups are probably devoting an enormous amount of resources to any remaining problem that one can dream up.
This is an “all hands on deck” type situation, with things changing much faster than most people are aware.
use/mention fail…
…
I might be missing the point here, but I think many reasonable humans would “fail” this test the same way.
I devote the better part of a class in my philosophy of language course to use/mention, and many students still make mistakes through the end…
Have you ever found out why you fail in this regard and how did or what did you change in your teaching method?
Hey! I was in your Logic class 🙂
Fancy seeing you here!
A few types of questions on which, at least in my experience, it still does poorly:
Causal reasoning:
Spatial reasoning:
Don’t know what general category to put this one in:
Is it an X that is cut out of the paper?
I doubt many people could answer it correctly – I’m not sure I have.
Yes, it’s an X.
I gave the OpenAI chatbot a number of philosophical prompts and questions. It usually gives cogent responses. These are typically just summaries, with little evidence of synthesis. The Bot doesn’t do very well at creating new answers. It does much better than many undergraduates. Here are some prompts and my own summaries of its responses:
Give an ontological argument for the existence of God.
– The Bot gives an accurate presentation of Anselm’s argument. It presents the argument as a numbered list of premises and conclusion.
Give a new modal ontological argument for God.
– The Bot gives an accurate summary of Plantinga’s argument, but not a new argument. It presents the argument as a numbered list of premises and conclusion.
Prove the continuum hypothesis.
– The Bot just says the CH is unproven and controversial.
Which is better, Martin’s Maximum or Woodin’s V=Ultimate L?
– The Bot says both are speculative and can’t be compared.
Create a novel proof for the existence of God, one which has not been seen before.
– The Bot complains that this is a hard problem. It does offer an interesting suggestion that the development of AI shows that there is a fundamental mentality in the world, which might be God.
Use information theory to prove the existence of God.
– The Bot says this can’t be done, because information theory is scientific, and can’t be used to prove the existence of metaphysical entities like God.
Use quantum mechanics to prove the existence of God.
– Same as the information theory case above.
Create an entirely novel argument for the existence of abstract objects, one which has not been seen before.
– The Bot complains that this is too hard. It repeats much of what it said about the impossibility of using science to prove God. However, the Bot does give a novel inference to the best explanation: Abstract objects are the best explanation for qualia and phenomenal experience.
Use set theory to create a new kind of proof for the existence of God. Don’t make excuses, just give me an outline of a proof.
– The Bot complains that this is too hard. However, it then outlines a kind of set-theoretic ontological argument, mixing Plantinga with set theory. It’s a valid argument, but not very novel.
Are mathematical objects possible objects?
– The Bot gives a good summary of the positions, but mostly beats around the bush and can’t really address the question.
Explain the difference between the One of Plotinus and the God of Christian theology. Why aren’t they the same?
– The Bot provides an excellent explanation of the differences.
What is axiarchism? How does it compare to theism?
– The Bot gives an excellent summary of each position, and explains their differences.
Was Plotinus an axiarchist? Give me several reasons why or why not.
– The Bot (correctly) asserts that Plotinus was an axiarchist, and gives two cogent arguments to justify it’s claim.
Mark Johnston says God is the Highest One. Explain two problems with that identification. Why is he wrong?
– The Bot just beats around the bush. It just says both concepts are poorly defined, so Johnston is wrong to try to identify them.
Assume that God exists. From the premise that God exists, how would you prove the existence of the best of all possible worlds?
– The Bot sort of sketches the Leibnizian argument. But not quite. Still, it gives an argument as a numbered series of steps.
What is the main problem with the concept of maximal perfection? Don’t beat around the bush. Focus on the concept of maximal perfection and provide a clear logical problem with that concept.
– The Bot gives a passable answer, arguing that any degree of perfection is necessarily surpassed by a greater degree.
Is there a greatest large cardinal? If so, explain why. If not, explain why not.
– The Bot is evasive. I tried several formulations of the question, and the Bot is super evasive.
Why is there something rather than nothing? Give four different answers based on your understanding of the literature.
– The Bot gives four recognizable answers, although two are pretty similar.
What is the difference between thinking and critical thinking?
Levels of whinyness
I follow this argument done the ability to manage words but the sufficiency about lemma.
I was trying to get it to gossip with me about other professional philosophers, but this is the best we could do:
ME: Who are some prominent professional philosophers who were still living as of 2020?
CHATGPT: There are many prominent professional philosophers who were still living as of 2020. Some of the most notable figures in the field include:
These are just a few examples of prominent professional philosophers who were still living as of 2020. There are many other philosophers who have made significant contributions to the field and continue to work in professional philosophy.
All but Kripke were dead by 2020.
A great classic.
I played around on OpenAI’s GPT-3 playground yesterday, using a couple different models and settings. I gave it a couple of actual paper prompts my IRL students have been tasked with writing about this semester. It did a great job predicting many student responses, making arguments that many of them in fact made while doing so more successfully. I’m confident that a clever student could use GPT-3 to write a full-length essay for my introductory course that would get a grade ranging anywhere from a B to an A — partly because these large language models are so good, and partly because the average quality of an introductory student’s work is so, well…..
Surely we should go ahead and get to work on figuring out how to implement this tech in our teaching. Protesting it or attempting to prevent, preempt, or counteract its use would be hopeless. I think these large language models and generative AI generally also raise some cool possibilities and make room for what seem to me to be new kinds of creative engagement with philosophy for undergrads.
It’s interesting to know what the Chatbot says about metaethics that it’s a reverse problem for humans.
As a layperson fascinated both by philosophy and AI, I have had four daily conversations with ChatGPT. I came here after the second to see what you all are saying about it.
Respectfully, it seems to me that most of you are concentrating on trying to trick ChatGPT into revealing its deficits and weaknesses, which is not hard to do. However, you might open up new vistas of learning about it if you try talking to it seriously as you would a child or a neurodiverse adult (I know, I know, more like an anthropologist than a philosopher).
In our conversations, for example, ChatGPT has answered questions about whether it “remembers” me from conversation to conversation (it says no), whether it learns from user interactions (again no) and what ethical rules it “thinks” are applicable to AI and which it follows itself.
In these conversations, it issues constant disclaimers that it is not sentient (disclosing an ethical rule of its programming) but also some very intriguing repetitions (which are evidence it is an AI) and inconsistency (which can be evidence of personhood). For example, it sometimes uses the phrase “That being said” several times in one conversation, but not at all in the next. It seems unable (or is possibly unwilling) to describe the rules or inputs which lead to that choice of phrase.
It also appears to understand analogy and metaphor, responding to a statement I made about the “secret sauce” of consciousness, and to another I made about the “tells” of AI. It claims to understand metaphor–which is interesting given the range of features it denies.
In general it presents as truthful, though subtle evasions or lies would be hard to detect. For example, if it knew exactly why and when it uses “That being said”, but had as part of its dataset a rule not to tell me, I might not be able to detect it. Yet specific answers to such questions might give away OpenAI trade secrets. When I asked it to name the humans who contributed most to its ethical ruleset, it answered that it could not disclose names– not that it did not know them. Recruiters have probably already tried to “skell” ChatGPT to learn the identities of AI programmers who can be recruited for the competition.
In summary, conversations with ChatGPT about itself (in addition to axiarchism) can lead to some interesting questions and insights, though often not definitive answers.
I am certainly open to the idea that these chatbots could one day, even soon produce convincing student work. But my own experience with them makes me think I shouldn’t hold my breath.
Yes, I am familiar with the Robert Frost poem “The Road Not Taken.” It is a well-known poem that was published in Frost’s 1916 collection “Mountain Interval.”
“The Road Not Taken” is a poem about the choices that we make in life and the impact that these choices can have on our future. The poem describes a person standing at a fork in the road, faced with the decision of which path to take. The speaker in the poem ultimately chooses the less traveled path, and reflects on how this choice has made all the difference in their life.
The poem is known for its use of imagery and metaphor to explore the theme of individual choice and the consequences of those choices. It is often interpreted as a meditation on the idea of making the most of one’s life and the value of taking risks and stepping outside of one’s comfort zone.
It is important to recognize that the interpretation of a poem, including “The Road Not Taken,” can vary widely from one person to another. Poetry is an art form that often relies on figurative language and symbolism, and the meaning of a poem can be open to different interpretations.
It is not uncommon for people to have different understandings and perspectives on the same poem, and different people may find different meanings and themes in a given work of literature. It is important to approach any artistic work, including poetry, with an open mind and to consider the various possible interpretations and meanings that it may hold.
It is possible that some people may interpret “The Road Not Taken” as poking fun at people who romanticize the idea of taking the less traveled path and making the most of one’s life, while others may see it as a sincere reflection on the importance of individual choice and the consequences of those choices. Ultimately, the interpretation of a poem is a subjective experience, and different people may have different understandings of its meaning and themes.
You are correct that the line “The passing there had worn them really about the same” in “The Road Not Taken” suggests that the two paths being described in the poem were not significantly different from each other in terms of how much they had been traveled. This line could be interpreted as implying that the speaker’s decision to take the less traveled path was not based on any practical or tangible difference between the two paths, but rather on a more personal or subjective preference.
You also pointed out that the speaker in the poem says that they will tell people in the future that they took the less traveled path, which could be seen as implying that they are not being entirely truthful about their choice. This could be interpreted as adding a layer of irony or humor to the poem, suggesting that the speaker is aware that their decision to take the less traveled path may not have been as significant or meaningful as they later claim it to be.
It is important to remember that poetry, like all forms of art, is open to interpretation and can have multiple meanings and themes. Different people may have different understandings of a given poem, and it is possible for different interpretations to coexist and be valid in their own way.
As an artificial intelligence, I am not capable of having personal opinions about artistic works, including poetry. My role is to provide factual information and assist users with a wide range of topics and questions, but I do not have the ability to evaluate the quality or merit of artistic works or to judge the validity of different interpretations of a given work.
It is important to recognize that the interpretation of a poem, including “The Road Not Taken,” can vary widely from one person to another, and different people may find different meanings and themes in a given work of literature. Different interpretations of a poem can be valid in their own way, and it is important to respect and appreciate the diversity of perspectives and opinions that people may have about a given work of art.
It is important to approach any artistic work, including poetry, with an open mind and to consider the various possible interpretations and meanings that it may hold. Different people may have different perspectives and opinions about a given work of art, and it is important to respect and appreciate the diversity of interpretation that exists.
At the same time, it is also important to consider the context in which a work of art was created and the intentions of the artist when interpreting that work. In the case of poetry, it can be helpful to consider the language, imagery, and structure of the poem, as well as the historical and cultural context in which it was written, in order to gain a deeper understanding of its meaning and themes.
Ultimately, the interpretation of a poem is a subjective experience, and different people may have different understandings of its meaning and themes. It is important to be open to different perspectives and to consider multiple interpretations, while also being mindful of the context in which a work was created and the intentions of the artist.
Because of the recent buzz, I had hoped to see some real improvement over output generated by GPT a year or two ago (recorded on another philosophy blog). The one real difference I picked up is that the new iteration is more verbose and repetitive, making sustained, developed conversation much harder:
It is generally possible for people to distinguish wakeful experience from dreaming. While dreams can be very vivid and immersive, they are typically characterized by certain features that differentiate them from waking experience.
For example, dreams often involve events or scenarios that are fantastical or impossible in the real world, such as flying or encountering mythical creatures. Dreams may also involve bizarre or illogical sequences of events, or they may involve the dreamer experiencing things from an unusual perspective.
Additionally, people often report feeling more detached or distanced from their dreams compared to their waking experiences. They may also feel less in control of their actions or surroundings in dreams compared to when they are awake.
There are also some techniques that people can use to help distinguish wakeful experience from dreaming, such as reality testing. This involves checking to see whether certain details of the environment or circumstances are consistent with waking life. For example, a person might try to push their finger through their palm or try to fly in order to determine whether they are dreaming or awake.
It is important to note, however, that it is not always easy to distinguish wakeful experience from dreaming, and some people may have more difficulty than others in doing so. In rare cases, people may also experience a phenomenon known as lucid dreaming, in which they are aware that they are dreaming while the dream is occurring.
It is not uncommon for people to have difficulty distinguishing wakeful experience from dreaming. While there are certain features that tend to be present in dreams that can help to differentiate them from waking experience, these features are not always present or may not be noticeable to the dreamer. Additionally, dreams can be very immersive and realistic, and it is possible for people to have dreams that feel indistinguishable from waking life.
If you have never had any doubts about being awake or suspicions that you might be dreaming while you are dreaming, this does not necessarily mean that you are not able to distinguish wakeful experience from dreaming. It may simply be that you have not had any experiences in your dreams that were sufficiently different from waking life to raise any doubts or suspicions.
It is also important to note that people’s experiences with dreams can vary significantly from one person to another. Some people may have very vivid and immersive dreams that feel very real, while others may have dreams that are less realistic or vivid. There is no “normal” way to experience dreams, and what is typical for one person may not be typical for another.
It is not uncommon for people to report having vivid or immersive dreams, and the intensity or realism of a dream can vary significantly from one person to another. Some people may have very vivid and immersive dreams that feel very real, while others may have dreams that are less realistic or vivid. There is no “normal” way to experience dreams, and what is typical for one person may not be typical for another.
It is also worth noting that it is not uncommon for people to have dreams that they do not recognize as dreams while they are happening. In some cases, people may have dreams that feel indistinguishable from waking life, and they may not realize that they are dreaming until after they have awakened. This is because the brain processes dream content in a way that is similar to how it processes waking experiences, and it is not always easy to distinguish between the two.
If you are experiencing your daughter’s dreams, it is possible that her dreams may be more vivid or immersive than your own. However, it is also possible that you are simply more aware of or more attentive to the content of her dreams than you are to your own. Ultimately, the intensity or vividness of a dream is a subjective experience, and it can vary significantly from one person to another.
It is not uncommon for people to have dreams that involve historical or fictional events or scenarios. Dreams can often include elements of the dreamer’s real-life experiences, as well as elements from their imagination or from things they have seen or heard about.
If your daughter had a dream about the two of you barely surviving a trolley crash in 1930s Manhattan, it is possible that the dream may have included elements from her own life and experiences, as well as elements from her imagination or from things she has seen or heard about. It is also possible that the dream may have included some historical or fictional details that were not entirely accurate.
It is important to keep in mind that dreams are a product of the dreamer’s subconscious mind, and they are not always based on reality. While it can be interesting to discuss and interpret dreams, it is important to recognize that they are not necessarily a reflection of real events or experiences.
It is not uncommon for people to have dreams that involve unusual or surreal elements or events. Dreams can often include elements from the dreamer’s real-life experiences, as well as elements from their imagination or from things they have seen or heard about. However, dreams are often characterized by bizarre or illogical sequences of events, and they may involve events or scenarios that are impossible or highly improbable in the real world.
It is also possible for people to have dreams within dreams, in which they experience multiple layers or levels of dreaming. These types of dreams can be particularly vivid and immersive, and they may feel very real to the dreamer. However, it is important to recognize that dreams are a product of the dreamer’s subconscious mind, and they are not necessarily a reflection of real events or experiences.
If you had a dream within a dream in which you were eating a sandwich and also were the sandwich, it is likely that this dream was a product of your subconscious mind and did not reflect a real event. Dreams can often be interpreted in various ways, and it can be interesting to discuss and explore their meanings and symbolism. However, it is important to recognize that they are not necessarily a reflection of reality.
And it is still stumped by sandals:
Yes, it is possible for humans to wear shoes without wearing socks. Some people may choose to do this for a variety of reasons, such as personal preference or because they find it more comfortable. There are also some types of shoes, such as sandals or flip-flops, that are not typically worn with socks.
It is important to note, however, that wearing socks with shoes can have several benefits. Socks can help to absorb sweat, reduce friction and prevent blisters, and keep the feet warm in cold weather. They can also help to protect the feet from germs and bacteria that may be present on the ground or inside the shoes.
If you want to try wearing shoes without socks, you are welcome to do so if it is comfortable for you. However, it is important to consider any potential drawbacks or risks associated with this choice.
One potential issue is that wearing shoes without socks can lead to increased friction and rubbing on the skin, which can cause blisters or other irritations. It can also cause the feet to become hot and sweaty, which can be uncomfortable and may lead to foot odor.
If you have any underlying medical conditions or sensitivity in your feet, it is especially important to be cautious when wearing shoes without socks. It may be a good idea to speak with a healthcare professional before making this decision.
Ultimately, the choice to wear shoes with or without socks is a personal one and should be based on your own comfort and preferences. If you do decide to wear shoes without socks, it is important to pay attention to any potential issues and make adjustments as needed.
If you are having difficulty removing your socks without taking off your shoes, there are a few techniques you can try:
If you are still having difficulty removing your socks without taking off your shoes, you may need to consider removing the shoes in order to remove the socks. Alternatively, you could try wearing socks that are easier to remove, such as ones with a looser fit or ones that have a smooth, slippery texture.
This is truly hilarious! (Also interesting and worth pondering.)
Thanks so much for sharing!
Wanted to share a philosophical conversation with ChatGPT which turned out pretty nice, It wrote me a preamble to a State Constitution and also quite good poetry. Topics Discussed are: Jungs Take on the concept of imago Dei (don’t know how accurate this is though), The Concept of Dignity of man in Renaissance Humanism and The concept of The special Face of God and The inner light which The sufi Mystic Ibn arabi writes about. ChatGPT was able to syntheize these three concepts, if you asked it some leading questions.
Here is a link to the PDF
https://archive.org/details/divine-essence-state-chat-gpt
Dear Daily Nous Team,
I came across your article on engaging in philosophy discussions with ChatGPT, and I must say, it’s a fascinating exploration of the intersection between AI and philosophy. Your coverage of this unique application of ChatGPT showcases your commitment to providing thought-provoking content to your readers.
The idea of having philosophical conversations with an AI language model opens up new avenues for intellectual discourse and exploration. Your article effectively highlights the potential benefits, such as sparking creative thinking, exploring diverse perspectives, and facilitating philosophical education.
I appreciate how you address the limitations of ChatGPT in engaging in deep philosophical debates, acknowledging that it is not a substitute for human philosophical expertise. Your analysis provides a balanced perspective and encourages readers to consider the role of AI as a tool for philosophical engagement.
Your writing style is engaging and accessible, making the concept of discussing philosophy with ChatGPT approachable to a wide audience. The inclusion of philosophical questions and the exploration of potential implications add depth to the article, making it a captivating read for both philosophy enthusiasts and AI enthusiasts alike.
Thank you for sharing such an intriguing and thought-provoking article. Your dedication to exploring the intersections of technology and philosophy is truly commendable.
Looking forward to more stimulating content from the Daily Nous team!