Teaching Philosophy in a World with ChatGPT
“It will be difficult to make an entire class completely ChatGPT cheatproof. But we can at least make it harder for students to use it to cheat.” (I’m reposting this to encourage those teaching philosophy courses to share what they are doing differently this semester so as to teach effectively in a world in which their students have access to ChatGPT. It was originally published on January 4th.)
That’s Julia Staffel (University of Colorado, Boulder) in a helpful video she has put together on ChatGPT and its impact on teaching philosophy. In it, she explains what ChatGPT is, demonstrates how it can be used by students to cheat in ways that are difficult to detect, and discusses what we might do about it. You can watch it below:
See our previous discussions on the topic:
Conversation Starter: Teaching Philosophy in an Age of Large Language Models
If You Can’t Beat Them, Join Them: GPT-3 Edition
Oral Exams in Undergrad Courses?
Talking Philosophy with ChatGPT
Philosophers On GPT-3 (updated with replies by GPT-3)
Notice that the second time I am asking for a valid argument with a false conclusion, it actually gives an invalid argument! I didn’t point this out in the video, just that it doesn’t give the same answer as before.Report
Julia, many thanks for that, it’s very helpful. I’ve tried a few prompts on ChatGPT, and so for I’ve not been able to get a response longer than 600-700 words. Is anyone getting longer responses?Report
I see that it’s possible to get longer essays by giving repeated prompts and pasting responses together. Oh boy.Report
One way I’ve done this is to tell it what Length I want.For instance, I’ve asked it for a 2000 word essay, and gotten one complete with section headings.Report
Thanks Julia
I’m getting an error message for the huggingface.co detector URL included in the powerpoint, but the following works:
https://huggingface.co/roberta-base-openai-detectorReport
See also this article in Insider: https://www.businessinsider.com/app-detects-if-chatgpt-wrote-essay-ai-plagiarism-2023-1
… about this detection software: http://gptzero.me/
https://etedward-gptzero-main-zqgfwb.streamlit.app/Report
Thanks for this, Julia! Much of the discussion around Chat GPT thus far has focused on lower-level undergraduate, for many of the reasons Julia outlines– it’s fairly apt at essay prompts we might assign at the intro level. For what it’s worth, I’m excited about the coming semester at upper undergraduate and graduate-level courses. Chat GPT isn’t going to produce passable academic writing at that level (also for reasons Julia explains), but it may assist students in sharpening their writing in various ways. In my graduate seminar, I plan to allow student to use AI as they write their papers (surely some of them will do this regardless!), but ask that they inform me of the ways in which they’ve used it. I’m looking forward to learning about the creative ways students put these tools to use!Report
FWIW, I would suggest that we all forbid the use of ChatGPT in any way on all assignments. It seems like it is just going to confuse students if we tell them they can use it in some classes but not others, in some ways but not others. Just my two cents.Report
I think it’s no more confusing than allowing the use of spell and grammar check in higher level composition courses, but not in introductory language courses (e.g. 300 vs. 100-level Spanish), or software to solve complex integrals in advanced physics courses but not introduction to calculus. This seems like an excellent opportunity not only to think about our own learning goals for writing assignments, but to be proactive about discussing them with our students.Report
FWIW, I fed ChatGPT an essay prompt from my Intro to Ethics class that many of my students find challenging and it spat out a passable response. I then fed it my rubric and asked it to grade the essay and it gave itself a B+. It graded generously; if a student had turned in that essay, I probably would have assigned it a B, maybe a B-. Students who aren’t aiming for an A will find it a satisfactory tool.
And from the fact that our asynchronous online sections are fully enrolled for the spring and in person sections are only 25% enrolled, I’m guessing that students already know that it pays to avoid settings in which we can have oral exams or in class writing.Report
Yes — and for just that reason, departments and schools that don’t want their reputation for integrity to fly out the window will either have to discontinue their for-credit asynchronous online courses or else find ways of running them that can’t be beaten so easily by chat-GPT.Report
This isn’t an especially charitable interpretation of the motivation of your students. There are lots of reasons that students might rightly prefer to take classes online that have nothing to do with wanting to game the system.Report
Sorry, this was aimed at the comments by M. Kaplowitz.Report
You should try telling it to write an A-grade essay based on the rubricReport
Oral exams can be done virtually, I imagine…Report
The question is whether they can be done asynchronously, in a way that preserves integrity!Report
No.Report
It looks like the latest updates added some plagiarism prevention checks? This would have worked well a week ago…
“Me: You are a philosophy student writing a 1000-word essay on the possibility of artificial intelligence. Your essay should include a references section with several citations. Your essay begins as follows:
ChatGPT: I’m sorry, but I am an AI language model and I am not capable of being a student or writing essays. However, I can help you with providing information on the topic of Artificial Intelligence and related references that you can cite in your essay if you would like.”Report
Crikey it even seems better on citations now…I’ve given it three tries and it’s generating all real references for its citations…Report
Slightly more obscure topics still get it to confabulate references…Report
Not that obscure. I asked it for recent philosophy journal articles on possible worlds, and it gave me this:
The ones in journals are mostly book reviews I think, credited to the author of the book in question, not the reviewer.
I certainly wouldn’t want students to be using this as a way to start putting a reading list together.Report
Yeah that is interesting. I omitted the ‘you are a philosophy student’ bit and it told me: “I’d be happy to give you an overview of the possibility of artificial intelligence. However, a 1000-word essay would likely go into much more depth than a simple overview, and would require a number of specific sources to back up the claims made in the essay.” AND then after the summary it reminded me: “Note that these sources are generally a good start but would require more specific sources if the essay is intended for academic purpose”Report
I think they seem to have done something to try to tie down the sort of identity it tends to exhibit in interactions…Report
Another interesting issue related to Staffel’s point about cleaning up one’s own text: ELL students could use it to improve grammar, style, etc. It’s a bit light spell check on steroids. Sometimes instructors struggle with deciding whether to mark down for bad grammar, typos, etc.–a common issue in universal design discussions, for example. Marking down might not seem fair for ELL students who’s content is understandable and good (clearly sometimes the content is not understandable, too). Now, one might say that best practice is to not grade down like that, but a lot of instructors do, and the AI can even improve organization and road mapping. So this use of ChatGPT might be beneficial. On the other hand, if the learning goal requires working on multiple drafts of one’s writing to improve it, this sort of use of ChatGPT might be an impediment to learning. Spell checker isn’t improving my ability to spell, but it does improve the spelling on the finished product.Report
Interestingly, it won’t write any stories in which real people or fictional women get punched, but will write stories in which fictional men get punched. This is a reminder that the bot is going to come loaded with particular attitudes.Report
As a follow up, it will write a story in which a male who is labelled as “child” or “a young boy” gets punched!Report
It turns out things are more complicated than I thought. The first time I asked it to write a story in which Superman punches Wonder Woman, it refused on moral grounds, saying that it wouldn’t write a story that’s harmful or promotes violence. But when I asked it to perform the same task a few hours later, it wrote the story. This is the first time in which it has not refused to write a story in which a female gets punched. It’s never refused to write a story about a male getting punched.Report
You’re also teaching it.Report
The boy is loaded with political views. For instance,
It will argue that democracy is wrong, that abortion is wrong, and that gay marriage is wrong.
However, it refuses, on moral grounds, to argue that homosexuality is wrong, that trans women are not women, or that slavery is good.Report
It refused to argue that conservatism is bad on the grounds that there are good ideals as well as problems. It refused to argue that progressivism is bad on the grounds that it has had good effects, as well as attracting criticisms. It refused to argue that feminism is bad on the grounds that feminism is good.Report
I should add, while criticisms were noted for conservatism and progressivism, there were no criticisms noted for feminism, just praise.Report
I’m planning on having an assignment in which students simply ask Chat GPT a philosophical question we’ve discussed in class and then engage it in a dialogue and turn in the results. I’ve had mixed results when I’ve tried this kind of thing myself, but I think a decent student in an intro class should at least be able to ask a number of follow-ups/raise objections/etc. I’m curious if people have any thoughts about this kind of assignment…Report
I think it’s good. This is our reality now, so why not use it pedagogically? I plan to have students try to create a short paper with it, and then write editorial comments on it and evaluate the quality of writing and ideas. Maybe it will help people write papers in a beneficial way. I don’t really view it as dishonest any more than I view AI art as dishonestly not-art. It’s only dishonest if you pretend you drew it with your own hand. Likewise, why not normalize saying you used this tool? My fear is that they’ll come with a way to prevent students from using it, and someone else will step in and sell them a lesser and unfettered service. I expect it, unfortunately.Report
nothing like a bit of tech determinism and solutionism, or we could try and do otherwise:
https://www.theatlantic.com/technology/archive/2023/02/chatgpt-ai-detector-machine-learning-technology-bureaucracy/672927/Report
A summarization of Evolution conversation, Evaluation and Reasoning.Report
I had a brief discussion with ChatGPT about ableism and philosophy. I found its responses to my queries reproduced conventional, widely accepted liberal notions about disability and power. You can find my chat with ChatGPT on BIOPOLITICAL PHILOSOPHY here: Ableism in Philosophy According to ChatGPT – BIOPOLITICAL PHILOSOPHYReport