“I wonder if these people have ever seen a student’s face when they finally understand something for the first time.”
Jane Sloan Peters, a professor of religious studies and historical theologian at the University of Mount Saint Vincent, was talking with her students about changes she has made to her teaching so as to safeguard student learning from artificial intelligence when “a wave of sadness washed over me, and I actually got choked up in front of the class.”
“Before AI,” I told them, “Students used to work hard to come up with their own ideas. I’d help, and they’d struggle, but they’d come to something that was their own. That doesn’t happen anymore and I grieve that.” Then I felt embarrassed and went on teaching as though nothing had happened.
Her reflections on this experience will resonate with many Daily Nous readers. She identifies one of the many feelings she has been having about how AI is altering education as grief.
AI promises great gains, but many educators sense that with its advent, we have lost so much. In this particular instance, students have lost the freedom to sit comfortably in a space of silence and uncertainty, a space as dark and rich as the spring soil in which seedlings are born. And I have lost the joy of sitting with them, encouraging them, watching as their thoughts take root and grow…
The [American Psychological Association] defines grief as “the anguish experienced after significant loss, usually the death of a beloved person.” We are witnessing the death not of a beloved person, but of love as the grounds of education. Love is the heart of a liberal education—a love of the truth, as well as the kind of friendship-love (philia) between teachers and students that makes it possible to pursue the truth together.
AI sycophants would have you believe that teachers like me are simply scared of a new system that will expose their personal deficiencies and outdated pedagogical methods—intractability about AI is ultimately self-interested self-preservation. Dear teacher, you are not fooling anyone but yourself—those lecture notes belong in the bin.
I wonder if these people have ever seen a student’s face when they finally understand something for the first time. What’s more, I wonder if they’ve ever seen a student experience the unique delight in not knowing. Certainly, the unknown can produce confusion and frustration. But sometimes I see in students’ faces a flash of something like the relish of a traveler who knows the journey ahead will be just as delightful as the destination.
Professor Peters says that this “delight in not knowing” is something that’s especially valuable to cultivate in students studying theology, and I imagine many Daily Nous readers think the same is true of philosophy.
At the end of her piece, which you can read in its entirety here, she says: “despite being aware of the real losses in education, I still believe the love and wonder at the heart of education can be salvaged, somehow. The question is, how?”
(via Zena Hitz)
I have to say, while I feel sorry for these grievers, I have a hard time getting in their headspace. From the educational point of view, I find the prospects of learning via AI exhilirating! I have all these opportunities now to level up across all sorts of areas for free using LLMs; last week I was using it to practice formal methods in philosophy, this week I got sidetracked learning about the history of classical music and now I’m using it to better understand the origins of the Aenid. I’m also constantly curious , more so than ever, I think given the ‘affordance’ of knowing I’ve got answers at my fingertips to the most specific questions I could ask. From a learning point of view, I’ve never had a more exciting time to understnading the world (more deeply than i ever had before) and am constantly wanting to learn more!! (I have found Gemini’s tutor mode amazing by the way, as well as Claude Co-Work – and Notebook LM.)
I think the near-frantic energy conjured by the language choices in this comment versus the poetics/rhetoric of grief in the original post captures the disconnect almost perfectly.
I know the diplomatic thing to do is to do the big tent, equal validity thing (different people respond differently to different things) but I think the grievers are honestly being too dramatic about this! The grievy post says grief is ““the anguish experienced after significant loss, usually the death of a beloved person.” I mean this is an OTT comparison … it’s just ChatGPT we’re talking about, not the death of Grandpa! Chin up! I’m sure we also enjoyed the light in a child’s eyes (the grievers think this has gone out apparently) when they catch a ball outside for the first time, but then Nintendo came and they stopped doing that — their eyes light up now when they get to the next level on the nintendo game, etc. Probably a bit out of date reference but you know what I mean. We need to be able to adapt in life without throwing a navey gazey pity party every time a new technology comes along and catches our attention and shifts our routines/attitudes. And as I see it at least, if we wanna get knolwedge and truth, we coudl do a lot worse than having frontier model LLMs in our pocket (and for the record, I still get a twinkle in my eye of the sort the grievers are talking about, when I learn an interesting new fact from ChatGPT!! (such as that Istanbul is in two continents at the same time!! Learned that on the new 5.5 model) Twinkle in eye? Check!
“Every time a new technology comes along.” You guys always do this, trying to slip this passive language through the door, as though LLM technology just spontaneously coalesced out of the ether, and then one day we all woke up to find that our students had suddenly and mysteriously become mind-benumbed AI addicts.
The introduction of this technology to the general public, and its imposition upon academia, was planned and carried out deliberately. The same people who have profited from it are the ones who have paid exorbitantly to convince people like you that their product is a positive or neutral force in the world. It is not. It is poisoning the planet. It is making our children stupid. We do not and have never needed it. And the fact that you can’t bring yourself to care about what has been lost does not make us histrionic for doing so.
Amen.
Well said. Thank you.
I’ll move on from literary analysis to philosophical analysis, same effect:
“We need to be able to adapt in life without throwing a navey gazey pity party every time a new technology comes along and catches our attention and shifts our routines/attitudes.”
Setting aside the slippage between token (recent advances in generative AI) and type (technology advancement), this is a thesis about how to live a good life. It is substantive and by no means a given. It is, straightforwardly, exactly a negation of the substantive view expressed in the original post.
“By no means a given” is extremely charitable. The thesis under examination is simply an endorsement of nihilism, the claim that we shouldn’t allow ourselves to get attached to anything because it might change, that nothing is intrinsically more valuable than anything else and only weepy losers think otherwise.
Nah, I’m not nihilistic! It’s more like this: these people having the pity party are like “I’m grieving that children no longer have a spark or glimmer when they learn something the old fashioned way” . I’m actually appealing to the intrinsic value of knowledge (knowledge is valuable, we should ceteris paribus pursue it), to say “yeah, but we can gain knowledge more expediently now, something to be rightly celebrated not wept over” . Compare: it might have seemed a sense of achievement to overcome obstacles (walking a mile) to get to school, and through cars kids now just get dropped off and miss the special glow of achievement of overcoming obsatacles.. no sense wallowing in poetic metaphors and grief because we’ve lost these kind of obstacles that certain kind of old-fashioned achievements were predictated on. Having instaknowledge via LLM in our pocket removes obstacles to acquiring knowledge the old fashioned way (slugging it out with pen and paper and libraries) but getting the facts fast frees us up to pursue more ambitious intellectual projects, again, something to be celebrated given the intrinsic value of knowledge.
Why don’t you take a look at academic performance trends and get back to me.
Yes, yes, correlation and causation, etc. But one way or another your claim that AI is/has been good for learning doesn’t match the statistics. AI-usage is not correlated with macro-level academic improvement, and thus a fortiori certainly hasn’t caused it. It is, however, correlated (along with a complex of other related developments) with a massive downturn in academic performance.
That seems worthy of grief.
This is besides the point, but:
I don’t think Buddhism is generally taken to constitute an endorsement of nihilism, yet it entails (as far as I understand) the claim that we shouldn’t allow ourselves to get attached to anything because it might change (follows from doctrine of impermanence) and that nothing is intrinsically more valuable than anything else (follows from doctrine of no-self).
(And in fact, given the doctrine of suffering, maybe it even entails the “only weepy losers think otherwise” bit!)
I’m not so sure. The idea that “nothing is intrinsically more valuable than anything else” is the textbook definition of axiological nihilism. If Buddhism matches this description, then, well, it’s an axiologically nihilistic worldview. But I’m not really qualified to adjudicate that. Surely Buddhahood is more valuable than samsara-life, right?
Fair enough; this seems like a modus ponens/modus tollens issue. I was taking it as given that Buddhism surely constitutes/provides some sort of positive account in the realm of the good life and, insofar as it matches that description, there is therefore a gap between that description and nihilism. You are running the argument in the other direction, which is also fine.
That being said, the other direction seems like a much more heavy swing to me (epistemologically speaking). That is, I’d want to understand Buddhism on its own terms a lot more than I do and be much more confident in my understanding of the scope of my understanding it, before that direction becomes the direction I’d be comfortable running.
Well, if you want to understand buddhism…. a good way to do it is to wipe the tears of grief (i still don’t get all this grief talk!) and look it up on an LLM! (joking not joking)
An LLM wouldn’t be able to provide you with any understanding of Buddhism at all, any more than it could give you an understanding of grief.
Of course, it could tell you all about grief rituals practised in centuries past, and it could tell you amazing details of many interesting metaphysical doctrines.
Imagine a world filled with systems that can tell you about grief rituals without feeling grief or ever encountering death. Imagine a world filled with systems that can ramble on about metaphysical doctrines without curiosity driving them, and without the longing to connect with consciousnesses external to their own subjective consciousness.
Now imagine these systems as the conduit for human culture (knowledge) completely replacing humans, and as substitutes for mentors, therapists, best buddies etc. That would indeed be a dead coral reef world. Not nirvana.
By the way credit to Bilingual who used the dead coral reef metaphor first but further down on the page. I will love the phrase forever and ever as perfection for what it is intended to describe.
I am sorry but the dead coral reef metaphor is grotesquely inapt! The kids are using new tools to learn rather than old tools. Things change as tech progresses. There is no comparable parallel with the utter tragedy, worth genuine grieving, of the destruction of the coral reefs. The so called Weepers and Griever crowd about AI in schools (of which you seem to be right in the thick of) needs to not ‘cheapen’ grieving by bandying around the word so indiscriminately.
Yes it would. This is the case even on epistemically robust/demanding conceptions of understanding. So let’s set aside Elgin’s view and non-factive views and suppose understanding involves at least (in the case of objectual understanding, which is what I assume were’ talking about here) a bunch of true beliefs about the subject matter and in addition to that the satisfaction of some kind of grasping requirement. One can get all that stuff from an LLM; if the LLM is not in fact hallucinating and gives accurate information, then the mere fact that it could easily have hallucinated doesn’t undermine objectual understanding – it woudl do so only if objectual understanding were incompatible with environmental epistemic luck. but it’s pretty well established (going back to Kvanvig’s Comanche case from his 2003 book) that objectual understanding is incompatible with at most Gettier-style intervening luck. The LLM can give you unerstanding of buddhism no less than can reading a correctly printed library book about buddhism that happens to be surrounded by fakes that were typed by monkeys and are such that if you read them you’d have false beliefs about buddhism.
It takes chatgpt 5.5 for you to learn, as a professor, that Istanbul is in both Europe and Asia? *And* (much more importantly) you take this to be an example of what is meant by “twinkle” in the relevant sense? This example strongly suggests that your comments are an example of lazy trolling that your interlocutors here have failed to detect as such. Or it could be a reductio of your point. One or the other.
The Istanbul line makes me think that this is a joke. Is it? … Please?
Yes but the point is about students, not about those who already have been educated, graduated, whatever. Sure, perhaps we are all students in some sense, but there are differences. Besides, there is a difference between contexts: you presumably do these things because you’re excited to learn about things (for example). Whereas the standard context for students is not necessarily this (although it can be and it would be best if it was): they want a good grade, want their degree, want to finish their course of study. This often introduces a very different motivational mindset from yours.
I think this is the core of the problem: the spark in the eye is gone because some students are motivated only to get a good grade and degree and miss an intrinsic yearning for knowledge. AI can help the former kind of students, as well as the latter, to achieve their goal.
Instead of fighting AI – and the next new tool, and the one afterwards – we’d do better asking how education became a means toward an end instead of a goal in itself.
I think this is too idealistic. Students were always, at least in part, in higher education to get a degree and then get a job. There is nothing wrong with this. Sure, this is perhaps now overemphasized but my point was only that in a realistic setting, we cannot expect students to educate themselves for education’s sake. This will never happen and I don’t think this is actually wrong.
genuinely curious: given the penchant for errors especially in free llms–and i don’t mean hallucinations but simple factual errors–how is it that you can confidently feel that you are leveling up? in my experience, info about something like classical music or the aenid (topics i have only basic knowledge) would be highly suspect, at least in the details, coming from an llm.
additionally, i am curious to understand what you consider to be the innovation of llm-based learning over simply reading about a topic from vetted sources. let’s take the stanford encyclopedia for example, although you could use google scholar or academic search premier (presuming you have uni access), or even dare i say it strong wikipedia articles. presumably if one follows the links the llm provides, this is what one is already doing. if one doesn’t follow the links, one is subject to the llm’s confidently stated errors and misreadings of the vetted sources, something that very much continues to happen.
Over the decades I’ve leveled up a lot of knowledge and understanding through the use of Wikipedia, though I’ve often later discovered that the perspectives it gives are skewed and missing important facts, and occasionally even wrong. Neither Wikipedia nor LLMs alone can get someone to actual mastery of a subject, but they really can provide a lot of knowledge and understanding, and usually more than is possible from a single undergraduate class on a topic.
Also, over the decades, I’ve developed an understanding of which types of questions I can answer better through a Google search and which I can answer better through a Wikipedia article and which through a Reddit search. There are now some types of questions that are better answered through an LLM (often the sort of thing that involves a slightly “diagonal” angle through topics, that might be addressed by someone reading a couple Reddit posts and open-access academic articles).
No kind of knowledge that a non-specialist can build will be as good as that of a specialist – and it’s difficult for the non-specialist to get that even if a specialist is present to walk them through the literature. But non-specialists can use an LLM now to get a kind of knowledge that they couldn’t get before (just as Reddit and Wikipedia and Google each introduced kinds of knowledge to non-specialists that weren’t possible before them).
And I should add, I absolutely do grieve for what we have been losing in education, particularly for the inability some students demonstrate for holding uncertainty as a tool for achieving deeper understanding, and the unwillingness to do hard work. Some of this has been disappearing for years, between the rise of short-format video and social media, and students taking advantage of things that are intended as accommodations for some to avoid work for others. (I despair at the number of times I’ve seen students skip class because the slides are available on Canvas, and they think that reading slides is nearly as good as being in class, thinking at the pace of speech.)
But the rise of LLMs has created a new kind of shortcut that is making some students skip any kind of thought. The fact that they enable so many amazing new things does not cancel out the deep destruction they have caused to a certain model of education. We will have to find new ways to get students to think – will we find some way that forces them to do it, or some new method of presentation that gets them to an authentic interest in it, or some weird muddle that mixes these things? There will be new opportunities for sure, but we will have lost something important, and I worry that we won’t replace that thing.
i don’t deny your central claim or that you *can* use llms to learn stuff. but i don’t see this post quite addressing either question i asked. it seems to me i get more demonstrably false answers from llms than i do with reasonably thoughtful google search. further, it seems to me that the risk of these wrong answers negatively impacts the efficiency of using llms. to be clear, I’m not talking about coding or logic etc
it’s worth adding perhaps that often when i express something similar to this, i am told that i am using the wrong llm or that i ought to make a better prompt. but in either case (of checking another llm or iterating prompts until i get what i need), i am adding labor to the process of getting the correct information where as if i say check SEP and follow a link to my library’s ebook from the references, i have a clear pathway to the information that has a level of reliability built in.
For philosophy, I absolutely wouldn’t use LLMs to get information, because I have the skill and understanding to find the SEP, read an article, and follow the references. But they give me access to a kind of information about fields that I don’t have anywhere near this kind of expertise in.
(I have on a couple occasions asked LLMs for “deep research” reports on topics in philosophy, mainly to provide a “second opinion” on the SEP article. It’s sometimes added a few useful references getting at things from a Continental angle, or another one that was missed in the particular SEP article, but usually hasn’t added much.)
I’ve started finding it extremely helpful for math/physics queries (this is quite recent: it’s got a lot better recently.) I was able to discuss a particular (not precisely formulated) mathematical question in a fairly obscure area relevant to the paper I was discussing; it was able to tell me that it was basically solved by Szabo’s theorem in 1980, to point out some ambiguities and loose ends, and to link me to a good recent source on those loose ends. Certainly I could have found this myself but I’d estimate it would have taken me most of the afternoon instead of 20 minutes.
That’s not to comment on the OP issues, just to push back a bit against the claim that it’s not useful for learning.
A risk I’ve noticed in this territory (“it was able to tell me that it was basically solved by…” is that of being misled in peculiarly new ways.
Specifically: as the technology has gotten better, I have found myself rapidly losing all speedy heuristics for checking whether the content is hallucinatory. In some cases, this has led me to beliefs about the actual state of the math physics literature that I didn’t notice myself forming, which only reveal themselves to be confusions weeks later. (In some cases, I can trace my thinking back and conclude that the source of the issue was in something specific the LLM said about the scope/entailments of an old math result that I catalogued in my head for a different reason at the time.)
A personal moral has been to try to practice “typing as LLM” the technical information as it comes in. But it is difficult to hold onto the type when, weeks later, it has since cemented as a part of my general sense of what kinds of results have been obtained in an area.
(I think this is continuous with peculiar effects of using, say, math stack exchange. But in that case, community discussion and the built in voting mechanics provide additional resources to maintain speedy heuristics to check what is equivalent to the hallucinations in the LLM case. The one-on-one and tailored to my own idiosyncratic way of prompting at any given moment is the real issue with the latter case, from this perspective…)
Certainly you need to check things; it can’t just be uncritically trusted..
My point is exactly that the gap between “critical” and “uncritical” gets much murkier, when the content in question is a bespoke-to-your-prompting-interest interpretation of old mathematics results…
I’ve also found it very helpful for me in exploring some far-out reaches of set theory. As well as some areas of physics I knew only a little about (e.g. Xiao-Gang Wen’s theory of particles). And it’s really helped me understand artificial neural networks. Sometimes I have to take it at its word because I just don’t know how to verify or sometimes even understand the answers. But usually I can follow well enough along to see that it’s talking sense.
On the other hand, I write on Lovecraft, and when it comes to that, the LLMs are crazy wrong. Ask it a question about a character in the Lovecraft Mythos, and what follows will be entirely false, as well as highly unstable (slightly different queries producing wildly divergent answers). So, on the abstract stuff, it seems really good. But on the detailed unpatterned stuff, like facts about particular texts, it’s horribly wrong.
But, given that it’s wrong sometimes, what exactly is the point of taking it on its word? Why not roll a die and take it on its word when the die comes up 1-5, and distrust it when the die comes up 6? Or why not just live with the fact that there are some things about which you must remain in ignorance for now? I don’t understand what the point is of querying a tool that is sometimes wrong if you aren’t in a position to check if it’s wrong. How does that help you?
> why not just live with the fact that there are some things about which you must remain in ignorance for now? I don’t understand what the point is of querying a tool that is sometimes wrong if you aren’t in a position to check if it’s wrong. How does that help you?
This depends a lot on the question you’re interested in. For some things, it’s much more valuable to have the correct answer than no answer, and not so much worse to have an incorrect answer. For other things, it’s not much more valuable to have the correct answer than no answer, and quite a bit worse to have an incorrect answer. There’s of course all kinds of other ratios of value that could exist too. But for things like the former, getting your know-it-all friend or an LLM to tell you something is fine, while for the latter kinds of things you definitely shouldn’t. You shouldn’t think that an LLM answer is better than nothing in *every* case, but you also shouldn’t think that an LLM answer is worse than nothing in every case.
Having more options for more kinds of information calibrated at different levels of detail, each with their own kind of reliability profile, can be quite valuable, as long as you have some sort of grip on that reliability profile. (Of course, if you don’t know how reliable a source is, because it’s a random book from a library where you know nothing about the topic or author, or because you haven’t played with LLMs very much, then perhaps it’s best not to rely on it.)
Whatever you might think about the robots, it’s just bizarre to think that their expertise or lack thereof is random.
I’ve found AI good in two very narrow use cases:
(1) Helping me get up to speed on a new topic. I can get a good basic tutorial fast. Then I can head off into the literature myself.
(2) As an enhanced search engine. In this use case, it’s not about getting right or wrong answers, it’s about getting relevant or irrelevant leads. The robots are pretty good at providing relevant leads.
I haven’t found it useful for much else in philosophical research. It’s not creative. It’s very conventional. If I ask it about topics which I already know about, I find that it just rehashes old stuff. Current AI is mainly an enhanced Wikipedia or google scholar. That’s fine for cases (1) and (2).
It’s not that the robot is frequently wrong. It’s got the expertise of a good undergrad or MA student. It’s answers are decent, but don’t go much beyond the advanced undergrad or early grad level. The robots simply can’t do philosophical research yet. But they are good at the two tasks I mentioned.
I didn’t mean to imply I thought LLMs have random expertise or lack thereof. My point is that with respect to situations where you have to take the LLM at its word, why take it at its word 100% of the time rather than some percentage of the time closer to what you take to be its accuracy level with respect to that information?
Thanks, got it. The cases where I have to take it at its word are when I have really far out, wide open blue sky conversations with it about some obscure topic like set theory.
For example, I’ve had conversations with GPT about whether V=HOD. Nobody knows the answer to that question. So I wasn’t looking for a correct answer. I was just exploring, out of sheer curiosity. And how does HOD differ from L? After talking with the robot, I realized I had a lot more insight into the power set axiom and set theoretic existence. In these kinds of cases, I’m not turning to the robot for answers. And I do know enough of the math to recognize that the robot is plausible.
In another recent case, I asked GPT about why 5HT2a agonism causes humans to have religious experiences. The robot gave answers that were correct, as in based on some recent literature that I know, but that were also very shallow and weak. So I raised objections and told the robot to do better. It couldn’t do much better, but during that dialog I started to understand why Carhart-Harris’s entropic brain hypothesis has the explanatory power that it has.
I don’t think it’s very productive to treat the robots as providing answers. Likewise, it’s not productive to complain that they hallucinate. Apart from the really mundane literature searches or summaries, I find it more useful to enter into adversarial dialogs with the robots. That way I learn about the strengths and weaknesses of the competing proposals that are out in the literature.
I’m almost never asking some settled question that the robot could get right or wrong. I never treat the robots as prompt-answer machines (even my literature searches are highly dialectical). I’m engaging the robot in dialectical conversation.
Anti reductionists in the epistemology of testimony have well established answers to all this – anti reductionism in the dominant view in the epist of testimony for nothing
Is LLM output testimony in the sense non-reductionists have in mind?
That’s one other useful case (and the only one I turn to ChatGPT rather than Claude for). A few months ago, I was trying to recall the proof of the Caratheodory extension theorem to see how useful it would be to explain some amount of its details in something I was writing. ChatGPT was able to helpfully walk me through it, realize that the path I had been on at first wasn’t going to work, and eventually help me understand some of the subtleties of the sigma-algebras involved. The Wikipedia proof was not helpful at all, and even a more detailed proof in a textbook wouldn’t have been that helpful (for the reason Plato said books aren’t helpful – you can’t ask it follow-up questions to understand why it said what it said). It probably would have been better to have a measure theorist friend on hand (and they would surely have been much less annoying at constantly trying to offer clickbait at the end of every conversational exchange), but I don’t have one of those handy.
This is just about different kinds of questions. It’s not that often that I turn to an LLM for information (I’m much more often turning it to write some code for me). But here are the information questions I’ve asked Claude in the past few weeks:
A couple days ago, after getting a new phone such that all devices in the house were USB-C, I asked Claude about what the differences in kinds of USB-C cables and chargers are, and listed all the devices in the household, and asked for advice on the features cables would need if they were to work well for all devices. A few years ago, I would have turned to Google, but I would have had to look up each device, see the specifications for their chargers, then Google information about which of those specifications matter, rather than just having one interaction that gets all that information digested at once.
Two weeks ago I asked for advice about why Keynote was failing to save videos I was recording. (Unfortunately, for this, Claude was no more helpful than Google. I was eventually able to discover it was because my hard drive didn’t have enough space for the video, and Keynote wasn’t telling me.)
On April 4, I asked about the comparative violence of rugby and Australian rules football, and got some description of the fact that rugby “generally involves more deliberate, sustained physical contact. Scrums, mauls, rucks, and set pieces are built around organized collisions between large packs of players.” This gibes with what I have seen, but puts it more explicitly than I would be able to find with a standard Google search, and it goes on to say more about the different kinds of injury that are common in the two sports.
I still find myself opening Google, and Wikipedia, and Etymonline, for informational queries, more often than LLMs, but there’s a certain kind of question where the LLM is much more useful at getting to the aspect of the information I want.
For me part of the usefulness of LLMs is that it replaces the rapidly declining functionality of Google.
part of the usefulness of LLMs is that it replaces the rapidly declining functionality of Google.
Sadly, those two things are not unrelated.
On Kenny’s note, my wife recently got a (to me) fancy Garmin GPS watch second-hand, and something wasn’t working right on it. AI claimed to provide specs for it, and advice, but it turned out to be all wrong, and a waste of time. That, of course, doesn’t show it will always or even usually do this. But I think it does show that it’s less reliable than looking at than, say, in Kenny’s example, looking at the specs directly and figuring things out, and that may even be faster (and less frustrating) if you are given wrong info.
I’m talking about frontier LLMs; I’m leveling up like crazy on 5.5 gpt and opus 4.7 – the attitude that these things are full of hallucinations and unreliable is getting outdated quickly; these frontier models are about like wikipedia in terms of reliability (some errors but good enough!) – basically like having a permaent MOOC source (check out Notebook LM’s instructional videos) for highly specific things you want to know, and you get them immediately. My preference is to ask Notebook LM to make videos of papers I want to reaad and watch them, and then talk to Gemini about it asking more questions, and using study cards it makes me, quizzes, etc. all free and immediate and all on your phone.
how is it that we get access to frontier models “for free” per your initial comment?
you assert a low error rate (note, i specifically excluded hallucinations in my prior question. not because they don’t matter or don’t happen) but i’m curious how one would know the error rate level if one is getting new information? wouldn’t that mean that one had verified that information by consulting the provided sources? which in turn means that one would *not* be increasing learning efficiency but simply using the llm as a search engine? on the other hand, if one doesn’t do this, how does one establish that the llm output is relatively error free? it seems like we’re missing a piece.
measurements of things like error and hallucination rates seem to vary dramatically based on study and benchmark designs. we can get any answer we wish (kind of like the llms). and frontier models haven’t been around long enough to have robust studies of real world error issues.
i don’t think anyone doubts that those who already know material can get llms to produce strong accounts there days. but that’s not what we’re talking about here with students or with lifelong learners.
This talk of ‘levelling up’ is very Dragonball Z.
In most games where you level up, there’s a built-in metric: you know what the maximum achievable level is and you can check your exp to see your progress to the next level.
None of those metrics are available here.
I seem to be late to this, but let me state a bold thesis: You did not learn any of the subjects you listed nearly as well as you think you did. Possibly, you did not learn any of them well enough to pass the kind of exam that would be set by someone who did know these things.
Why do I say this? Because so far what we’ve seen is that students using LLMs consistently overrate their knowledge and do poorer on exams than those who do not. And why not? It’s an idiot-technology whose “knowledge” such as it is is all second-hand. It has neither reason nor experience. It has all or most of the qualities of Kierkegaard’s “crowd” or the most inveterate prisoner in Plato’s cave, and it has less experience with striving than Nietzsche’s last men. It would require a miracle for it to actually help in learning. You can test for this miracle and use yourself for a subject if you like.
But evidence so far suggests that what it does is take with one hand what it gives with the other and that those who depend on it do not grow.
Hence, my bold thesis: You have not learned these subjects to any substantive degree. Students who use it are not learning from it.
I would venture that someone who never talks to another human and only consults LLMs is likely to end up with no worse an education on a subject than someone who never talks to another human and only reads books. Books and LLMs have the advantage that they’re easily available, while humans who actually know a subject are scarce. One shouldn’t fool oneself into thinking that one has a good education on a subject from only consulting books, or only consulting Wikipedia, or only consulting LLMs. But each of them did mark an increase in the availability of second-class education to people who don’t have access to knowledgeable humans.
I would venture that someone who never talks to another human and only consults LLMs is likely to end up with no worse an education on a subject than someone who never talks to another human and only reads books.
One problem with this is that consulting books _is_ consulting another human, even if not in the flesh. I can say that, on a couple of topics I’ve looked at fairly recently (“what are the whitewater rivers like in Northern New South Wales?”, or “what are some good hiking tracks in Southern Queensland?”, to take two recent examples), books are _vastly_ better than LLMs – and it’s not even close. Maybe LLMs are closer to books in some other areas? Maybe! But I’d not bet on it, given my experience.
That’s a good point that, for simple factual matters where one already has all the understanding of the topic that is needed, books can be a sufficient resource. But the ability for someone to ask questions is (as Plato noted!) one of the important factors in developing understanding.
If someone already understand whitewater rafting and just wants to know the rivers in Northern New South Wales, that book you found is probably a great resource. But for someone who doesn’t understand, having a conversation with a person (rather than frozen information in a book) or a conversation with an LLM, is going to be more productive.
Peters has made the case beautifully. Devastatingly, even. When I think on what the billionaires have stolen from us with their environment-destroying, water-guzzling toys, it’s all too easy to plunge into a state of grieving.
Something I would venture to add: not only are the joys and generative agonies of learning beautiful to witness, they are also good for the brain. Being forced to think strengthens the mind in the same way exercise strengthens the body. To the AI sycophants: Here are the fruits of your beloved LLMs: flabby, stultified minds unwilling to stretch themselves around a difficult idea that requires more effort than formulating a 10-word prompt. I see this every day.
Going to work now gives me the same feeling I get when I look at a bleached coral reef. Incidentally the neoliberals are responsible for both.
Of course, at least to an extent and at least passively, we’ve empowered these people to do what they do, to cause the damage they are causing.
Who’s “we”? By the time I reached adulthood the groundwork for all this had already been laid. By the time I had attained even the tiniest modicum of clout in the discipline, the onslaught was in full swing. If by “we” you mean donation-hungry members of the academic establishment, then of course you’re correct, but I’d prefer not to be lumped in with them.
Well, change to ‘have empowered’ to ‘are empowering’. Aren’t we, all of us? Or are we, all of us, most of us, many of us, just victims? Sure, ‘we’ might overgeneralize but that is just a quick proxy, a placeholder. I cannot judge your situation, but surely, it is not merely just a generational matter, all mistakes and guilt in the past, nothing in the present?
Yes, I suppose we could all be doing more. To the extent that we’re not doing our utmost, we are all complicit. But I assure you, I’m trying my best.
I would say that by passively accepting the computer’s hegemony over postsecondary education we’ve been at least complicit. We all just normalized the use of the very devices that are hosting the demolition of education. None of us in philosophy had to do this.
All for minor gains in convenience. Premised on a bet, even at the time a pretty shaky one, that nothing like AI would ever be invented.
We should have reacted to this danger earlier; I recall those who did by sticking to traditional methods, and in my PhD department we smirked and giggled at them. Who’s smirking now, I wonder…
Yes, my considered opinion is that allowing high-capacity personal computers to be used by the rank-and-file public has been utterly catastrophic in its long-term consequences. AI is just one example. The devices have also facilitated political polarization, new avenues of sexual exploitation, hyper-materialism… the list goes on.
We should have regulated the technology immediately, as we do guns. Left unchecked, it has proven to be at least as dangerous and harmful.
But again, I was still in elementary school when this particular point of intervention would have been possible, so I’d say the onus for that one does indeed rest on the shoulders of the generations immediately preceding my own.
I’m pretty good at not letting awful stuff get to me emotionally, so I wouldn’t characterize myself as having engaged in much grieving, but otherwise I very much share the sentiment. The mind boggles at the amount of damage LLMs have done to people’s intellects in such a short time.
Perhaps if the LLMs had wiped out the brains of everyone over 40, it would be cause for great concern but relatively remediable, since the young could take their place. But the opposite has happened: the LLMs have started eating us at the bottom. Sometimes I think it will be a miracle if civilizations don’t collapse within a generation or two.
One ray of hope is that I’ve noticed a lot of young people turn against the LLMs. But at least in my personal experience this has happened mostly among the most excellent students. As someone with strongly egalitarian tendencies, I am not thrilled to see that the way this shakes out may result in a substantial widening of existing divides in education.
Dear Jane,
I know exactly what you mean. I had my moment eight months ago. But, in a spirit of antifragility, I started this:
https://certifiedaifreeskillsandknowledge.org/
Now, I feel awesome — and so do my students.
Join us!
Best,
Marc
My confidence in the face of name-calling, btw, stems from this not-yet-finished book: https://philpapers.org/archive/CHAEES-2.pdf
If you’re grieving, then it’s dead; if it’s dead, then it ain’t comin’ back. So what’s next? Do we just give up? That’s it? End of philosophy?
No! We do what we’ve always done: we reinvent philosophy.
I agree Eric — nearly anything is better than drowning in the tears of melancholy over some kids using chatbots to find their facts nowadays – a sadder state of affairs is bunch of grown adults here at Daily Nous using 50 dollar words to throw a pity party! If this teary eyed crowd can’t handle chatgpt , wait until they see what’s on the horizon next, which is non-therapeutic uses of Brain Computer Interfaces that will allow students to access information via thought commands.
What exactly would be the issue with brain computer interfaces? In all of your comments I haven’t really seen any acknowledgement of, let alone attempt to understand, and certainly no success at understanding, what people are actually upset about when it comes to what LLMs are doing to young people. Your comment here reinforces this. Our issue is not that they are using technology or whatever. I couldn’t care less if they can Google with their brain instead of their fingers. If they would start using brain computer interfaces to look up information in reliable sources, and if they would assimilate that information themselves rather than having the LLM predigest it for them, my unhappiness would vanish overnight.
What then is our problem? I think you’re deep enough in the LLM hole that you’re not going to get a real sense of it just by reading accounts online; certainly none of us have decided this is a problem based on secondhand accounts. This might be something one needs to experience oneself. But, to give you a sense of the issue, many young people are much less willing and able to think about complicated things, to form conclusions on the basis of having thought about complicated things, to investigate those conclusions by probing their strength and plausibility, to revise those conclusions on the basis of the evidence they have gathered and evaluated independently or with the help of trustworthy fellow evaluators, and so on. This incapacity seems to us to be very large, and hence very consequential, and it seems to have developed very quickly, in large part due to the role LLMs have started to play in people’s lives.
So what do we do?
At my university I am working to set up computer labs where students can work on assignments without access to AI. This will allow professors to ask for papers and so on with some certainty that they were not written by AI. This can be combined with many other techniques I and others in my department are implementing in order to attempt to get students to avoid LLM use. (E.g. I use oral exams in my courses now.)
Danny, when there are some details in place, could you please post them? Maybe Justin will give you a guest post for it.
It sound promising — I’ve been vaguely thinking about something like that, but I couldn’t figure out the details. (I don’t mean the technical details, since I’d get someone else to solve those — more like, what specifications I’d give the techies.)
Well, I can’t say anything about how well things are working out, since I gave the techies (just one techie, actually) the specifications and now I am waiting for the preliminary results. I gave the specifications to the techie verbally based on this document, which I wrote as a proposal for the university generally, not for the techie specifically. The parts I focused on for the techie were mostly items 4 through 7 in the “Ideal Testing Center Features” section.
Cool. Students working under those harder conditions deserve to publicly brag.
That sounds great! I love your idealized specifications for a lab, as well as the actual specifications. I’ve convinced someone from the School of Social Sciences here at UC Irvine to work with IT to get something like this set up here – it sounds like they’re doing a trial run of a couple assignments in one or two classes later this quarter, but the main pilot will be in the fall.
I really think that something like these labs needs to be an important part of universities going forward.
Well put.
I think you’re missing an important point here. The point of education is not just to get lots of knowledge, but to develop certain skills. Among those skills are the kind of ability to organize information in a way that makes it legible to readers – and also to the writer! A lot of traditional academic exercises involve writing, precisely to develop these kinds of skills. The thing we are grieving is that many students these days have rationalized to themselves that there is no value to this skill and have turned these exercises over to the robots (whether it’s out of sour grapes because they have the feeling that the skill is too hard for them to learn, or a Dunning-Kruger effect where they can’t understand the value of a skill they don’t have, or some sort of half-formed idea that if the robot can write the essay for me then the work has been done so I must already have the skill).
So what do we do?
We have to try a lot of things. One thing I am hopeful for is having some proctored AI-free computer labs where students can be assigned to do their writing assignments. I don’t think this addresses all the issues, but it might help cover a different set of cases than bluebooks and oral exams.
But we certainly need more ideas too.
I’m sorry to be all SSP as we used to say, but the start of a solution is to double down on commitment to that experience and clarify the centrality of that experience to teaching. how have books survived? teaching can, too. but it may not be in the institutions we know, and that in itself may not be a bad thing. https://jeffreyfisher.substack.com/p/transformation-over-information-transfer?r=2g7xpl
Books have survived because on my own I am probably responsible for 10% of all purchases… 0_o
I have to admit I have no idea what people are talking about when I read these discussions. I remember teaching before LLMs. Plenty of students back then had no patience for frustration, hated ambiguity, only cared about the grade, and tried to get away with cheating. Plenty of students now remain curious and care about ideas more than the grade. (I sometimes have to tell them to take on more manageable projects so that their grade reflects their understanding and effort.) Maybe the ratios have gotten a little worse, but the apocalyptic language doesn’t match with my experience at all.
There’s a new technology that makes cheating much easier. Previously the technology (Turnitin) made cheating much harder. We need to find methods of assessment that are harder to cheat at. Let’s just face the problem and solve it.
Yes. Not to sound cynical or blasé, but when I read this sort of stuff my main reaction is to wonder how can someone be so out of touch with the average student’s motivations, especially in the gen-ed service classes most of us teach. 90% of them never cared in the first place. Those who did will keep caring and will learn how to use AI for the better (just as we do).
I don’t think many (any?) people are claiming that students used to care more. I think the worry is that it used to be the case that you could get students who did not care to learn by having them do things like write papers. Now this is much harder. Similarly, Baker says that students always had no patience for frustration, hated ambiguity, etc. But they had to find a way to cultivate patience, tolerate ambiguity, etc. in order to pass the course. Now they can pass the course without doing these things.
Your immense patience and clarity of explanation is no match for the AI-apologist’s lazy appeal to incredulity, Daniel.
Sorry, I’m missing what he thinks I’m wrong about.
You think this a question of whether students used to be more authentically interested in school. We’re saying it’s a question of whether or not we have the tools to make them care, which is good for them.
That’s as simple as I can make it.
Well that’s what the original post was about. If the issue is just finding better tools to make it hard to cheat, let’s do that.
I don’t think anyone is claiming we ought not to do that. I think the claim is rather that the person who wrote the original article is sad about what we have lost. Maybe we can claw back much of what has been lost (but at what cost?). But that does not preclude being upset, or otherwise documenting, what has been lost, how fast it has been lost, whose fault it is that it has been lost, who has paid most dearly by losing it, who is being tasked with clawing back some of what has been lost, and so on.
Your sentiment seems to be something like: “Shut up! I don’t care! If you have a problem, fix it and stay quiet.” If that is your sentiment, I think you would be best served by staying out of these conversations. They are not for you, and others who want peace and quiet. They are for people who would like to talk and think about these things with others.
The original post was not about in being too easy to cheat. I can read. It was about all wonder and human interaction and meaning being sucked out of education. It was about something being permanently lost. If the problem is just students are cheating more, that’s a tractable and by no means permanent problem.
But, I think the complaint isn’t really that it’s easier to cheat. Because I can read the original post and that’s not what it’s saying.
I think you are not reading the original post and my gloss of some of the concerns expressed in the original post with enough charity to understand what is being said.
For example, you are attributing to the original post a rather strong claim about permanency. I do not find any such claim. Indeed to the extent I find any discussion of the topic in the original post, I find a rejection of the claim, namely in the last paragraph of the original post. Similar issues could be raised about your reading of my posts.
I don’t know any remedy for this other than to urge you to employ greater charity in your evaluation of the points your interlocutors are making.
You attributed a statement to me, “shut up!” that I never made, and now you’re complaining that I’m being uncharitable in reading others, because I think when they write about “grief” and “love” they mean grief and love. Okay. Cool.
From the original post:
That is not a complaint that Turnitin doesn’t catch cheaters anymore. It is not a complaint that we need to use in-class essay exams. Saying that it is, is pretty obvious motte-and-bailey.
If the problem is not permanent, why are we grieving? Why are we calling it *death*?
I am the one interpreting people charitably. It would be extremely uncharitable for me to think that the quoted sentence is about it being harder to catch cheaters. To interpret the sentence that way, I would have to assume the author is not a competent user of the language. It is much more charitably read as a claim that students don’t care as much about learning anymore. (See the word “love”?) Part of charitable interpretation is assuming people choose the words they do because they understand what those words communicate, that they are capable of basic social rationality.
To put this another way, I think you are not making any effort to charitably understand where I and others are coming from. It is not that we don’t like to talk and think about these problems. It is that, if the problem is fixable, grief is not a fitting response. So it is natural for us to think that either (i) you want to express unfitting grief without anyone challenging you on it, which does not sound like you want to think about these issues, and if that is the case, it is kind of annoying that people are using language implying that they have a deep moral and spiritual authenticity that their critics lack (and that language is being used, all over the place); (ii) you are actually upset about something else; or (iii) grief, death, love and so on are very bad metaphors to be using in this context, and are impeding rather than aiding thought.
I am afraid the number of ways in which I depart from your framing of the issue is growing too numerous to address completely, especially because each time I address some point, I make additional points which themselves are open to further misconstrual on your part. I will limit myself just to three representative points:
1) Your initial comment began like this: “I have to admit I have no idea what people are talking about when I read these discussions.” I took your point to thus apply broadly to more than just the relatively narrow sentiment expressed in the original post, but also to, say, what I have said in this thread. Given that my initial comment earlier in this thread explicitly abandons the grief language, your harping on grief here seems rather beside the point.
2) You claim that grief is not a fitting response for a problem that is fixable. Perhaps you mean a problem that is fixable by the griever. Even that to me seems contentious (and indeed probably false), but no matter. Surely grief is a fitting response to a problem fixable by someone else who is not inclined to fix it. I can grieve that my house has been stolen from me by the king, even if the king could return the house at any point.
3) I think you have a very fixed view of student attitudes and capacities. The thought is that students have always been more or less the same, and will remain more or less the same, and our job is to react to this. I and others I think have a much more fluid understanding of student attitudes and capacities. We think these can change over time in response to all sorts of things. Thus what we are upset about is mostly about how students have changed lately, and how much work it is going to be to change them back, and so on. Your attitude seems to be that nobody has changed, fundamentally. All that has changed is how people’s innate tendencies are happening to play out given the new circumstances. Again, this is not how I am conceiving of students, so I think we are drawing very differing conclusions on the basis of these different approaches.
It’s really unclear to me how I am misconstruing you. Maybe I should have been more precise about what you said about grief. Yes, in your earlier post you said you don’t specifically feel grief, but you also wrote that you otherwise agree with the sentiment.
Now the OP explicitly states that:
So, at risk of misconstruing you, I took you to be saying that you agreed with the claim that love as the grounds of education is dying. I took you to think that identifying this as a form of death is appropriate. If that’s not what you are saying, maybe you would like to clarify.
Given that, this is pretty confusing:
I’m sorry if I’m misconstruing you, but I think that’s in part because of how you are characterizing your own position and also the claims of others.
In any case, I’m kind of amazed that after lecturing me on being uncharitable in my reading of others, you interpret me to be saying this:
This is absurdly uncharitable and not based on anything I have written.
My three claims have been:
(i) There have always been lots of students–probably a majority of students–who don’t intrinsically care about learning. This isn’t a claim that students have fixed capacities or attitudes. Obviously some students change. I have seen this. But most don’t, or at least haven’t so far.
(ii) AI is having a bad effect with regard to those students, because it makes it easier for them to get away with cheating. I agreed later that this might reduce the number who come to love learning.
(iii) (And to be fair, this point was implied, but I thought the implication was fairly straightforward) Using apocalyptic language–like saying that you worry that civilization might be destroyed–to describe a relatively small problem does not help anyone solve it, and probably makes it harder to solve. It also probably makes people feel worse about it.
This was in response to Stan, not to you. I don’t think that you claim (like Stan) that in the past students cared more. (Indeed, as you object to in your latest comment, I think you have a relatively fixed view about students, a view which rules out substantial changes over time in the composition of features of the student body. Maybe I’m wrong about this, but whatever.)
Rather, I think that you view the increased number of students who fail to come to care as not a big deal, not worthy of certain descriptions, etc. since, after all, it has always been the cast that most never came to care, even before LLMs. My disagreement with you is thus unlike my disagreement with Stan, and my comments in the post you are quoting only refer to your own view in the sentence that begins “Similarly, Baker says that…”
As for whether using apocalyptic language to describe a relatively small problem doesn’t help anyone solve it, and probably makes it harder to solve, and probably makes people feel worse about it:
1) I don’t think the problem is relatively small.
2) I think mischaracterizing the problem as small doesn’t help solve it, because one obstacle to solving it is that many people do not see it as a problem requiring urgent solving.
3) Anyone reading Daily Nous comment threads has long ago maxed out their capacity to feel worse from reading Internet comments, or they’ll hit that point pretty soon no matter what I post. So, no harm done.
Well when you figure out what your position is please let me know.
I’ll just say that I wrote that students getting away with cheating was a *relatively* small problem. And it is, when the comparison class is *civilizational collapse* or *the death of the love of learning*. Relative to lots of other problems I deal with, it’s big. But the problems I deal with are generally not world-historical in importance.
But you’re right, I don’t think the problem is worthy of certain descriptions. That’s because those descriptions are false. They are completely outlandish.
For me the issue is actually not with the students who really didn’t care at all to learn. Whatever. I was never getting to them, really, in any meaningful way (I mean, maybe you were– you might be better than me!—but I wasn’t.) Its with the ones who WERE interested. Given the pressure they face to cheat–any my inability to tell the difference– I just can’t justify doing the exercises I used to do where they got the change to have that experience. And they used to be able to give it a shot and then see how they were learning, really get there. Now those exercises had to be killed altogether in favor of whatI take to be fair less valuable memorization quizzes and handwritten exams. And that feels like a real loss that I grieve (in addition to the weirdly more oppositional stance it leave me taking with students.) Maybe that wouldn’t be true if I could do nifty oral exams with them or had other resources. But I have far far too many students to manage anything like that. I could do with old style papers and outlines. I can’t d it anymore.
I totally sympathize with this. And yeah, getting to the less interested students is tough. I got better at it after teaching at the same university for over a decade, but it took me a while. And then I changed universities, and the methods I had developed for getting students to read and participate in class worked really badly.
Regarding assessments. I don’t know your age, but my impression is that a lot of younger instructors are horrified by blue-book exams because they never (or almost never) took them, and so they don’t know how they can work. They were still pretty common when I was an undergraduate, especially in lower-division courses, and in my experience you learned a lot preparing for them. They don’t have to just test memorization, and generally didn’t. I don’t think they are as good as take-home essays, but they can be serious assignments.
In terms of take-home essays, I think we should think about continuing to use these as a trade off. Some students will take the assignment seriously and get a lot out of it. Some students will simply cheat. Some students will fall in the middle–they will do some reading and be interested and have ideas, but they will feed half-baked ideas into Chat or Claude and let the AI turn that into a structured argument.
It’s not clear to me that the middle group isn’t getting anything out of this. It’s likely that a lot of our students end up working jobs where being able to use AI to turn half-baked ideas into something more serious will be the kind of skill they want. (Most of our students go on to do work that is much less intellectually interesting than what we teach them, and that was true before AI; so, half-baked ideas that they worked out with Claude might be sufficient for the job they’re doing.)
So I don’t think these students are getting nothing out of the assignment. The problem is that they are doing a different, much easier assignment than the students who really take the assignment seriously. Those latter students effectively get penalized for trying to learn more. To some extent that happens already. Very good students often take on overly ambitious projects and hurt their grades that way. But this obviously makes it much worse.
So it’s a question of giving students more learning opportunities, at some cost to fairness in the grading. Whether that’s a tradeoff that’s worth making in any particular case requires the instructor’s judgment call. But I don’t think we have to completely give up on them, just understand what the costs of continuing to use them are and make reasonable decisions about whether it’s still worth it.
Yes, I think that’s a problem, and we should try to think of how to design assessments to make sure they can’t do that. But that’s quite different from the tone or content of the original complaint.
I agree – the “Grievers and the Weepers” are acting as though there was a mythic past where students all loved learning for its own sake. 90% of students have been disengaged and intellectually sluggish for ages, it’s just the 10% that should be compared – and there are still nowadays close enough to 10% of kids in a class who are sharp as tacks and come ready to work, AI or no AI.
I think this is all too black and white. There are a bunch of for-profit companies which control this technology without little restriction on their conduct. They pocket the money and we are left to fend for ourselves and face the consequences. This is deeply problematic in and of itself. Perhaps there is a solution to the AI challenge (it is not obvious to me that there is or at least that the solution won’t be sub-optimal), but even if there is, a technology like this should not be allowed to run roughshod on education in this way. I find this unacceptable. I find it very much acceptable that people grieve what they have lost even if ultimately they have to move on to find a solution (assuming there is any).
Here you are, making the same trite, baseless points you’ve been making this entire time, despite your refusal to engage substantively with any of our positions.
Have you actually looked at the statistics? Good grief, you can even ask one of your beloved LLMs to explain them to you. Students are coming out of high school dumber than they were before all this started. That is a statistically confirmable fact.
I’m starting to agree with Grieving Adjunct, below, that you are either trolling or performing an auto-reductio argument.
I hear 90% of statistics are made up on the spot. the other 10% are made up by LLMs.
Well I’ll be danged! First you mean mouthed Derek Baker and now you’ve mean mouthed me! Let’s take some very deep breaths: I know you are throwing in with the Grievers and Weepers crowd, which is your right. I’ve simply suggested that maybe this grief-framing is OTT and overlooks all the exciting and intellectually responsible (and frontier boundary pushing!) ways we can use LLMs in education. Regarding statistics: there is a framing assumption here that rejects extended cognition; should students be tested with or without calculators, for example? (An old question and one that is increasingly answered “with”, even by non-extended minders). If something similar holds for LLMs use, then studies that show how students perform without LLMs don’t show that they are dumber any more than you are bad at picking up children from school if someone steals your car. I reiterate: chin up, to the Grievers ane Weepers, as there is a lot to celebrate here and the calls for tears are (as I mentioned before) not only not fitting but also based upon a mythic past of wonder-eyed eagerness in the classroom.
If you’re being sincere about having no idea what’s happening, then you’re either exceedingly lucky or exceedingly oblivious.
I implore you, look at the macro-level trends in academic performance across the United States. You will quickly learn that your disbelief in a recent, substantial decline in the ability of young people in this country is completely at odds with the data. By every metric, math and reading proficiency are both tanking. This is a well-documented downward trend that began prior to, but was exacerbated by, the COVID pandemic. In fact, there is obviously a complicated array of factors at work here, but AI is fairly clearly one of them (whether that has more to do with essential features of the programs themselves or the ways people are using them).
Whatever the true cause, I’m dismayed by all of this, and I confront its effects on a quotidian basis.
I honestly don’t get why you are responding to everyone who disagrees with you with dripping contempt. They disagree with you. Maybe they’re wrong, but that’s it.
In any case, as you note, the downward trend in test scores began before Covid, let alone LLMs. So AI is probably not the main problem.
But even if it were, let’s use different methods of assessment they can’t cheat on.
I understand that my tone is vehement, but I would have thought the reasons for that were obvious. This is an incredibly important issue, and one whose negative impacts are being felt daily. Minimizing it, or implying that those who care about it are just overly emotional, is counterproductive, and indeed insulting.
I am distressed by all factors at play in this trend of decline. I hope they can be reversed. AI might not be the sole factor, but it is certainly contributory. And the reason that it is such a particularly galling factor is that we know exactly who is pushing it on us and why they are doing it.
Anyway, since we’ve now descended into tone-policing, I think this thread has run its course.
I’m not tone policing, I’m asking you to stop being mindlessly insulting to people who disagree with you, especially since you seem to expect us to regard you as some sort of reservoir of moral wisdom.
“Mindlessly” insulting? Criticizing my rhetorical choices is one thing, but I’ve been clear about my reasoning and have presented arguments. I admit, I was dismissive towards Expat Adjunct here because they have refused to engage constructively below (as others have also noticed and pointed out). Perhaps the “beloved LLMs” comment was a bit snarky. But don’t act as though I’m just throwing expletives around. A quick review of the conversation will confirm that this is not the case.
As for the accusation that I “expect to be taken as some sort of reservoir of moral wisdom,” I have neither said nor implied as much. Of course I think I am in the right here, as anyone with strong opinions would. But you are projecting qualities onto me which are incorrect.
At any rate, you are no longer responding to my points, presumably because you have no more of your own to offer.
Bilingual , I think Derek Baker makes a reasonable point here: there are some complexities about AI we’re all trying to grapple with, and we’re looking for solutions (how to work with AI, which ain’t going away, to make learning better), and you’ve been rabidly defending the Grievers and Weepers crowd with a bunch of ad hominems. Let’s take a breath and just focus on a shared goal: we all want students to gain knowledge and abilities; so, how can working with AI best help us to achieve this without underminding it? Moving in that solutions direction is more productive than the whole Weeper and Griever routine, and especially more productive than ad hominems and the blame game.
Can you point me to the ad hominems that I’ve made? Something more than a snide tone?
While you’re doing so, here’s something you posted not too long ago:
“A sadder state of affairs is bunch of grown adults here at Daily Nous using 50 dollar words to throw a pity party! If this teary eyed crowd can’t handle chatgpt, wait until they see what’s on the horizon next!”
But you get a pass on ad hominems, I guess?
You’ve gone from singing the praises of cutting-edge LLMs and actively ridiculing those who disagree with you to advocating for sober cooperation. So forgive me if the lesson you’re trying to impart here seems a little disingenuous.
By all means, let’s look for solutions. How is that incompatible with feeling sad about the need to do so?
First, you’re obviously insulting people quite mindlessly. You accused me of being exceedingly oblivious for not agreeing to your interpretation of a trend, in a post in which you went on to admit that the trendline doesn’t match your own interpretation that well. I mean, come on.
I think it’s fine to be sad that Turnitin doesn’t catch plagiarists anymore, and so we have to change our assignments. The language being used in much of this discussion is the sort appropriate to a great life tragedy, however. You of course are free to write however you like. But when you write things like, “we know exactly who is pushing it on us and why they are doing it,” but then it turns out that what “they” did is invent software that, among other uses, makes it easier to cheat on schoolwork, that’s really confusing. It sounds like there’s a much bigger complaint going on. This is the language of a someone suffering a deliberately engineered attack. If that’s not what’s going on, if the complaint is, ugh, we have to change the kinds of assessments we use for determining student grades, well, the mismatch between the rhetoric and the content is misleading, and people are going to misunderstand you.
It is at the very least not obvious that
no one in the tech bro sphere (where that term designates angel investors rather than inventors of architectures or discoverers of algorithms etc.) is pushing anything down the throats of the education system.
https://newrepublic.com/article/208667/silicon-valley-disrupt-community-college
Reading the article, it sounds like the problem in this case has very little to do with AI (which doesn’t come up until very late in the article), and certainly doesn’t indicate a plot by the tech bros to make cheating easier or kill love of learning.
Even if some tech bros do have such a plot (and some might–these are extremely weird people), I think it’s not likely to be successful.
I didn’t realize the claim at issue was whether there was a plot to cause cheating and kill love of learning. (I certainly don’t think that’s what earlier commentators on this thread were saying.) I thought the claim at issue was whether there has been a deliberate effort to integrate AI into all aspects of teaching and learning along with every other area of human intercourse, because of the insane amount of profit and power resulting. Same thing that causes corruption more generally I suppose.
Maybe I need to reread it, but I think it’s pretty irrelevant to the content of the linked article that the word “AI” appears half way down. (I wouldn’t myself have structured the article in the way the author did – who writes it in a way such as to make it about a personality.)
I think you probably do need to reread it, because AI plays very little role in the article. It identifies problems with an online community college, but most of the problems would be serious problems even if AI had never been invented. In fact, the only problem identified with AI in the article is speculative. The article notes that student website interaction could be provided to AI companies and that the online community college has a conflict of interest–it owns an educational AI.
I thought the claim at issue was that more students are using AI to cheat on papers, because that’s what the original post was about, and also that’s what my post beginning this particular thread was about, and no one brought up questions about integrating AI into teaching and learning in this thread.
The original post also stated that we were witnessing the death of the love of learning as the basis of education, along with some other similarly extreme claims, which I’ve been referring to repeatedly in the discussion. People seem to really like the original post, but not actually remember anything it said, and to act like I am strawmanning everyone if I use the same language it did. I took you to be in part defending the original post, which is why I thought that maybe you took killing the love of learning to be part of the plot. But I’m sorry if I misunderstood your intent.
In any case, if what you are upset about is the integration of AI into teaching and learning, I agree that instructors should get to decide which AI tools are appropriate to their class and which aren’t. I also agree that tech is overused and deployed pretty mindlessly in lots of K-12 education right now. But that latter problem was true before AI. And it seems like a different issue–and in my opinion a much more serious one–than the question of whether we need to change assessment methods.
OK; I shall reread.
I don’t know if we are witnessing the death of learning full stop, but I think we are witnessing the death of post secondary institutions as we have known them for a few centuries. But my focus is not entirely on education in post secondary institutions. Instead my concern is with all the exchanges among humans that make our lives what they are. Lots of people (at least in my city) are imagining all sorts of “use-cases” for LLMs that do not really serve to solve any pre-existing problem, but stand the chance of making the new business owner a buck or two for the novelty and the mantra that with AI, all is cooler and more up to date, even if in the end the AI fails to be more efficient, taking into account externalities or not. Few are asking the question what the point is of simulating humans in every human relationship or community (often or always economic) role. Quite understandably many on this blog want to focus on the practical problem of designing assignments to inhibit cheating with AI; I worry that that’s not solveable in isolation from considering the bigger trends.
“I think we are witnessing the death of post secondary institutions as we have known them for a few centuries.”
For what it’s worth, I don’t think post-secondary institutions have existed in the form we know them for multiple centuries. The small liberal arts college probably had it’s start around 2 centuries ago, while the major research university probably goes back about a century and a half. There are many continuities that current institutions have with the institutions that existed before then (several dozen of them are even the same institutions!) but there are many important features that most institutions only adopted within the past century (like tenure!)
It’s a definite shame that teaching methods that worked for the past few decades are failing in the new technological environment (though I have some hopes, mentioned above, about strategies to save them) but universities have undergone larger shifts over the past few centuries than this.
Derek,
This will be my last post on this thread, because it’s become evident that you’re mainly interested in well-poisoning and throwing out accusations of melodrama against those who aren’t approaching this issue with what you consider to be the appropriate degree of sobriety. I’ll respond to your accusations, but beyond that I feel I’ve said enough.
First, to your allegation that “You accused me of being ‘exceedingly oblivious’ for not agreeing to your interpretation of a trend, in a post in which you went on to admit that the trendline doesn’t match your own interpretation that well. I mean, come on.”
Apparently, this is the most you could muster to back up the idea that I’ve engaged in ad-hominem reasoning. I guess this is what made you lose your composure and start ranting about how I’ve been insulting. Not to mention you once again use an appeal to incredulity with your “I mean, come on.” Evidently, you and your allies are allowed to be edgy in this way, but your opponents are not.
But even in this case, you are misquoting me. The statement was a conditional: IF you are being sincere about not understanding X, THEN either Y or Z is the case. This was a stylized way of making a substantive point, namely, that if you’re in the dark about a certain trend, then you must either have simply overlooked it (oblivious), or have been fortunate enough to find a lot of outliers (lucky). This, Derek, is not an ad-hominem, even if you find it ill-posed.
As to the allegation that my interpretation self-admittedly doesn’t match the trendline: Here again you betray your own lack of charity. My contention was never that AI caused the downward trend, but that its introduction to the public is correlated with an acceleration of the trend. The relation is one of correlative contribution, not causality. I oppose all factors, but AI is a particularly worrisome one, as well as the main subject of this thread.
“It turns out that what “they” did is invent software that, among other uses, makes it easier to cheat on schoolwork, that’s really confusing.”
This is a rather optimistic and uninformed assessment of the motivations of the Silicon Valley tech class. I would recommend looking more deeply into Palantir, as well as statements from other prominent tech CEOs. Perhaps reading The Technological Republic would be elucidatory.
You conclude with some baseless speculation and performative concern. I’ll let that speak for itself.
Yeah, I disagree with your characterization of the debate and your own rhetoric so far. But I think we’ve both said what we’re going to say.
One point worth flagging, though. Regarding baseless speculation: my speculation was that maybe you had a larger complaint than just the use of AI to cheat on class essays. In your response, you say that there is a larger trend you are upset about of which AI is only the part, and you raise more general concerns about Silicon Valley and Palantir–the latter of which has not made AI software that students use for cheating.
My experience this semester very much resonates with Peters’.
First, I should say that I agree with what some have pointed out here: AI obviously creates opportunities for learning. You can use it in order to learn together with it. Sometimes, I read texts in fields that are not my own, and don’t understand, say, statistical methods, formalism, equations, etc, and ask AI to help me understand these. This can also happen in philosophy: say, you don’t understand Jackson’s ‘Mary’s room argument’ while carefully reading through the text, and ask an LLM your specific question. This seems similar to me to googling for further explanations, resources, textbooks, etc., to help you understand a piece of text. I call these ‘good cases’. I just don’t think the author talks about the good cases, and addressing the author’s concern by talking about the good cases is simply a red herring.
The good cases are obviously radically different from simply outsourcing your thinking altogether. Cases like these are the obvious cases of having an AI do all your responses in a given domain: write your proofs, write your essays, write your discussion notes, and even read for you (say, when you don’t want to read a given text yourself, but simply ask it for a summary). I call these ‘bad cases’. Sometimes the line between bad cases and good cases is blurry, but what I take to distinguish these is that the good cases still involve a lot of actual learning and thinking (maybe even an excess of it!), and the bad cases don’t. I take it that the author is talking mostly about the bad cases.
The overusage of AI in the bad-cases-sense has, for the first time this semester, seriously started to worry me. It’s just becoming apparent that students outsource nearly-everything to gen AI. As the author notes, they simply cannot grapple and sit with anything remotely difficult. They don’t even know the line between AI and non-AI any more; when confronting many of my students on their AI use, they respond to me that they simply use AI to ‘draft their responses’, or that they use Grammarly to rewrite their text (Grammarly is fully AI at this point). The unwillingless to learn makes them, unsurprisingly, much, much, less smart. I’ve seen a substantial drop in basic intelligence in my students this semester, ranging from class participation, to email writing, practical skills (them not being able to read the syllabus is now one of their lesser problems), and following arguments that have more than one step. And the unwillingness to learn also makes teaching just so much less enjoyable and rewarding. It all seems way less… pointless. I just feel all I can do is resign to it, not care, and just teach hoping to reach the few students that do want to learn.
Of course, a big part is the incentive structure that’s given by grades: students want to optimize on good grades, and this is just better-achieved by GPT as opposed to sitting through the difficult but formative experience of learning.
I basically agree with this way of seeing things, and your experience closely tracks mine (regional public university). In fact I think the feeling of pointlessness is more what I feel than the grieving that Peters describes. Thanks for putting it so well.
Some days I do feel like quitting because of how sad it is. But most days I don’t, probably in part because I am so early career. On the better days I see it as a puzzle to solve, not something to give up on. I make no claims about whether this is the correct way to see it; it is just useful for me to do so. I also have vague and only partially substantiated hopes that the future might be better.
I am in complete agreement about the terrible downsides of AI in relate to education, and also have speculated that dedicated paper-writing workspaces that insulate students from AI might be needed. I hope other institutions arrange that, because I doubt we will until it’s become commonplace. I’ve had students write essays in blue books for several years now, and this semester set aside significant in-class time for reading excerpts of essays and responding, and plan to do even more of this next year, because otherwise the students don’t read. I grieve how it’s affected class discussion in lower level classes, and worry it will do the same to my upper levels in a few years’ time. I have started looking forward to retirement.
All that aside, I don’t believe it helps the discourse when AI critics (not necessarily here, though I see something close in a comment or two) insist that AI is all downside. Those who have used it and found it helpful in some way will not be receptive to arguments about how disruptive and dangerous it is to education if the critics overplay their hand. If it’s terrible in one way, that doesn’t mean it’s terrible in every other way. (Obviously, but often critics don’t seem to understand this). Many critics appear to have used AI little, if at all; or long ago, before the newer models; or only to do things that they might have done some other way–a better, mentally healthier way–before. That’s not how I’ve used it at all.
Extremely idiosyncratic example (AI is capable of assisting people in highly idiosyncratic ways): my partner and I live in different cities, so I often have long drives. On those drives I listen to books, mainly novels, on audible. I prefer to switch between books as I drive, and usually listen to four books at the same time, arranging it so I finish around the same time. I’m a voracious reader of fiction of all sorts, and like to balance literary picks, genre picks, and in-the-middle (often called upmarket or book club) picks against each other, and throw in a work of general nonfiction. Occasionally my picks have been serendipitous, relating in unexpected ways.
A while ago I fed Claude all my favorite and nonfavorite reads over years, explaining what I liked and didn’t like about them, and describing my four audible at a time habit. (It was spot-on about why I liked the books I did; and why I didn’t like the ones I didn’t). I added that in the summer I have time to read physical books, but prefer them to be ones I might want to keep. I asked it then to help me develop booklists that meet all my preferences and requirements AND will jangle against one another in interesting ways, and also to use one of the audible books I had purchased and not yet read, and on and on. Not only did Claude give me three cycles of books I might read that sound fantastic, it also introduced me to a bunch of new writers, one of which isn’t read much anymore and I doubt I would have ever read otherwise, one of which I had heard mentioned only briefly before, and another which I had deliberately, stubbornly avoided for reasons that are not now clear, and told me their works are the clearest gaps in my reading based on my predilections and interests. Also, Claude told me which book I should buy in hardback. Now it’s possible Claude will be wrong (and then I’ll tell it so), but I don’t think so; and anyway recommendations can always be wrong. It’s true I could have done some very weak version of this elsewhere. My whole life I’ve picked up books because of reviews, and nowadays there’s goodreads and amazon. There used to be a website which visualized the connections between writers, so that readers could find new things to read. None of that got me to these three writers, and helped me to arrange my reading in exactly the idiosyncratic way I wanted.
I do workshops with my students and a supervision process is also included. However, this ‘solution’ has obvious limitations unless one’s institution has no resource limits in terms of manpower and space (mine does).
As for AI as technology is concerned, of course, it is not evil. Like any technology, it has good and bad sides. Overall, I wouldn’t even say we shouldn’t have it. My problem with it is more, shall we say, system-level (as I write above in another comment): since most of these companies are based in the US (but also perhaps if they were based elsewhere), there is little restriction on their conduct. They generate enormous profit for their owners/shareholders while, as usual, they are made to care little or none at all about the externalities they produce. One such externality is the effect on education. I think this is entirely and wholly unacceptable any kind of personal attitude and emotion aside.
These companies should be severely regulated and made to pay for the damage their technology causes. This includes being forced to deal with the effect their technology has on education. It is no panacea but at least it would be something.
“They generate enormous profit for their owners/shareholders while, as usual, they are made to care little or none at all about the externalities they produce.”
If you mean frontier labs like Anthropic and OpenAI that’s plainly false. They are private companies generating enormous debts and making exactly zero profit for now. That is not saying they ‘care’ about externalities.
Right, I did have a look at this now. But these companies do have a lot of funding, which I suppose means their owners are doing well? I am referring to this list: https://www.forbes.com/lists/ai50/ I am guessing that their business strategy is to opt for rapid growth at the cost of lack of profitability so that they can grow out of debt (apparently, this is called the Amazon model).
Plus there are all the tech giants who do make a lot of profit, although not necessarily out of AI.
Thanks for the correction.
It’s striking how horribly uncomfortable we are even talking seriously about grief, no? So much avoidance/deflection. Not a judgement. Just an observation.
“sycophants” is just an insult, not a debating term. It can easily be flipped it into the opposing term “old learning sycophants” and neither advances the discussion one bit. AI isn’t going away. We are all entitled to some grieving, but for the long run we will need to draw on human faculties other than grief.
I am generally an optimistic person, but since the past year or so, I have been increasingly feeling that this is a nightmare that I couldn’t wake up from. I am certainly more afraid than excited. It is just a gut feeling. But I usually have good ones. I just hope the feeling is wrong or that we can still end things here.
On the subject of so-called “externalities,” I haven’t read any comments about Open AI’s culpability in the mass shooting that took place in Tumbler Ridge, Canada, in February. (I haven’t followed these discussions closely, so I may have missed something.)
Here’s a recent BBC article about the lawsuits against Altman stemming from the events: Fresh wave of lawsuits filed against OpenAI by Tumbler Ridge victims
The upsurge in hiring in AI Ethics seems disastrous for efforts to mitigate the dominance of nondisabled male philosophers in the profession.
Probably just as well there are not too many comments yet. I expect the case will be a game changer. Already, I believe, two cases in California (one involving – shall we say – AI supported suicide of a minor) are helping force changes to US 230 law (which in the past enabled social media companies to say of themselves that they are mere platforms, and not culpable for anything their users do). Most of what I see in print about Tumbler Ridge suggests that the prosecution will be about the company’s negligence in reporting concerning behaviour to the RCMP, and this will raise all kinds of difficult privacy questions, but I am guessing the prosecution will also be about the persuasive power of AI, even if they don’t quite put it in those terms. Philosophically, I don’t myself usually have anything to say about various perspectives of various classes of people – I lean towards the view that everyone – even non-disabled male philosophers – can be good philosophers if afforded an appropriate environment to do their work. However, I have to acknowledge that its worth while to raise cautionary flags about the importance of opening the door to radical paradigm shifts right now – however they can come about – so I’m glad you raise this issue.
I don’t begrudge anyone their grief. Grieving can be important. Venting on Daily Nous is probably healthy. Still, what I’d most like to hear about is practical suggestions for how we can mitigate the harm, and we can use AI constructively.
Mitigating the harm and using AI constructively are not the same thing. Like calculators LLMs can do a lot of good and of course it’s a good idea to make the most of them. But like calculators they can also be used to cheat in certain contexts. What we need, by way of practical suggestions, is a way to curb their use in cheating, which is what undermines the practice of teaching philosophy (on which the profession of researching philosophy largely depends).
This is not as easy as it sounds. Two proposals have become routine. One suggests we incorporate AI in teaching and testing. On the surface, this option has all the trappings of the clever, resourceful, flexible kind of solution — it sounds cool and constructive. But it’s just a gimmick and in practice it’s pure confusion, like proposing “Taking an elevator TO the elevator!” as a solution to the problem of elevators replacing stair-taking. Gimmicky nonsense.
The second answer isn’t quite as confused: in-class hand-written exams as the main assignments. But this will lead to a lot of bitterness and opting out, on the part of students who associate philosophy with at least some longer-form writing. Eliminating virtual classes will likewise have this effect. We won’t be able to sell the product at all, and that, too, will undermine philosophy teaching.
I’ve been trying to develop a third way, which is a combination of Answer 2 and cheat-proof longer-form assignments. But the latter is hard to maintain: every time I come up with something, it works for roughly a semester, then AI thwarts it in a new version (the latest case involved a Gemini feature).
Still, in terms of “suggestions for how we can mitigate the harm,” that’s probably the way to go.
Three things can be true together.
The ‘enough’ is important. Obviously you can have mastered some subject even though a lot of the relevant information is NOT stored in your own brain but in books and online resources. But a substantial dollop of knowledge must be there in your head or you won’t have really mastered the subject. Suppose that economic necessity compelled me to put on a course on ‘Stuart Britain’ for the benefit of High School students, a U3A group or even a 100-level history class. I could do a decent job with a week or two’s preparation because fifty years ago we did Stuart Britain as a GCE subject and I have retained an interest in the subject ever since. Consequently a lot of information about Stuart Britain is indeed stored in my brain, and this knowledge is structured. This means a) that I would be in a position to evaluate texts and online resources and b) that having a structured understanding of the period I would be able fill in any gaps in my knowledge with relative ease. I would also know what I needed to know but didn’t. (In this area there are, for me, many known unknowns but not many unknown unknowns.) I would probably be in a good position to detect AI ‘hallucinations’. Because I have internalised ENOUGH information about Stuart Britain I would soon be in a position to lead discussions and answer questions off-the-cuff. Now suppose I was asked to put on a similar course on high-school physics. Could I do this? Perhaps (because I am a competent teacher) but not without months of preparation. The O’level physics that I did all those years ago means that I would (just about) know where to begin but I would have to put myself through an intensive course of study and it would be much harder me to evaluate texts and resources and to distinguish good information from bad. Furthermore, to begin with, I would not know much about what more I needed to know. LLMs alone would not enable me to do a decent job. The reason is that when I was a teenager I did not take as much pains to internalise what I was taught about basic physics as I did to internalise what I was taught about Stuart Britain. And in the interim a lot of that knowledge has gone to rust.
The point generalises. The things you ‘know’ as an extended mind depend upon your knowing ENOUGH as an unextended mind. In order to be a competent doctor, lawyer or engineer, you have to have internalised enough structured information to use external resources effectively, and that includes AIs. And unless the education system takes serious steps to prevent or mitigate AI-based cheating, many graduates will end up not knowing ENOUGH. Furthermore they will not have learned to learn. That’s why Mark Champagne is very much onto something.
However, ‘Bilingual’ probably wrong in supposing that the threat posed by LLM’s to higher education was part of the plan. ‘Increasing productivity’ (which means producing the same or a greater quantity of goods and services with less labour input) certainly was part of the plan, but I don’t suppose the executives in question deliberately aimed to facilitate cheating, thereby undermining higher education and devaluing the worth of credentials. For it was not in their interests to do so. They may hope to get by with less people to work for them, but they will still need some people to work for them, and those people a) must know enough about their stuff to be effective and b) must be known to know enough about their stuff to be effective. And this means that they must have genuinely qualified for their credentials. Thus the damage done to higher education and to education in general was probably an unintended consequence. There probably was no conspiracy to bring about an education crisis by enabling students to cheat with ease and impunity. But an unintended consequences is still a consequence and sadly we will have to put the educational clock back (with invigilated exams etc) in order to deal with it.
Now for the self-pitying and self-dramatising rhetoric about grief. People tend to acquire a love of learning if they are incentivised to learn and the learning experience is enjoyable (bearing in mind overcoming obstacles and doing something difficult can be fun). LLMs create an environment where students are incentivised not to learn but to fake learning (since try doing so they tend to get higher grades). The robots help create an environment where cheaters tend to prosper and in which, accordingly, many people cheat. To re-incentivise learning we have to disincentivise cheating by reconfiguring our assessment regimes, so that cheaters tend NOT to prosper. This can be done, for instance by putting a large proportion of the mark in a cheat-proof final exam. Is it otherwise undesirable to do this? Yes, because final exams are, in the odious jargon which we have to put up with, ‘summative’ rather than ‘formative’: you don’t learn by doing them but only manifest the learning that you have already acquired. In the absence of cheat-facilitating LLMs an assessment regime based on long-form take-home essays would definitely be preferable (at least in philosophy) since students learn about the topic in the course of writing their essays. So in minimising the share of the grade that goes on take-home essays (for example) and maximising the share that goes to cheat-proof mechanisms such as final exams, there is undoubtedly some loss. But it is absurd – simply bonkers – to suggest that nobody can acquire either a love of learning or a love for their subject in the context of a (relatively) cheat-proof assessment regime. After all, many of us have done precisely that. From the age of eleven until I graduated from Cambridge at the age of 22, all or most of the final mark for the vast majority of my courses went on a final exam.
If there were no solutions to the problem of LLM-based cheating then grief might be appropriate. But since there are such solutions (suboptimal though they may be) the wails of sorrow are somewhat premature.
We’re getting a little far afield from the original post, but my main worry with final exam classes isn’t that they’ll fail to inspire a love of learning or whatever, but that:
1) You can’t test research skills, long-form paper composition skills, or extended writing and revising skills with such an exam, and I think these are important skills for students to cultivate.
2) Unless you have open note exams, these exams substantially test people’s memories, and I am not interested in sorting students according to their memories. And if you have open note exams, people will have AI write their notes…
Do you have any solutions for these? For #1 you could have a final exam where a relatively vague topic is announced ahead of time and it’s the job of the students to acquire sufficient knowledge of the topic themselves and that would help with the research skills part. You could also provide tutors who help students develop their knowledge over the term by reading papers the students write. These two together form what I believe to be the Oxford model. But this requires sufficient capable tutors, and almost no professor is in a position to swap their course over to the Oxford model unilaterally unless they have a tiny number of students.
Regarding 2, if the students are using AI-produced notes, they still have to understand the material well enough to explain what those notes say in an essay. Also, while we definitely don’t want to teach rote memorization, the difference between sorting students according to memory and sorting them according to knowledge of the course material is pretty thin. And for anything complicated memory and understanding don’t cleanly separate (my ability to remember Aristotle’s function argument at the start of NE is greatly aided by my ability to understand it).
We had blue book exams when I went to Berkeley, which certainly had too many students to follow the Oxford model. It worked fine, and it had its upsides. I think the fact that it required you to remember the whole class was actually good. By the end of the semester you had a very comprehensive understanding of the topic. We met in groups to study, by going over the possible test questions and making sure we were prepared to answer them.
I think take-home essays are better, but it’s worth keeping in mind that the shift to almost all take-home essays in humanities is fairly recent and was enabled by widespread personal computers and Turnitin. One reason to think education is still possible after AI is that it was possible when sit-down exams were the norm (including at big state schools) and when plagiarism was very hard to catch.
“the shift to almost all take-home essays in humanities is fairly recent and was enabled by widespread personal computers and Turnitin.”
I apologize for being argumentative on a factual matter, but it’s not true. I went to Berkeley too and we had many papers (as well as some bluebook exams) but with type-writers and onion skin paper, not computers. (In fact I still have a couple that I’ve carried around with me for all this time.) So computers didn’t cause (enable) take-home essays. I think essays for undergraduates were for the most part enabled by teaching assistants.
Please be argumentative about factual matters if I got something wrong.
I may have overstated things, or at least expressed myself poorly. I also had a lot of papers at Berkeley–a huge number, in fact. What I meant was that the shift to *almost all* take-home essays was enabled by the technology. I meant that even though I had a lot of take-home essays, blue-book exams were still pretty common. Sit-down exams becoming very rare seemed, in my experience, to happen later. I agree that personal computers were probably less important: almost everyone had them by the time I was in college. My sense is that of the two, Turnitin was the bigger factor here.
As someone who did grading before Turnitin, I think there used to be a lot of plagiarism that was very difficult to prove, and basically had to be tolerated.
But please let me know if this doesn’t match your experience.
Turnitin has never loomed large in my consciousness due to my institution’s concern for student privacy (much to the consternation of many faculty members). Remember that personal computers were around for quite awhile (like 8 or 10 years I think) before the WWW. I miss the cheating of the good old days – there was more variety and innovation involved.
I taught in Canadian universities from the early 90s. In that milieu, the thinking seemed to be that exams were good for gauging breadth of knowledge but less good for assessing depth of understanding and analytical aptitude. Essay questions on exams could help for the latter purpose but were still inferior to essays written outside of class (which also afforded more opportunities for learning and developing research abilities). For these reasons, exams tended to be required only in first-year courses (if at all).
Time lines were another factor. The grades for a final exam had to be in by a date set by the university, and the university set the exam’s date (with no input from instructors). If the university scheduled your exam late in the exam period, you might end up having only three days in which to grade all the exams. This was especially galling if you had included essay questions on the exam.
To avoid this predicament, professors started scheduling in-class tests in the final week of term. Since these were not officially final exams, professors had more control over the scheduling and could allow enough time for grading the tests. University admin’s responded with new rules (e.g., no test in the final two weeks of term could account for more than 10% of the course grade), at which point more professors shifted more of the course grade’s weight to out-of-class essays.
Regarding old-fashioned cheating, I recall the Toronto police department raiding an essay mill, one of those illicit businesses that charged students a fee for writing their papers. Yes, in the before times, the police helped the universities fight cheating.
Okay, I found the info about the Toronto police busting an essay mill. It happened in 1989. Quoting from the Toronto Star: “Metro police, called in by York University officials after a student confessed to handing in an essay purchased from a service, obtained a search warrant and raided the Collier Street service in April. … The information was shared with all of the province’s 16 universities at a special meeting two weeks ago and since then university officials have been calling in students to explain. … Investigating officer Constable Graham Hanlon of the Criminal Investigation Bureau says, … ‘These people are going to become politicians, doctors, and lawyers…. It makes you wonder about their moral and ethical responsibility. This is just another example of where our society is going.’” (Trish Crawford, “Universities Get Tough With Cheaters,” The Toronto Star, May 9, 1989, p. A20)
What law was the essay mill breaking?
It is a very old law in the province of Ontario pertaining to poor citation practices, of course 😉
The reporter said that York University initially called in the police for cases in which a student had someone else write an exam for them. Those students were charged with “personation,” an offence in Canada’s Criminal Code at the time (later replaced by a law against identity fraud).
The reporter said nothing specifically about the laws applied to the essay mill, but the universities might have tried to stretch the personation law to cover those cases, too. According to Canada’s personation law at the time (Section 403 of the Criminal Code), “Every one who fraudulently personates any person … with intent to gain advantage for himself or another person … is guilty of an indictable offence….”
I take it this law was then (or later) found not to apply to essay mills. I found a CBC report from 2015 about people advocating for a new Canadian law against essay mills (“Windsor law student calls for action against ‘essay mills’,” CBC News, April 7, 2015).
Other jurisdictions have developed laws against essay mills. According to the Wikipedia entry for “Essay Mill,” 17 US states have laws barring essay mills. For example, California Education Code Section 66400 bans the sale of term papers, etc., and Florida Statutes Section 877.17 prohibits selling assignments. The Wikipedia article says similar laws exist in Australia, New Zealand, Ireland, Austria, and England and Wales.
In England and Wales, essay mills were banned by the Skills and Post-16 Education Act 2022. I found a publication about whether using gen-AI for assignments runs afoul of that legislation: Gaumann, N., & Veale, M. (2024). AI providers as criminal essay mills? Large language models meet contract cheating law. Information & Communications Technology Law, 33(3), 276–309. https://doi.org/10.1080/13600834.2024.2352692
We had blue book exams when I went to Berkeley,
When I moved to Australia from the US (in 2017) I was shocked to find final exams being done on bluebooks. This was in part because it was really difficult to physically deal w/ all the bluebooks. (I often had more than 100 students in a class. Sometimes many more.) But more so, it was bad because many students had literally unreadable handwriting. And it was most shocking because exam software that locks down computers and sends electronic copies to the markers had been around for at least 14 years at that point. I know because I had used it then. There is no reason to have blue books for most exams. Exam lockdown software is easily available and not hard to use. Most students and almost all markers will appreciate it. (The only downside is that it makes marking at the pub harder.)
Wrt to Problem 1). Part of the mark can go can go on one or more essays which, if done right will develop the students’ research skills etc. These won’t be cheat-proof. However, cheating is disincentivised because cheaters are unlikely to prosper when it comes to the final exam which should be worth at least 50%. Students are likely come a cropper if they do not write their essays themselves because the exam questions should not correspond directly to the essay prompts but should require an in-depth understanding of the issues. (The exam questions should cover much the same issues as the essay prompts but they should come at them from a slightly different angle. Thus memorising and regurgitating an AI-generated essay that is addressed to the essay prompts rather than the exam questions will probably lead to failure, especially if the lecturer marks hard for relevance.) An alternative strategy is to make essays mandatory, in the sense that students have to write them, even though all of the mark goes on the final exam. Something like this was the system at Cambridge when I was young. (I should add that we got extensive feedback on our essays.) However I think it is usually better if some of the mark goes to the essays so long as at least 50% goes on the final exam. That’s the system that I have reverted to ior the last three years of my teaching career in order to deal with the problem of LLM-based cheating.
Wrt Problem 2) that ‘these exams substantially test people’s memories’, I am simply staggered that you consider this an issue. One of the skills we should be cultivating and testing for at universities is the ability to process and critically assimilate information, and this is partly a matter of memorising it (in the right kind of way). Nobody would question this wrt vocational degrees, where, as I have argued, it is important to retain a substantial body of structured information in your head. If you can’t remember exactly how the knee-bone is connected to the thigh-bone you are likely to be positively dangerous as surgeon, a doctor or even a nurse. Though it is less important that philosophy graduates retain the details of what they have learned in their subsequent careers, the ability to at least temporarily master a body of information (which is partly a matter of memorising it) is enormously useful in later life. It’s what lawyers have to do in mastering a brief and its what lots of us have to do when mastering and acting on reports when serving in a governance or managerial capacity. So what I am inclined to say is that as a university lecturer, you jolly well OUGHT to be interested in ‘sorting students [in part] according to their memories.’ I agree that an exam which simply tested for rote-learning ability would not be worthwhile, but that is not what I am suggesting. I am suggesting an exam that tests for memory, understanding and the ability to answer questions relevantly which requires some ability to ‘think on ones feet’. But these are the kinds of exams that I was accustomed to setting for the first twenty-five years of my teaching career.
So although dealing with the AI cheating threat will leave us worse off than we were before with all the paraphernalia of exams to invigilate etc and although it will force us to rely on summative rather than formative methods of assessment, it is not as if there are no solutions, many of which have been operating with moderate success for decades. AI-based cheating is a Bad Thing, yes and a genuine threat, but there is no need for despair or for wailing and gnashing of teeth.
PS Most of my teaching since about 2000 has been at 200 or 300 level and I have usually marked BOTH the essays AND the exams. A chore, but doable in a class of up to fifty.
2) Unless you have open note exams, these exams substantially test people’s memories, and I am not interested in sorting students according to their memories. And if you have open note exams, people will have AI write their notes…
When I was in law school, most classes had one big exam as almost all of the grade. Most of these classes allowed people to bring in “outlines”. There were often banks of outlines available. But, in my experience, the most important part of having an outline was either making one, or at least going closely over one’s notes while revising one. When I did that, I in fact hardly resorted to the outline during the exam – and that was good thing, because if I had to regularly resort to it, I wouldn’t have had enough time to answer the questions. So, I am not overly worried that people who have used AI rather than do the readings will use AI to make notes for an exam and then will do well, if the exam is made well at all. Those people will flounder and not be able to answer the quetsions. (This is my experience from teaching students with open note exams now.) I should note this requires exam lock down software, but since such things were easily available since I started law school in… 2003… there is no good reason at all for any university to not use it now.
I am glad to see, as this debate has aged over the past week and the dust has settled, the tide is starting to turn against the melancholic lamentations of the Weepers and Grievers crowd. Thank you Charles Pigden for articulating (more successfully than I have so far) about why these melodramatic wails of sorrow are premature. (Bilingual and Runa have been cheering on the weeping and the navel gazy melancholia and no doubt will continue to do so, but I’m glad to see this more solutions-focused and productive kind of thinking here.
I don’t think the (as you put it) Weepers and Grievers crowd is necessarily inconsistent with also being forward focused and solutions driven. My understanding of the grievers’ line in that they think that grief merits a place within the wider response we should have to AI in education and where the wider response can include solutions
Having grown up in a time where I was able to experience the “before” and “after” of learning via AI, I do have to say that my own approach to learning new material has vastly changed. Before AI systems, for example ChatGPT, came about it was a matter of trying to connect the dots in order to fully understand a certain topic. It would require time to be set aside and having to go through a lot of confusion and frustration. Nowadays I can generate the easiest possible way to understand a lesson with close to zero or minimal effort. I tend to wonder how I ever managed to get through school without AI tools and even classmates joke about “feeling smarter” before AI took over education.