We’ve asked before: “What is your department’s AI policy?” Perhaps you think a good academic AI policy should, to the extent possible, be sensitive to the kind of thing the AI is being used for, and by whom. So what should a department’s policy be about graduate students using AI to write a dissertation?
I ask because, according to recent analyses, some students are using AI to write at least part of their dissertations, and the number of dissertations containing AI-generated text in the United States is increasing.
According to a new working paper by Daniel P. Gross (Duke), Dror Shvadron (Toronto), and Hansen Zhang (Duke), between a quarter and a third of the dissertations that earned people their PhDs in STEM fields this year in the US contained AI writing.
They say:
We measure AI-generated writing in the dissertations of a near-census of US STEM PhD graduates from 2019 to 2026 and link graduates to their subsequent employment. [“We process all abstracts and a stratified random sample of complete dissertations (stratifying over fields and years), which we use to verify that AI-generated abstracts are a surrogate for AI-generated writing in the full text—consistent with prior research which has found the same.”]
AI writing is absent before 2023, present in 29% of dissertations filed in 2026, and rapidly growing. Use is more common among students from non-English-speaking countries and at lower-ranked programs. Among graduates in the same program and cohort, those whose dissertations contain AI writing are less likely to enter academic research careers and more likely to enter non-tenure-track roles or industry, with this pattern strongest among US-origin students. At the same time as AI may increase research productivity, it thus appears to also be displacing a potentially important portion of the work through which scientific expertise is developed.
That’s in STEM fields. What about philosophy?
I asked the team that produced the above study if they could run a similar analysis on philosophy dissertations. Very kindly, they agreed to do so. (And very kindly, my department at the University of South Carolina agreed to fund the use of the Pangram API with which they did it.)
As with the STEM fields, in philosophy the trend is an increase in the number of dissertations with AI writing (operationalized as containing at least 2% AI-written text). In 2024, 1.3% of philosophy dissertations included AI writing. That figure jumped to 7.8% in 2025, and continues to rise. Only about a quarter of the philosophy dissertations expected to be filed in 2026 were analyzed, but so far 11.1% of them contain AI writing.
What to make of this?
One cautious point is that the numbers on that graph do not tell us how the AI was used. Perhaps not all of the types of AI use that could lead to a piece of writing having a “high” Pangram score are objectionable? Relatedly, we might ask whether we can adequately craft policies that make the right fine-grained distinctions about the permissibility of AI use, and whether such policies can be enforced.
Another thing to note is that although the rate at which AI writing shows up in philosophy dissertations is about one-third of that in STEM fields, this is not much comfort to those in philosophy who object to such writing. One reason is that shifts in academic norms are not necessarily confined by disciplinary boundaries. If the prevalence of AI-written text in dissertations becomes normal in other disciplines, the expectations we will have in philosophy will certainly be affected, as will the expectations of administrators. Number-crunching deans may want to know why they’re paying for graduate students to spend a few years writing a dissertation without AI when, with it, they could be done with their dissertations in a few months, or perhaps even weeks or days.
If departments aim to prohibit or strictly limit how AI is used in the writing of dissertations, best to try to do that now, while norms are in the air, rather than waiting until later, when perhaps largely unrestricted AI use by graduate students on their written work will have been normalized.
Discussion welcome.
*Note: There were some challenges determining the set of philosophy dissertation abstracts to study. Initially, the team searched available abstracts and metadata of dissertations in ProQuest that listed “philosophy” in the department or any subject search field. That overcounted philosophy dissertations by around 300% (which we noticed by comparing those figures with the number of philosophy PhDs earned each year). This is not surprising, since the word “philosophy” is used by more than just philosophers. Limiting the search to “philosophy” in the listed department and ignoring its presence elsewhere in the abstract and metadata had its problems, though, too. For one thing, some multi-disciplinary departments include philosophy; and so by this method, all of the dissertations produced by a department of “Psychology & Philosophy” would be counted as philosophy dissertations, even if the department only offers a PhD in psychology, not philosophy. So we eliminated those departments. But then we noticed a different problem: the dissertation counts of some departments were bizarrely low. For example, according to the department-only ProQuest search, Stanford University’s Department of Philosophy appears to have produced just 1 dissertation between 2016 and 2026—obviously a mistake. Several other departments had this undercounting problem. When I pointed this out, one of the researchers said the likely explanation is that “department information may be missing or recorded differently in ProQuest”. We did not search for these “missing” dissertations, and so the set of dissertations on which the study is based is not identical to the set of all philosophy dissertations filed in the US during the years studied.
This is all very troubling, and I’ve wondered about the fate of dissertations for a while now. I do want to say, as someone whose second language was English, that I greatly sympathize with those who use AI who are not native English speakers. While we rightly prize clear and concise writing, there are many brilliant philosophers who lack the ability to write with the clarity of a Harry Frankfurt, who nevertheless have valuable contributions to offer. And it is easy to dismiss a philosopher who speaks with a thick accent and is not as skilled in English as inadequate because they lack the ability to fully articulate their arguments even though they may actually be far better thinkers than those who dismiss them will ever know. This isn’t really anyone’s fault, but AI can be a corrective here. At the same time, AI is also very harmful in so many other ways. I really don’t know what the future holds anymore.
I think this needs to be tempered with the data on Pangrams FPR in evaluating abstracts according to this recent study (https://arxiv.org/pdf/2608.11256). It seems to show that there are higher FPR for non-STEM fields, that light editing can trigger the tool to categorize the text as AI generated, and that basic humanizers can bypass it. The 2 percent threshold here seems wildly low, as there are inevitably stylized constructions within a paper that will trigger the detector. We must be very careful to take these detectors as sure signs of the AI phantom
All of that seems right to me, but the 0’s at the left end of the graph are some evidence in the other direction.
From the description, I would have expected quite a few false positives, for just the reason you mentioned. The other day I was looking at something whose syntax made me think “That’s AI”, then I saw it was published in the mid-2010s. It isn’t hard for the occasional piece to have just 2% sound like Claude’s distinctive syntax, and I thought there would be more pieces where Pangram had the same reaction as I did. But if they’ve got a script that puts out 0’s until 2022, that’s interesting, and shows they aren’t making the same mistake as my intuition. It would be good (though expensive) to run the study back even a few more years, to make it clearer what the comparison is.
That paper shows that Pangram flags 0.0% of abstracts from 2013-2015 as AI-written, and 14.9% of abstracts from 2023-2025 as AI-written. One possibility is that in fact all the abstracts from 2023-2025 were human-written, so that we can interpret 14.9% as an FPR But there is an obvious alternative explanation. (And ~15% of abstracts being AI-written or AI-assisted averaged over 2023-2025 is qualitatively compatible with the data on dissertations in Justin’s post.)
The basic thesis of the paper is that humanization can fool AI, but that AI detectors will often flag even light AI editing. The former would just make Justin’s numbers higher (though I think Pangram’s response would be that it will have a much lower FNR on humanized long-form text than on humanized brief abstracts, which is what the paper looks at). The latter is explicitly acknowledged in Justin’s post.
“The 2 percent threshold here seems wildly low, as there are inevitably stylized constructions within a paper that will trigger the detector.”
Maybe, but I have seen no evidence of this. I and others have looked quite hard for any verifiable report of a piece of long-form academic writing from before 2022 that gets lower than 100% human-written in Pangram, and we can’t find any,
I’m confused as to how this comment addresses the original concern brought up by Skeptic. What is the significance of the 2022 date in this instance? Isn’t the idea that Pangram and other AI detectors would be used for papers written recently – not from 4 years ago?
This is true but I think understates the significance of their findings:
These are not inconsequential. You rightly suggest you wouldn’t use it for an academic integrity investigation but elsewhere have suggested using it for filtering out journal submissions and here for population level studies. Given what we know about Pangram, I don’t think we can make any sort of inference from studies using this API other than Pangram flagged it as potentially being AI written. That is not the same as evidence that it was AI written and it seems there is a high risk that Word’s own AI grammar checking could be causing false positives.
This is possible in theory, but it seems like high conjecture. The study does acknowledge that they cannot confirm that these papers are truly original since they didn’t observe them being written (though that would be a different, more discriminative study) but you and the researchers are leaning two two different interpretive biases: they give the benefit of the doubt to human writers, and you’re giving the benefit of the doubt to Pangram’s API.
At the very least, do recall, the researchers specifically marked these as false positives.
I don’t want to live in a world where the default is ‘prove this isn’t AI’. That gives way too much credibility to an otherwise unproven tool whose basis for truth is that people still write a bit differently from AI. Give it a few years and I wouldn’t be surprised if increased student interaction with chat bots causes linguistic crossover in a way that is much harder to detect than it already is (or AI becomes much, much better at mimicking humans).
The actual quote you embed as evidence that “ the researchers specifically marked these as false positives” explicitly says that they are *not* false positives, so I don’t understand what you are trying to say here.
2% is very low for a threshold, especially for a very short chunk of text. 10 words of a 500 word abstract? I think Pangram is a wonderful tool but I don’t think that’s a useful threshold.
2% on an abstract is low enough that I might feel uncomfortable using it in an academic-integrity investigation, but I think it’s fine for a population-level survey. The researchers Justin is describing did the obvious thing and ran their protocol for dissertations from 2019-2022; they report 6 abstracts out of 126,000 detected as AI-written given their threshold. Those six are presumably false positives; that’s a FPR of about 1 in 20,000, which incidentally is pretty much Pangram’s own in-house reported FPR.
Sure, and it’s certainly interesting that it doesn’t pop pre-2022; but it’s also compatible with more benign explanations. We’re all soaking in AI-generated text that it’s not surprising that we might have a small percentage of people that pick up a couple of turns of phrase. (Or alternately, people like AI for punchy endings, which would mean about one sentence in an abstract.)
Logically compatible, sure. Many things are. Plausible? Scarcely.
Fair enough. I’d still be interested to see how many fail a higher threshold, or to learn why 2% was how they operationalized ‘written by AI.’
Me too; it’s a rich and interesting dataset, there’s probably a lot more to get from it.
There are two big benefits I see with using AI for dissertations. Firstly, if by using AI better philosophy is done, then it should be used. As philosophers, doing good, meaningful philosophy is our ends. If the means to those ends are using a controversial technology, bucking romantic notions of what ‘philosophy’ is, burning down the rainforest, giving more power to a class of elites that actively hate us, so be it! Secondly, AI makes it easier for young philosophers to do all the important stuff in philosophy, e.g., writing emails, streamlining teaching, writing articles for journals. Prohibiting AI in graduate school prevents students from learning how to optimize these tasks. It’s a disservice, really.
“If the means to those ends are using a controversial technology, bucking romantic notions of what ‘philosophy’ is, burning down the rainforest, giving more power to a class of elites that actively hate us, so be it! “
This is a joke, right?
It’s definitely a joke, since the next paragraph says it helps young scholars write more emails.
One would hope (one would think) that philosophy as a discipline has more going on right now than issues surrounding AI (as important as they may be). I’m a bit tired of only reading AI-related things on DailyNous. Again, this is not to devalue how important these issues may be, I just feel that we might be losing sight of other equally if not more important things.
That’s a lot of AI, for sure. But this is Justin’s blog, and it’s his privilege, as the sole editor, to publish what he wishes. What he posts about is what he considers interesting, important or relevant, and should be read as such. I don’t disagree with him. But if one does, one can just not read the blog.
I think he’s at least somewhat sensitive to the interests of his audience, so it doesn’t seem unreasonable to simply note that it’s a bit much sometimes (though I disagree; personally I find the AI stuff really interesting and important, if horribly depressing).
Surely you’re aware of what just happened in mathematics.
I am disturbed by the prevalence in STEM, and heartened by the lag in philosophy.
I will say this, though. Many of these people will go on to experience impostor syndrome, since most of us do. But in their cases, that anxiety will track the truth: they really are impostors.
Why are you disturbed by the use of AI in STEM? If scientists use AI to develop new vaccines faster, find cures for diseases like Alzheimer’s, or build new green energy technologies, would that be disturbing?
I am disturbed because they used it to write their dissertations. A few years ago this would have been called contract cheating.
If they used it to cure Alzheimer’s–they didn’t–then great, they’ve cured Alzheimer’s. But they have not earned their doctorate. They are impostors.
Perhaps the can be given an honorary doctorate for their success. Fine. But they have not earned their doctorate.
An AI edit to fix my poor grammar will be flagged as AI. What am I being judged on for my doctorate? My ability to speller and grammer (sic) with perfection? Or my ability to conduct research and extend a body of knowledge?
End of the day, I have to stand in front of a panel and defend my thesis. That is the point at which my work stands or dies, not the fact that I used an AI research agent to find papers Google Scholar missed, or used an AI to fix grammar, unwieldy sentences, or check my work for logical fallacies.
Or the fact that I am disabled and have no hands and cannot type, so I use another AI to convert my voice into text, to layout my work and issue commands to move and edit text on a screen.
But hey, at least I do not have a mentor that thinks my AI use equates to not having earned my doctorate. You would think that a philosopher might be able to see past their own bias and find the good things that AI could bring the world. Its opened up a whole new world for the disabled, its given them opportunities in work and education, but I guess, to some, we will always be second class students hey?
We’re talking about using AI to write the text of a dissertation.
That is a hill I will die on, so to speak. If you use it to write some or all of the text of “your” dissertation, then you have not earned the PhD. Likewise, if you plagiarized a few paragraphs, you have not earned it. If you had your mum write a few pages, you have not earned it. Full stop.
I can accept some uses, and it should not be very hard to determine what those uses might be. But I cannot and will not accept text generation.
I think you’re making an assumption that writing and authorship are the same thing. They’re related, but they’re not identical.
If the dissertation exists to evaluate my originality, research ability, argumentation, and subject mastery, then why should correcting grammar, restructuring awkward prose, or suggesting clearer wording count as disqualifying? None of those activities generate the underlying intellectual contribution.
Plagiarism is wrong because you’re claiming someone else’s ideas as your own. The mere fact that software helped refine the expression of my ideas does not automatically make it plagiarism.
The difficult question is where assistance becomes substitution. Simply calling all AI-assisted text generation “not earned” sidesteps that questions I posed rather than answering them.
Put this text into an AI text tester and it will fail as 100% AI` (intentionally so). The harder part to prove is that they are not my thoughts, my arguments and my contribution to this discussion.
When I stand in front of an evaluation panel,what ultimately matters is not what produced the text on the page. What matters is whether I understand the arguments, can defend them under scrutiny, and can demonstrate genuine mastery of the research. If I can do that, then the claim that I have not “earned” the degree requires a stronger argument than simply pointing out that AI was involved somewhere in the writing process.
I have worked as a copy editor.
I have no problem with checking spelling and grammar, though I suspect that appeals to it are used to rationalize much worse behaviour. Indeed, since I have worked as a copy editor, I know that there’s copy editing and then there’s copy editing, where the copy editor ends up substantially rewriting chunks of text to make them more intelligible. I rather suspect that’s what a lot of the “spell checking” devolves into.
I remain firm in my assessment: the more you outsource to the chatbot, the less of the work is your own. (Pro forma) Viva or no viva, if you didn’t write it yourself, the credential is not yours. Them’s the breaks.
” have no problem with checking spelling and grammar, though I suspect that appeals to it are used to rationalize much worse behaviour.”
That’s my suspicion too, at least in many cases. Claude is a very good copyeditor, better than nearly any human I’ve worked with. I accept its spelling fixes, often accept its grammar fixes, consider its objections to style and structure and occasionally revise in response to them, but always draft any relevant change myself. None of that flags Pangram. Nor do literal translations between languages. But Claude* has a distinctive style and you can often tell when people are going beyond that kind of thing and actually getting it to draft phrases, sentences, paragraphs, or the whole paper.
*so do other AIs but Claude’s the one I’m most familiar with stylistically.
Reply to David and Michel:
“But Claude* has a distinctive style and you can often tell when people are going beyond that kind of thing and actually getting it to draft phrases, sentences, paragraphs, or the whole paper.”
“I know that there’s copy editing and then there’s copy editing, where the copy editor ends up substantially rewriting chunks of text to make them more intelligible”
One assumption in these comments seems to be that something like using Claude or other AI programs to help revise sentences or a few paragraphs, restructure awkward prose, or suggesting clearer wording, is a terrible thing. Obviously there are degree to this, but these comments seem to ignore that.
I agree that using AI it to write a whole paper is not good. However, suppose I say to some AI “You know, I feel like I’m doing a poor job of getting my point across in this paragraph. Something seems off or missing” and then the AI offers a few slight revisions of some of the sentences and I go with some of those revisions. That doesn’t seem like misconduct, anything done in bad faith, or harmful. Friends and colleagues will often offer such suggestions for journal articles. I’ve made such suggestions. Book, magazine, and newspaper editors will often offer such advice, as Michel says. Of course, there is “substantially” rewriting and “lightly” revising. What I’m talking about here as non-problematic is the latter.
My issue is that a student could plausibly use this method (light revising) a few times in their dissertation and then their thesis gets flagged as AI above something like 2%. If they use it often, maybe even 20%, especially if they are not a native English speaker. If another poster from another thread is right, the Pangram score might even come out substantially higher in some (many?) cases as Pangram seems to flag wholly human written text sandwiched between AI-edited sentences as AI, even sometimes (from my playing around with it) human-written sentences that come before or after AI-revised or written ones.
So, a student might use this light style of editing, have all the arguments in the dissertation be their own, and the policies being suggested here would prevent this student from graduating. That seems wrong to me.
On the other hand, a student could have used AI to come up with ALL the arguments in their thesis and then go and write the dissertation in their own wholly human words. That dissertation would not get flagged by Pangram. Now, THAT seems like a student who should fail. Yet the policies I see mentioned in this thread would not apply to that kind of dissertation. Or if they did, it seems there’d be very little probability of catching that use (even an oral examination might not if the student did a good enough job – as most do – of understanding, memorizing, and rehearsing their arguments).
TLDR: In short, I see no problem in using AI to edit one’s own words, even beyond spelling, grammar, and punctuation. I do have a problem with people using AI to come up with or improve arguments. The former can be detected. The latter cannot reliably. And yet a student using the former former method would be flagged by Pangram and not allowed to graduate while the latter kind would not get flagged by Pangram and still be allowed to graduate. Makes no sense.
***
I’ll add two other thoughts.
1. I’ve seen David elsewhere claim that he thinks the argument can’t be separated from the writing/words — or something to that effect. I think one of the posters in that thread had a good point about argument theft. They suggested argument theft is often done by the thieves changing the wording, which seems obviously true from my own experiences. David has also elsewhere admitted to using AI to help improve his own arguments by asking for feedback or critique. If David is right, however, wouldn’t this mean that David’s own arguments (at least those parts influenced by AI) won’t be separable from the AI’s writing or words and so that David is also doing what he is preaching against in effect, just that he’s writing it by hand? Or does David not believe that, which would seem to go against his own inseparability thesis? Also, I have not seen David really give his own positive argument for why we should think the separability thesis is true. I’ve only seen him make the claim and then critique arguments against it, yet he seems to hold that view very strongly.
2. AI is a tool. Some philosophers of art have already written on this regarding the visual arts. By saying no to any creative uses of AI, we are in effect limiting philosophers (who are arguably artists of a sort) from experimenting with a new tool and expanding the art form. We’re deciding before anyone really has a chance to play around with it that it is a tool that should be banned from the craft completely (aside perhaps from grammar, spelling, and punctuation editing). Artists have been using new tools in new creative ways from the dawn of time. Some painters in the past and present used the camera obscura method, or used stills from movies projected on walls to aid their painting. Many musicians have used a producer to help them make their music. David Bowie even used a random word generator. Most filmmakers now use editors. Some filmmakers use digital effects. Most use sound effects. The list goes on. Some artists are now using AI in various ways, though there is obviously a debate in the visual and auditory arts as well about the reasonable limits. In the arts, however, a blanket ban on AI has not largely been accepted. Filmmaking is perhaps the one art where there does seem to be growing general opposition, yet in that case it is due to concerns about replacing workers and costing them their jobs. In any case, it’s puzzling to me that out of all the artists out there in all the various art forms, it is in philosophy that we find the most intense calls for blanket opposition. At least in the other art forms there are worries about artists being displaced by AI. Yet, in philosophy that doesn’t seem to be the core worry at all — instead it seems to be about the use of ANY AI in any way (aside, again, for the correction of grammar, spelling, or punctuation, or in David’s case the use of AI to correct and revise one’s arguments, which still seems possibly suspect to me and much more so than light editing of word or sentence revising).
I should clarify:
1. Some philosophers have clearly embraced the use if AI in various ways and expanded the art form. I’ve found more than a hand full of philosophers (some tenured, some not) at top American, UK/European universities whose papers get flagged at higher than 20% on Pangram. Some of these philosophers are quite famous (as far as fame in philosophy goes).
2. David’s argument appears to be that the lack of separability between arguments and their expression in words is just obviously true. Of course, someone (myself included) might claim the opposite (i.e., that it is obviously true an argument can be separated from the way it is expressed in words in some paper).
I don’t think it’s “obviously true”; I think it’s true (at least if you replace “separated” with “cleanly separated’). I wouldn’t spend extended time on a blog trying to persuade someone who disagreed, any more than I’d try to persuade them of compatibilism; it’s not a good or rewarding medium to do substantive (meta)philosophy, especially with anonymous interlocutors. I admit I would be surprised to find an experienced philosopher whose work I respected who disagreed, as I think reflection on philosophical method makes it fairly apparent. There is a reason why serious study of an area in philosophy requires you to read the actual arguments and not just their textbook paraphrases.
> “However, suppose I say to some AI “You know, I feel like I’m doing a poor job of getting my point across in this paragraph. Something seems off or missing” and then the AI offers a few slight revisions of some of the sentences and I go with some of those revisions. That doesn’t seem like misconduct, anything done in bad faith, or harmful. Friends and colleagues will often offer such suggestions for journal articles.”
I don’t necessarily think it is always done in bad faith. But I disagree that it’s not misconduct (depending on the specifics of the case) and not harmful. To me, finding the right words for what I want to say is part of the thinking work. And I’ve done what you’re talking about – I have always liked experimenting with new tech so yes I have tried it. I’ve gone to Claude with a rough, vague, fuzzy idea that I can point to but not articulate, and it has put words to it. It has done it well, shockingly well. And I do believe this harmed me. For a moment, it felt good – wow! The words to say the thing I wanted to say! There they are! There’s that feeling of the “click.” But, i realized, it’s fake. What I think is actually happening in that moment is a kind of theft. The machete that hacks through the thickets of the messy forest of my conceptual tangle is actually incredibly valuable, and for the thought to be mine, it needs to be my machete. When AI puts words to my thought, it isn’t my thought anymore. Something has been stolen from me. It’s different from talking to another person, because there, there is actually another mind in conversation with me. If my friend helps me find the words for what I want to say, nothing has been stolen, and I am not harmed in this same way. Usually.
It also robs me of confidence. As if my son were trying to learn to write his letters and I took the crayon and did it for him. He’d see that it looks better than he could do, so why should he try next time.
So you have moved from a principled argument to a much weaker quantitative one that is easier to defend. Interesting.
I do not disagree that the more you outsource the less it is your own. But look at the OP, the threshold was 2% of an abstract. 10 words is the level that someone has not earned their doctorate?
But the deeper question is how much outsourcing is compatible with authorship? If the answer is Zero, then we need to consider someone like Heidegger and his tools being ready to hand and look and see if AI is just part of the academic process now, rather than wholesale claims like “If you use AI to write some or all of the text of your dissertation, you have not earned the PhD. Full stop.” being grounds to not earn a dissertation. 10 words is some yeah?
So perhaps, its not really a matter of whether assistance was used, or that grammatical errors were fixed, its a matter of whether that amounts to change of authorship.
“If you didn’t write it yourself, the credential is not yours.”, Sure we can beg the question, but why define sentence production as the test of a PHD? Why not intellectual ownership? Understanding? Responsibility for the research or idea production?
I dictated this reply using Wisper.ai and cleaned up the text using ChatGTP, do you reject my argument because its reasoning is unsound? Or do you reject it because software mediated it expression? Those are very different objections and the latter requires argument rather than assertion.
Michel seems to have a pattern of doing this in comments here. They will boldly put forward some implausible claims (e.g. — if you use AI to write some text in your dissertation, you haven’t earned your PhD), and then retreat to a much more banal thesis (e.g. — the more you outsource, the less the work is yours). Then they will claim they are “remaining firm” in their original judgment and pretend they never asserted the original claim.
If you say so.
But I won’t pretend I didn’t say that if you use a chatbot to generate chunks of text in your dissertation, then you haven’t earned the credential. You haven’t.
FWIW, I’ve consistently said the same thing in all the plagiarism scandals of recent years, too. Defences of ‘just’ a few paragraphs and appeals to dubious disciplinary norms about technical explanations don’t cut any mustard with me.
Ok, so let’s try this again.
Is it your view that if I take a few sentences I’ve written in my dissertation, feed them into an AI system and incorporate some stylistic edits verbatim (thereby using an AI to write some text in my dissertation), then I have not earned my PhD?
An affirmative answer is implied by your previous comment, but I’ll be surprised if you don’t disavow it. (Partly because there’s no significant difference between this and incorporating verbatim some stylistic edits from a human you’ve sent a paper to, and that is clearly not so egregious an offense that it undermines the credibility of your entire PhD.)
I… do not recognize this characterization of me. I have not, in fact, adopted a quantitative position. I have, however, tried to keep the issue of checking spelling separate from that of text generation, which you keep running together. I think everyone is happy to grant that if you wrote ‘phenomena’ for the singular ‘phenomenon’ and the chatbot catches it, that’s fine. But, as I said above, I am also deeply suspicious that that’s all that’s going on when people cite spelling and grammar as legitimate uses. Because I have worked as a copy editor, I know that copy editing can very easily slide into full-on editing. And because of my experience in that field, I can also fully believe that users may not really notice what’s going on, or conceptualize it that way.
I ‘reject’ your argument, such as it is, because I think it is wrong, and transparently so. That your replies keep directing us further afield does not inspire much confidence in me that continuing the exchange is a good use of my time. I appreciate that you’ve been upfront about your reliance on ChatGPT. That does not lead me to ‘reject’ it on those grounds. It does, however, make me very disinclined to continue what appears to be an increasingly one-sided conversation. I have enough unasked-for chatbot bullshit in my life, I’m not particularly interested in wading through more.
The question I have repeatedly asked is where authorship changes hands, and I still do not think you have answered it.
Your experience as a copy editor may give you good reason to suspect that editing can become substantive rewriting. I have no objection to that point. What I object to is treating that suspicion as sufficient grounds for concluding that AI assistance necessarily undermines authorship.
If ten generated words in an abstract are enough to invalidate a doctorate, then say so. If not, then some amount of AI assistance is compatible with authorship, and the real question becomes where the line is drawn.
That is a hill I will die on, so to speak. If you use it to write some or all of the text of “your” dissertation, then you have not earned the PhD.
Its not really a hill you are prepared to die on, because you have clearly retreated from this position without the grace to acknowledge it. This original claim was categorical, and rather than defend it, your new position relies on your subjective experience as a copy editor. That is no longer a categorical objection to AI use, its a quantitative one of where the line is drawn between editing and authorship and that is the question i have been posing to you from the beginning.
This is becoming quite a good performative demonstration of the problems with permitting AI writing.
You appear to be desperately goading Michel into a line drawing trap, something resembling a sorites paradox.
Suppose Michel responds to you that n number of AI generated words are needed to change authorship. You will reply, “A-ha! Why not n+1?” There’s likely as good a reason to draw it at either n or n+1, therefore you will think you defeated Michel with a ‘gotcha.’
Any academic philosopher will see your trap from a mile away, yet you’re puzzled why Michel wouldn’t walk into it. Are you really a philosopher if you thought that ploy would work here?
No, it’s not an N+1 game. I’m not arguing that no line can be drawn. I’m trying to get Michel to defend the position he actually holds, but appears unwilling to argue for.
He began with a categorical claim: “If you use AI to write some or all of the text of your dissertation, then you have not earned the PhD.”
Since then, he has retreated into concerns about copy-editing, substantive revisions, and the possibility that authors may be outsourcing more than they realise. Those are concerns about where the line is drawn, not arguments for the original categorical position.
If there is a line, defend it. If there is no line and any AI-generated text invalidates authorship, defend that instead.
And if the line is truly zero, then its no longer merely defending a theory of authorship, its defending a theory of authorship whose consequences disproportionately burden disabled people who rely on speech-to-text, dictation, grammar correction, and other assistive technologies. That may be the correct position, but it is a consequence that requires argument, not silence.
“If there is a line, defend it. If there is no line and any AI-generated text invalidates authorship, defend that instead.”
Have you heard of a false dilemma or a sorites paradox? Your retort is precisely the next obvious move in setting up the trap.
The alternative, if you’re not trying to entrap Michel, is you’re making the basic mistake of assuming a bright principled line can be drawn. But authorship appears to be a hazy concept like a heap of sand, a bald head and many other things. It’s not always possible to draw clear and bright lines.
I don’t see that Michel has said such a line exists or that “any” AI-generated text invalidates authorship but in fact disavowed those positions.
How about if Claude dictates a list of interconnected ideas, I write down what I gather, and write a dissertation, all by myself, based on those notes: Am I deserving of my academic title?
Substitute “Claude” for “your dissertation director”, and not a few PhDs have a form not too different from this. (Which again is my point about implementation being what matters. If your advisor suggests an idea and you develop it, it’s still your PhD.)
Maybe. The problem is what’s going unsaid; presumably you then have to find and read the sources for those ideas, and explain them yourself. If so, then it’s probably fine. If you’re relying on Claude’s ‘notes’ about them, then your dissertation is garbage.
But also, if you need Claude to do that preliminary work for you (depending on the details), I have no confidence that (1) that’s all you are using/will use it for, and (2) you are ready or perhaps even able to write a dissertation ‘all by yourself’.
I am happy with David’s analogy above/below (wherever it nested). But I would raise an eyebrow if I found out a supervisor had, say, given a student their topic and a list of interconnected items to address. I’d frown if I discovered the supervisor had given their student an outline of the dissertation. And I would fail the dissertation if I discovered the supervisor had provided ‘notes’ on its sources.
A lot hangs on just what you’re outsourcing. And at this point, I just don’t take most people at their word about what they ‘just’ used it to do, because comments here and elsewhere (as well as personal experience) strongly suggest a large number of people simply do not have a very clear idea of what is and is not acceptable.
We’ve had discussions here in the past, for example, about acknowledging RA work, and I’ve found those absolutely appalling; this has not been helped by discussion of plagiarism scandals, where it’s clear that many posters have no idea how to paraphrase properly in the first place, or have atrocious citation practices. These are all different issues, but I think they all come together under the heading of outsourcing work to a chatbot.
“When I stand in front of an evaluation panel,what ultimately matters is not what produced the text on the page. What matters is whether I understand the arguments, can defend them under scrutiny, and can demonstrate genuine mastery of the research.”
That can’t be the criterion. I understand the general theory of relativity; that doesn’t mean I wrote it. Ask Claude to come up with better arguments to insert into your text.
A dissertation is not about demonstrating that I understand existing knowledge. It is about proposing, researching, producing, and defending new knowledge.
So perhaps it is not that Claude failed to provide me with a suitable argument. Perhaps it is that you forgot to Ask Jeeves before replying and started from a premise that does not fit the case at hand.
So my original point still stands. If I can defend the research, the data, the conclusions, and the arguments as my own, then it is difficult to see why 2% AI-generated text in an abstract invalidates the doctorate.
I’m curious what you think of services like Lex Academic, which many academics (not me) have been using since before Gen AI, which pride themselves on being purely human and yet, in my view, offer exactly the same sort of writing outsourcing that you find disqualifying. Would you say that someone who has used their services for their dissertation or a journal article hasn’t earned their PhD or publication?
https://www.lexacademic.com/services/substantive-editing/
Most of their services do indeed go way too far in my estimation, yes. The rest might or might not be okay, depending on how they play out, but my guess is not. Ironically, their copy is poorly written and features a number of errors and infelicities. Shrug.
I don’t think authors who use such services are necessarily setting out to undermine their own work. But the people who most want or need it are probably not best-placed to notice or understand quite how much it ends up altering their work. And the sea of red can be quite overwhelming, and can easily deflate what confidence one has in one’s own judgement, making ‘accept all changes’ seem more attractive.
Me too, and I am in STEM. Nowadays, personally, I always put a disclaimer to all my papers saying that no generative AI was used anywhere in writing. No matter how well or bad I write, at least it is still my voice.
For a long while now, dissertation defenses have been largely pro-forma affairs. At the University of Michigan when I defended, the form the committee was given after my defense did not include a “fail” option.
I think that one option that needs to be seriously considered is making oral defense a far more onerous obstacle to earning the PhD. In a world where students can have a machine produce a dissertation for them, their ability to adequately defend the thesis in person, without those tools at their disposal, needs to be much more carefully evaluated.
If a system of oral defense is going to work in our current system, it will have to be implemented as a more regular part of the supervision of a PhD, since most students are going on the job market and getting letters of recommendation before they’ve defended. Right now, those recommendations are largely based on written work. We would need to reorchestrate things so that the recommendation letters can attest to the candidate’s mastery without AI tools.
Oral defense as a regular part of the process is, sort of, already a requirement at my university.
During the dissertation phase, PhD students have to complete a certain number of progress report meetings with the supervisor and at least two jury members. First, students send the jury members some work (typically an outline and a chapter or two). Then, in the meeting, the student responds to questions from the jury. Finally, the jury votes on whether to pass the student; it is possible to fail one of these meetings.
AI aside, I think these mini-defenses are a good way to keep students on track and also ensure that students get feedback from people other than the supervisor. Perhaps they have an even greater benefit in the age of AI.
While its a lot more work for lecturers, viva voce is becoming a lot more prominent in undergrad and masters assessments at the uni I attend. Making a phd defense more onerous seems like a natural progression in light of AI.
MIT had, and still has for all I know, a system for this that I think was right.
There was a private oral defense that was really substantive. People could fail it. It was private so other students wouldn’t know it was happening, and people could fail without it becoming public knowledge. If they did, they went back, worked harder, and did it again.
Then there was the public defense like you had at UM that was basically ceremonial, family could come along to, etc.
I thought that was a good idea pre-AI, and I think it’s an even better idea now.
That sounds like an excellent approach for oral defenses.
But I wonder: if oral defenses start becoming more rigorous and less pro forma, then that becomes a critical gate for certifying new doctorates. Maybe it used to be a major hurdle in earlier times, but now we’re sensitive to things like fairness, accessibility, justice, etc.
Could stricter oral defenses become a filter that prevents some neurodivergent scholars from passing the requirement?
Many folks are more scared of public speaking than death. Maybe we want good public speakers if we assume these PhDs will also be lecturing, but we can’t assume that.
Some new PhDs may go to research-only positions, incl. outside academia, and others may teach online. Anyway, lecturing in front of a live audience isn’t the same as being interrogated by honorific people who have the power to end your career on the spot.
I understand we need new ways to assess academic mastery. Maybe there isn’t a good way that doesn’t have important tradeoffs, but it’s important to recognize what the tradeoffs are in the first place…
Substantive oral defenses in philosophy have been common all over the world, and I see no evidence that there are fewer ‘neurodivergent’ philosophers in e.g. the UK or Italy than the US.
Let me step back from the specifics of various sub-threads and make a couple of broader (and related) points:
1) I am really intrigued at how many people in this and related threads seem viscerally opposed to the idea that reliable AI detection is feasible. The evidence that Pangram, specifically, can do this is just very good. I was quite surprised by this and my prior was very skeptical, but I spent some time reading the third-party research, searching for supposed false positives found by others, and doing my own explorations, and I think the evidence is compelling that purely human-written long-form text pretty much never shows up as AI-written on Pangram, and that AI-written and AI-edited text generally does. One can debate how well it distinguishes “good” from “bad” use of AI, how well it foils attempts to fool it by humanization, and whether it will still work well next year (I’m optimistic in all cases but the evidence is weaker). But right now, it just works as a tool to determine whether or not a long-form academic document is entirely human-written.
2) The very fact that such a tool exists, potentially makes a difference to what our policies should be towards (e.g.) AI use in dissertations. Most people in this thread seem to agree that it would be bad if you just used AI to write large chunks of text (e.g., “Give a three-paragraph explanation of Wallace’s case for the Many-Worlds theory, and then a two-paragraph explanation of how it should be rejected via the Incredulous Stare”). Let’s stipulate that there are some much more minor uses of AI (to suggest a pithy phrase, rewrite an awkward paragraph, turn clunky prose into something smoother, whatever) which are themselves harmless or even desirable, but that Pangram can’t distinguish from the bad uses of AI. Depending on your relative assessment of how bad the bad uses are and how actively good the good uses are, you might think that we should still ban the good uses, simply because we then have a clear and transparently enforceable line. (And we might revisit that as and when the tools change: for instance, Pangram’s most recent version claims to distinguish AI writing from light AI editing, and as and when that’s verified as reliable, the options for enforceable lines would broaden.)
Please don’t count me in your 1) — I’m hoping beyond hope that Pangram works as advertised so that we can rely on it as we determine what the acceptable boundaries of AI use are. My skepticism, better expressed than I did upthread, is about whether 2% is a meaningful threshold practically, and that it might have been chosen because at a 5% threshold, there would not have been anything interesting to report.
(Abstracts are also short, and until recently it was conventional wisdom that Pangram struggles with shorter chunks of text — although the fact that it seems to be reliable pre-2022 suggests that at least we’re comparing apples to apples.)
No, I didn’t mean to include you; I thought your comments were sensible.
Students whose work is tagged by Pangram will often give the following atrocious argument after I point out that the research showing it’s quite accurate and, on the very rare occasions when it does err more prone to false negatives than false positives: But it’s still *possible* that this report is wrong! And a 1/10,00 error rate is far too high for something as important as a discussion post in a philosophy class! And I always think: If one in 1/10,000 chance of innocence is too high to give you a 0, then we need to let every single prisoner in the U.S. out tomorrow and abolish criminal punishment going forward. I understand why they serve up that argument. Why any educated person would take it seriously, much less offer it themselves is beyond me.
I too share the worries about the future in this arms race, but one good move here is for the U.S. to do what the EU is moving towards and require “watermarking” of some sort in AI text.
we should indeed abolish criminal punishment but that’s another can of worms
But as you hint, isn’t there just a simple argument against long-term reliance on any such tools as Pangram? In the long run, someone will surely reverse-engineer any AI detector and build camoflauge into their GenAI model. Moreover, this camoflauge will presumably involve making the model generate more humanlike text, increasing the risk of false positives that can ruin a student’s career. Do we really want to hitch student assesment to a bet/hunch that this arms race will almost always work out in favor of the detectors? This seems like an in-principle argument, not one that is contingent on the effctiveness of any particular detector.
I never said anything about long term policy. At present, AI detection works, and we can make policy based on that. As and when the technological situation changes, we can rethink the policy. This whole problem is only three years old and no-one can reliably predict where the tech will be in the future,
(As an amusing case in point, quite a lot of people seem to have assumed that reliable AI detection would *never* be possible, and so felt free to use AI to write their work without disclosing it. This is now having embarrassing consequences for them.)
Not so worried – it’s also in Pangram’s interest to continue the arms race. Their AI can fight the chatbots, and when it’s not useful, we won’t rely on it any more than any of us take Copyleaks or Turnitin seriously.
I think the caveats you include at the end of your (1) are in some tension with (2). I use Pangram and I think it’s quite good; it’s almost always right on the cases where I know whether the prose was AI-generated or not, and in cases where I don’t know for sure–student work–it agrees with my intuitions.
But I’m very hesitant to build it into my policies in any official way. A colleague of mine has a student who submitted work that triggered pangram a few times and got warnings, and recently submitted that my colleague is confident is AI-written–I saw it and I agree–but which is flagged as 100% human by pangram. My colleague assumes that the student has found some way around pangram. I tend to agree.
I have also tested a “humanizer” that in my experience very reliably beats pangram. In my judgment makes the prose a lot *worse*, so I suspect it’s not all that tempting a cheating tool, but the fact that it can be done makes me worry that maybe there are already better humanizers on the market I haven’t tested, or if that’s not already true that one is just around the corner.
So even while I’m impressed by pangram, I’d rather not build policies around the assumption that it works. (David Deming was defending a line like this on a recent podcast appearance, and I thought he made a good case.)
I’m not sure we’re in serious disagreement. The really crucial thing about an AI detector is that it doesn’t show false positives, so that if it says “AI” we can believe it. The price for that may be relatively high false negatives, and we just need to build into our framework that a Pangram negative is not fully reliable proof of no AI. It doesn’t fully obviate other shifts we might make to (e.g.) teaching protocols.
That said, there has always been cheating. Creating friction around cheating is good even if it doesn’t catch the really committed cheaters.
This is a bit off topic from the post but I’m curious to hear what people think of the following scenarios. Would you say the AI use described is acceptable or not? These are all scenarios I imagine that are beyond conventional editing.
I ask because at my university, students are not penalized for using AI to write content. Our code administrator has decided AI is a “tool” like using an editor. I also have colleagues who allow AI in their classes for some of the uses described below. Finally, many data analysis apps (which had already transformed the speed and capacity for quantitative and qualitative data analysis) are increasingly incorporating AI elements (e.g. Nvivo). I think graduate students are going to be using more and more AI. Thus, as AI tools get more ingrained in research and teaching, prohibitions may impact emerging scholars more than earlier generations.
Peer review: reviewing an article and writing up draft notes on it, but then putting those notes in an AI tool and asking it to clear up the prose and ensure it is logically structures (note: the article being reviewed is not supplied to the AI tool).
Reviewing archive material, taking rough notes and transcribing direct quotes, then asking an AI tool to clean up the notes and organize the quotes according to a set of themes provided.
Writing up selected case studies and then providing them to an AI tool and have it suggest an outline for an article, which the author then tweaks and writes the article themselves.
Writing rough notes for a class lectures based on assigned readings and then providing it to an AI tool and asking it organize the material in a way that will be best for teaching the content.
Providing parts of an article in progress to an AI tool and asking it to identify problems with structure or logical flow, and then using those suggestions to make changes to the article but without having AI draft the text itself.
Writing the first draft of a book review and asking AI to make sure the tone is collegial.
Taking lectures slides that were fully human created and using an AI tool to make them more visually compelling and asking it how to present the material in a way that will be easiest for students to understand.
The second to last item is problematic, depending on what suggestions are made and how they are implemented. The other items are either clear academic misconduct or acceptable for other reasons (the last, for example, would merely be poor behaviour if it were disclosed to the students, but much worse if not).
If that’s your university’s position then I suggest you use exams only.
Unacceptable*
I’m confused. Why is it poor behavior to use AI to improve your slides, especially if, by hypothesis, it makes the content and its presentation better for students themselves?
Because the alternative (working out yourself how to explain material well to students) is part of your job.
No. My job is to teach effectively. If a tool, whether it be AI supported or not, can help me teach more effectively without diminishing my competence then I have a strong reason to use it. It frees up time I can use for more meaningful things. We’re not talking about asking AI to create lectures for you on material you don’t know. My job description does not include painstaking hours manually redesigning PowerPoint slides.
The examples are all things I’ve recently heard colleagues mention. Institutions are increasingly providing integrated AI tools for faculty to use in their work (e.g. a colleague mentioned using an employer-provided PowerPoint application with co-pilot integrated to improve their lecture slides). I had a meeting with a textbook publisher who explained they’ve integrated AI tools into their platform so faculty can use AI to generate exam questions based on material they choose (fully editable, of course) and assist with grading (the faculty member grades the first 20 assignments or so and then the AI tool can continue based on being trained by the professor).
I see an increasing integration of AI into academic workflows. It presents challenges, one of which is increases in submissions to journals and conferences. Many of these submissions are just bad work that can be desk rejected. I am a journal editor and see these myself. In my opinion, someone who doesn’t know how to do research can’t generate a good article with the help of AI, though that could change. But some of the increase will also be the consequence of people being able to work faster using AI tools. I’m not sure how journals, etc. will handle this or what the future of academic publishing will look like. However, I wonder if the argument that using AI is inherently academic misconduct and that everyone should choose not to use a tool integrated into qualitative data analysis software, employer-provided apps, textbook platforms, etc. might not be meeting the moment.
Why would you care about ‘meeting the moment’ – I simply don’t accept that we should accept the intrusion of computer generated text into our field simply because the university leadership class is short sighted.
Some of “tenured realist”’s examples involved computer generation of text, but not all do.
Personally, I’m interesting to responding to changes to how academic work is being done and the overall norms of my professional community. As an individual, we are each free not to use AI tools, but describing the use of them in general as academic misconduct is making a broader normative claim.
None of these seem problematic to me in themselves, for what it’s worth.
Interesting work, but I do think there may be a step between using AI in abstracts and more substantively. Surely the first indicates a greater chance of willingness for the next, but abstracts can be annoying and particular (150w here, 250w there, only 75w some places, etc.) and it’s easy to have AI whip one up from either the paper or even a prior abstract. I myself have used AI to whip up an abstract for journal submission purposes while the paper is entirely human written. (I will typically rewrite them for accepted manuscripts and always disclose this specific practice of mine if applicable during submission.)
They explore this question: they evaluate 339 dissertations in toto to see how predictive AI in the abstract is of AI in the main text. From the paper:
“a dissertation with an AI-generated abstract is 55 percentage points (p.p.) more likely to have AI detected in the full text (quadrupling the base rate), has 11%more AI content (nearly quintupling thebaserate),and is 20 p.p. more likely to have at least 20%of its text be AI-written (6xthebase rate). Similar patterns emerge when evaluated at higher levels of aggregation.”