Editor of Ethics Floats AI-Use Guidelines
In a recent editorial, Douglas Portmore (Notre Dame), the editor-in-chief of Ethics, sets forth some “initial guidelines” about AI use for the journal.
He says his remarks “represent only my own opinions, not necessarily those of the associate editors. But I’m hoping to have some official policies and guidelines posted on the journal’s website sometime early next year.”
Here is what he says about the matter so far:
First, AI tools cannot author or even coauthor a submission. Authors and coauthors must be able to take both moral and legal responsibility for their submission. As nonmoral entities, AI tools cannot, as authors must, take responsibility for the accuracy and originality of their products. It is not enough, then, for authors to take responsibility for employing AI tools; they must also take responsibility for the assertions that they make, the arguments that they give, and the sources that they acknowledge or fail to acknowledge. And, as nonlegal entities, AI tools cannot make any legal declarations, nor can they sign off on the necessary copyright agreements. Thus, AI tools don’t meet the requirements for authorship.
Second, if an author does use any AI tools, they must be transparent about this, disclosing which AI tools were used and how they were used. So, if an author uses an AI tool to, say, generate possible objections to their view or rebuttals to those objections, they must cite the AI tool. The point is that authors shouldn’t take credit for what’s not their own work. What’s more, if an author’s submission borrows some text generated by an AI, that text needs to appear in quotes just as it would if it had been borrowed from a person. And even if an author uses an AI tool simply to edit their writing, they should be transparent about this. I would suggest including something like the following in the note containing their various acknowledgments: “The text of this article has been edited for clarity by ChatGPT, OpenAI, April 16, 2025, https://chat.openai.com/chat.” Thus, you should acknowledge the assistance of an AI tool in editing your writing just as you would the assistance of a copyeditor. And, given that Ethics is published by the University of Chicago Press, these citations should follow the guidelines set out in The Chicago Manual of Style.
Third, it’s important not only to avoid taking credit for work that is not your own but also to give credit where credit is due. The problem, then, with just citing an AI tool as the source of, say, some objection is that the AI tool is unlikely to be the original source of that objection. The source of that objection is more likely some author whose work has been used to train the AI. Thus, attributing an objection (or some other substantive point) to ChatGPT is just as problematic as attributing an objection to your colleague when all they did was tell you about some objection that, say, Rawls raised. So, ideally, you would track down the original source of the objection and cite that, while acknowledging the help of ChatGPT in bringing it to your attention.
Fourth, it should go without saying that editors and reviewers should not use AI tools to help generate their reports on, or assessments of, submissions. Doing so would violate the rights of authors and breach their own professional responsibilities. Consider that files uploaded to an AI tool such as ChatGPT are retained indefinitely by that service and are then used in training their models. So, even uploading a submitted manuscript to an AI tool violates the confidentiality of the peer review process. Besides, editors and reviewers have a professional responsibility to come up with their own independent assessments of submissions.
The full editorial is here.
The second paragraph seems like it requires too much of authors for me. My basic approach is this.
Assume the paper is coming from a grad student or a recent grad (like most of them are). They will have got a lot of feedback from their advisor, and some from other faculty. Those folks should be thanked. But how much more than a general thanks is needed? Do they have to say precisely what sections the advisor was particularly helpful on. If the advisor suggested a rewording somewhere, do they have to put those words in quotes, and attribute them to “Advisory Meeting, September 29, 2025.” or whatever? I don’t know, this doesn’t sound like an improvement to me.
To get the second paragraph here we need to believe either.
Neither of these claims is absurd; they both seem like reasonable things to believe. But I disagree with both, so I don’t think authors need be nearly this precise in what we cite.
Well, there does seem to be a rather stark difference between (i) receiving feedback from an advisor, and (ii) having (say) Chat GPT generate a rather large chunk of written text (e.g., using “an AI tool to, say, generate possible objections to their view or rebuttals to those objections”). The former requires interpretation and integration, in addition to the work implicit in the task of both writing and editing. The latter requires a bit of curatorial work, but doesn’t require generating new text or editing that written thoroughly. When (ii) happens, then authors should (I think) “articulate precisely what contributions [Chatbots] make”. After all, this is less akin to receiving feedback from an advisor, and more akin to quoting a written text verbatim.
Sure if they are quoting whole paragraphs. (Which they shouldn’t be doing – cutting and pasting from the chatbot rather than doing at least some editing is rarely a good idea.) But it looks to me like “if an author’s submission borrows some text generated by an AI,” implies that even a suggestion for how to rephrase a sentence should be in quotes.
It just feels in general like we should be treating these tools more continuously with the tools we’ve had for decades. I don’t think people should be noting that they used the Microsoft Word spell checker to edit their work, or including quotation marks and citations when a friend points out a misspelling. I think there are uses of LLMs that differ in degree not kind from using spell checkers and friends for minor editing.
I do think that, when a specific objection or other bit is suggested by someone, it’s at best bad form to merely generally thank them, as opposed to putting in a footnote that says something like, “I owe this possible objection to X”, or at least, “Thanks to X for suggesting this possible objection.” In some cases it might go beyond bad form. Now, sometimes I get general comments or discussion, and they are useful, but are not that specific, or (especially when presented at a workshop with an audience) I may not remember who, specifically, suggested something, despite my trying to do so. Then a genreal thanks seems like enough. But, I don’t think that’s what would happen with a chatbot, so I think you’d need at least something like is suggested here. (I think this is largely in line with Kimberly, though with a slightly different emphasis.)
I had the same thought! My committee members quite often suggest small in-text line edits on drafts, and I often accept these suggestions. None of them would ever expect to be credited for such small changes! I don’t see how accepting suggestions from ChatGPT would be any different, except that the ChatGPT suggestions are often worse in quality and so I accept them far less often.
I understand the worry about copying and pasting sentences that were completely generated by ChatGPT, but copying and pasting a sentence that is 98% mine with two slight grammatical/terminological changes seems trivial enough to not require special disclosure.
When I was a student, my advisors seriously declined any acknowledgment in my papers beyond a general thanks. Not sure if this is right or common, but it’s what it is.
I think, to the contrary, Paragraph 2 does not go far enough. It suggests, first, that it would be fine for objections to one’s thesis, and rebuttals to them, to be generated by AI as long as it’s acknowledged. A rebuttal to an objection, though, is ordinarily supposed to be the author’s defense of her own work – an act of substantive philosophical argumentation a little too close to coming up with the thesis in the first place. Directing AI to answer objections it generated in effect asks AI to improve the thesis, altering it so it’s more defensible, which is precisely what we expect of the author.
Re. acknowledgments, I think the point of the guideline is not to invite the onerous practice of sentence-by-sentence acknowledgments, but to suggest that it is not acceptable to present the ideas and sentences of others — including AI — as one’s own. At present, many of us infer that an uncredited sentence or argument is the author’s and nobody else’s. The guideline in effect admonishes authors not to abuse this presumption by taking from AI uncredited. Insisting on crediting is just a roundabout way of putting this admonishment.
True, PhD advisors sometimes help students with wording, even adding whole phrases and sentences. But that’s often because it’s hard to say no to an advisor once they propose a rewrite, and some are overbearingly hands-on that way (I’m probably a little too far in that direction myself). Refusing the input of advisors who pass on your dissertation can be unwise. But I don’t think taking their sentences or phrases is ever done with the thought that it’s fine. You’ll never see an author acknowledge these borrowed sentences, even though they acknowledge plenty of objections and examples they took from others. That’s probably because they know it’s a little below board. It’s certainly no model for what we should permit from AI.
I suggest ways to rephrase points, suggested wordings, etc, to my students all the time. I take it as part of my job.
I could be very mistaken, so I hope someone with more technical expertise can speak to this. I believe the assertion that submitted materials are used for training LLMs may be incorrect. My understanding is that submitted texts might be used to refine a particular user’s experience, they aren’t used as part of the training dataset.
It depends on the LLM. Many of them do use your data for training. Anthropic for instance recently changed its policy to allow much more data to be used for training. And Anthropic is one of the most restrictive with respect to using data for training. Plus, if a company says it is or isn’t doing something, you may not necessarily want to trust it.
While I certainly agree with the final quoted sentence, I’m not convinced that “even uploading a submitted manuscript to an AI tool violates the confidentiality of the peer review process.” (This matters since some referees might legitimately find it helpful to get an AI summary of a paper before reading it carefully themselves. Others might find a back-and-forth chat helpful to check their understanding of the paper, ensure that their objections aren’t uncharitable or trivial to address from the author’s perspective, etc. Frankly, I’d love for my referees to do more of this sort of sanity-checking before submitting their final reports, so I hope that editors don’t discourage it!)
Firstly, it isn’t true that AI tools necessarily train on user data. You can opt out of generally permitting it in your privacy settings on both Claude and ChatGPT (or tick the ‘incognito’/’temporary’ chat option on a particular occasion).
Secondly, I’m not convinced that training on our data is a “confidentiality violation” in any sense worth caring about. I take it that confidentiality in peer review serves the goals of (i) protecting authors against being “scooped” by others before their work is published, and (ii) protecting authors against reputational damage (or something vaguely in that vicinity?) from sharing an early/unpolished version of their work. But neither of these goals seem threatened by AI data training, given what a tiny and diffuse effect each particular document used in training has on the end result. (It’s not like it’s going to reproduce the essay in someone’s chat the next day.)
Am I missing something?
I think the concern is much more pressing in some fields than others. I could definitely imagine that OpenAI or Anthropic or Google might want an advance peek at the newest thoughts of Geoff Hinton or Dave Chalmers or Alison Gopnik. Given the history of companies like Uber using their access to track the movement of journalists who have been negative about them, I think it makes sense for someone who has reason to believe that OpenAI might want to scoop some of their work would want to ensure that their work does not get shared with them early.
Of course, this is likely to be irrelevant for well over 90% of philosophy papers. But I think it’s definitely a non-zero number where it could be relevant.
Is the worry that the company may have some kind of algorithmic screening of content on their servers that they might then direct their employees to snoop upon? If so, do you think this is relevantly different from, say, storing a paper critical of Google on Google Drive?
The main difference here is that Google Drive is mainly used by authors of papers, while these AI tools might naturally be used by reviewers. An author who is critical of Google might choose whether or not they think it is a good idea to store their drafts on Google Drive, but probably isn’t worried that a referee will download the paper, upload it to Google Drive, and then review it. But an author who is proposing a new benchmark for AI consciousness or an article about applied AI ethics, might well be worried that a reviewer might upload the paper to a system that then recognizes it and flags it as from that author.
(I recall in January 2023 that a prominent cognitive scientist I know was very annoyed that he couldn’t try out experiments on ChatGPT’s abilities without OpenAI recognizing that these experiments were coming from his account, and thus were abilities that they might specifically focus their training on for the next generation of model.)
There’s a nearby issue of consent, though. When I share work in progress with a peer, I trust that they won’t share it with others — whether fellow meatbags like myself or LLMs — without my consent. This is almost always implicit, of course, but I thought this was widely accepted. Like another commenter here, I am surprised to see the “flexibility” in the attitude towards this norm (and others) that some here are taking. Perhaps I and those who agree with this norm should make this more explicit when we share our work.
Supposing that this is a reasonable norm and it ought to be respected, there’s of course a further question about how general it is. Does it apply to reviewers assessing work submitted for publication? Does it apply to preprints and drafts made available on someone’s website? I would assume yes in both cases, but I can perhaps see a case being made for treating works of these kinds in these contexts differently than early drafts being circulated privately for feedback. Pretheoretically, though, I would expect reviewers and others to respect this norm.
Isn’t the question of consent downstream of whether there’s a legitimate interest to protect?
For example, even if an author is an anti-Microsoft ideologue, I don’t need their consent to us MS Word to read their paper, because they have no legitimate interest that calls for granting them such veto powers over what software I use to read their work. Their demand would be simply unreasonable.
Now imagine them arguing that we already have an extant norm that work wouldn’t be “shared with others—including MS Word—without consent”. Surely that’s an egregious misinterpretation of the extant norm. MS Word isn’t a person, and neither are LLMs, so neither is obviously covered by extant norms regarding “sharing with others”. They need to propose (and argue for) a new norm if they want to rule that out.
More generally: some argument is needed before demanding veto powers over what computer tools others are allowed to use while interacting with one’s work. There’s no presumption of such control.
First, the burden of proof is on your side too. What’s the argument that reviewers can do what they want with a manuscript? Reviewers are asked a specific task which doesn’t include providing an author’s work to an LLM for training purposes.
Second, this is not a question of ideology but of copyright. It cuts to the core of intellectual property: https://www.nytimes.com/2025/09/05/technology/anthropic-settlement-copyright-ai.html?smid=nytcore-android-share
I don’t transfer my copyright to the reviewers of a paper of mine!
I’m puzzled by the suggestion that how we value and conceive of intellectual property is not an ideological question. fwiw, I’ve written more about the philosophy underlying intellectual property, and how it should affect our thoughts about AI training, here.
Again, focusing on what you perceive as weakest link in the argument instead of answering the tough question. I responded to your post where you use ideological as someone rejecting the use of Microsoft. You think that such an ideology is of the same kind as the set of beliefs involved in grounding the legal definition of intellectual property? Your reply rests on this identification.
I’m not sure what you think “the tough question” is. The adjectives used in my thought experiment aren’t particularly important. The point was just to illustrate the general principle that issues of “consent” (or demands for veto powers in the absence of such consent being granted) are normatively downstream of legitimate interests.
To replay the dialectic:
* I set out what legitimate goals or interests I perceive as underlying “confidentiality” in peer review, and suggested that neither seems threatened by LLM training.
* Will suggested that concerns about “consent” might have independent force.
* I countered that consent only seems to have normative force when it is needed to protect some legitimate interest (as the MS Word case illustrates), so we’re back to needing to identify some legitimate interest that is threatened by LLM training on one’s work.
I don’t really see your comments as directly engaging with this dialectic. You asked, “What’s the argument that reviewers can do what they want with a manuscript?” The obvious answer is that there’s always a presumption of freedom: acts are permissible unless there’s some reason why not.
You then wrote:
> “Reviewers are asked a specific task which doesn’t include providing an author’s work to an LLM for training purposes.”
This is question-begging. The specific task is reviewing the paper. If engaging with an LLM is an effective means to completing this task more successfully, then obviously the task could include using such a tool. (The task doesn’t specify using MS-Word either, but using MS-Word may still be a fine way to complete the task.) Then there’s a further question of whether the possible side-effect of entering the training pool would qualify as sufficiently problematic that it would outweigh the reasons to use a useful tool. Just asserting that the task specification “doesn’t include” the side-effect doesn’t settle this normative question.
Finally, you appealed to copyright and intellectual property, which seemed the most substantive point, hence why I responded with a link where I discuss those issues at greater length.
Copyright/IP fetishism was bad when it was publishers trying to take down LibGen and SciHub, and IMO it’s still bad today.
Thanks for the explication. I see that you have thought a lot about this and appreciate the level of sophistication in your arguments. I nevertheless think that your thought experiment introduces a suggestive and misleading analogy. My point about the task is that we should define as precise as possible what we expect from reviewers and then, as a scholarly community, decide whether AI is a tool that we need to apply. I thus argue for the application of the precautionary principle which, imo, is the best guide for introducing new technologies (recent empirical studies on the psychological impact of social media show that we should have been more cautious introducing them, for instance; as I will argue later context matters). Arguing with hypotheticals (AI could be useful as a tool) is simply feeding the hype. This would be my take: when you are asked to review a paper, the editors expect you to evaluate its merits. They ask you because of your specific and distinctive expertise. To fulfill this task you must read the paper. Hence, you will need some kind of technology to open the document etc (and you are right, an anti-Microsoft ideology of the author is not sufficient to restrict the author in the choice of their software product). The question is whether the available AI tools help you in fulfilling this specific task which rests on providing your distinctive expertise. LLMs are good in finding general patterns. Hence, as you note, you can get a good summary—but this is something the abstract should give you too. I thus do not think that the summarizing power of LLMs is sufficient to override the precautionary principle. But, maybe, AI enthusiasts can do much better than that and demonstrate the usefulness of AI tools for reviewing philosophical papers. This is what I meant with burden of proof.
I see that you are critic of intellectual property rights. Again, an analogy is at the core of your argument. You believe that „“AI is theft” is the modern-day version of “taxation is theft”“. I completely agree that „redistributive taxation that successfully serves the greater good is not any kind of “theft” worth worrying about“. But the concept of taxation contains assumptions which are as important to its legitimacy as its moral value: we assume that legitimate governments collect taxes and, today, political legitimacy includes some kind of democratic control. But I can think about cases where the missing legitimacy undermines the moral value. I could expand on this but I think you get what I mean.
I do not think that the current reality of AI meets any criteria of legitimacy we assume in the case of taxation. A few private companies have introduced tools from which there are making enormous of money—not even by the products but the assumption of future productivity. There is not even basic regulation happening—not to speak of democratic control whatsoever. The digital landscape of AI remembles the Gold Rush of the Wild West and its diggers are grappling all available tools to get more for themselves. So there are two questions AI supporters must answer before we can override the precautionary principle:
—Can the introduction of AI serve the greater good in principle (as redistributive taxation does)? The development of social media should remind us that there is no clear-cut answer to this question.
–Do we use AI in an appropriate context that it can serve the greater good? I think we are far away from such a legitimate framework.
I haven’t used Microsoft Word in a while, but I would assume that it generally keeps data local, and doesn’t send that information to a a centralized database.
That’s the legitimate worry I see here, even if it’s only for a small amount of work that it would matter.
One possibly relevant thing not raised so far is that we don’t know exactly how copyright might interact with submission to an LLM, or – perhaps more importantly? – how in future publishers might de facto treat questions of copyright in this respect. One can imagine a publisher treating work pasted in an LLM as having already been put in the public domain, and that might affect their decision to publish it. You might be relaxed about this possibility with your own work, but it might suggest caution about submitting others’. Corrections are very welcome on this from those who know more.
How could they possibly know? We could similarly imagine a (completely insane) publisher treating work that’s privately backed up to the cloud as “having already been put in the public domain”, but this would be unenforceable because entirely undetectable. So there seems no chance that this could affect publication decisions.
Whether it’s detectable presumably depends on what rights the relevant companies have over any data you send them, and what they choose to do with that data in future (perhaps next year I’ll be able to ask “send me any early drafts Richard Chappell has asked for feedback on recently”, and get a helpful reply; I can’t see why this is impossible). So again, it seems to me that there is a risk here that you might be willing to take yourself, but ought not impose on others.
I would indeed be concerned if you are uploading other people’s work to the cloud in a way that gave insufficient copyright protection to that work (e.g. you store it on a publicly accessible website). Without having looked in detail, I’d guess that most (all?) universities will have data protection policies that specifically speak to this sort of thing, and that storage services like OneDrive will specifically be designed to satisfy concerns of this kind. (“Personal” storage spaces like Dropbox I’d be less confident about.)
I should perhaps have added: This only applies if you are using an LLM in a way that shares data. If you run a purely local instance, these same concerns might well not apply. I am less clear how much protection is offered by ticking the relevant privacy boxes; though I am antecedently sceptical.
There are some disciplines where people obey a norm of not posting drafts on websites because the journals in that discipline have this sort of rule about publication on a website. If any journal had any hint of such a policy regarding work that has been sent through a corporate LLM server, that might be relevant, but I haven’t heard any suggestion that any journal might adopt such a policy (or how they would attempt to enforce it if they did).
(Remember that it’s not too hard to take free open source LLMs like LLaMa from Meta, or DeepSeek, or Mistral, and set it up on your own computer in such a way that its data never leaves your own device. My university, UC Irvine, has even contracted with OpenAI, Anthropic, and Google, to run our own local instances of their LLMs in such a way that data never leaves the campus servers.)
Thanks, it’s interesting to know that some journals have had related policies (and the agreement at UC Irvine sounds great; my experience with open source perfectly local models is that they just aren’t as good, and this sounds like a good compromise).
But just to repeat the point I started with: The claim was in part about not knowing what the future might look like (including what policies publishers might decide they need to adopt), and just the modest claim that if there is some uncertainty here, then whatever you might reasonably decide in relation to your own work, it would be unreasonable to impose risks on others by sharing their work without their consent.
Yes, right. I would add that under the privacy agreement, some business license of AI do not permit any data to be transferred and stored at AI companies’ service. This is stricter than personal privacy setting, which is about temporary storage (typically 30 days) and deletion afterwards.
I am not a lawyer. I am under the impression that uploading a manuscript to an LLM without permission from the copyright owner (presumably the author, if the manuscript is under review) is a violation of copyright law.
For the reasons Kenny Easwaran points out, I think this form of infringement should be considered morally wrong and a violation of professional norms. As a legal matter, it should not be counted as fair use.
LLMs often “hallucinate” when asked questions about well-known, widely studied philosophical texts. It is completely unreasonable to expect an LLM to provide an accurate summary of a manuscript that is trying to make a novel philosophical argument.
Just speaking to your final point: In my experience, current-generation LLMs are pretty impressive at accurately representing specific works uploaded into their context window (note that this is very different from relying on their background knowledge—the latter is where hallucinations emerge), and human philosophers are often quite bad at this (especially when ideologically opposed to the position being argued for).
We don’t have to rely on a priori “expectations” (especially ones based on failing to understand the difference in access reliability between data in the context window and background training data). The most reasonable way to form judgments about these sorts of things is to actually try them out and judge the results for oneself.
My comment was based on experience, not on a priori expectations. Maybe you’ve gotten lucky with the articles you’ve asked LLMs to summarize. Or maybe your standards of accuracy are low.
We’re all just sharing anecdotes, but my experience is similar to Richard’s. I don’t use LLMs for refereeing papers, but they help me
to generate first sketches of handouts for my classes, and for this I’ve found Claude Sonnet especially good at summarizing and extracting key ideas and arguments. I’ve experienced very few cases of hallucinations about materials added to the context window (none that I can remember). Of course, you can only know if you know the material, so it’s no substitute for reading, but it’s honestly quite close to giving you what you need if you’re under a time crunch. Hallucinations on training data do remain frustratingly high though.
I should admit that my sample size is low, partly because my experience doesn’t make me think further experiments worth my time, partly because of my view that it is unethical (as well as illegal) to submit someone else’s work to an LLM unless they have given permission.
Should your experience be nonexistent if you really think it’s unethical and illegal?
It’s not unethical or illegal to upload one’s own publications, or to upload works that are in the public domain. There’s some complexity about works issued with Creative Commons licenses. It matters which license was used.
I also have indirect experience from other people uploading work to LLMs (including mine, without my permission) and reporting on the results.
I see. Thanks for the clarification.
I guess I’m with Richard on this. Let’s focus on published works because I agree papers under review may raise slightly distinct issues. If a principle entails that it is impermissible to use an LLM to read and discuss published works by living authors without their permission, the principle may need revising. It’s not quite a reductio but it’s close.
I agree there are unresolved copyright concerns about training (e.g. NY Times vs OpenAI lawsuit), but as noted, most of the use cases discussed here need not involve adding the materials to the training data. Setting those aside, to me, talking about a paper with Claude is like discussing it with fellow philosophers at a seminar, which would make it fair use. What is the morally relevant difference? That we can assume that authors have consented (tacitly or by signing a license agreement) to the discussion of their work in seminar rooms but not in context windows? (I’m assuming of course that their papers were legally downloaded m, e.g. through an institutional subscription.)
“The problem, then, with just citing an AI tool as the source of, say, some objection is that the AI tool is unlikely to be the original source of that objection. The source of that objection is more likely some author whose work has been used to train the AI.”
I think this depends a lot on the structure of the interaction with the AI tool, the structure of the view being objected to, and the structure of the objection.
If there are particular patterns in text that occur frequently, these AI tools will often copy them. If you ask it for an objection to utilitarianism, it may respond with some version of the repugnant conclusion or one of the trolley problem cases. That’s because versions of these passages occur thousands of times (perhaps tens of thousands of times) in the training data. It’s unlikely to provide the same sequence of words that exists in any of them, because it’s processing these things semantically, in a way that is sensitive to the context of the current conversation, including stylistic features of the word choices in the current conversation, rather than just copying words.
But if you provide it with a novel theory that hasn’t been discussed anywhere in the literature, there’s a reasonable chance that it responds with a novel objection, that isn’t the sort of thing that would be appropriately attributed to anyone else. As a simple example of this sort of thing, not in the form of the objection to a view, but a novel creation, nearly 2.5 years ago, before many of the modern advances in language models, I asked ChatGPT to write an essay comparing and contrasting the work of Michelangelo and John Cage. One of its paragraphs was about their similar use of “spiritual themes”, in Michelangelo’s case often Christian imagery while in Cage’s case often ideas from Zen Buddhism. It was able to put this together because there are enough discussions associating each of these artists with these ideas out there, and other discussions about how Zen Buddhism and Christianity can both be spiritual practices. But I doubt there was any person who had ever written down this specific association of these people. I don’t think it would be appropriate to attribute this specific comparison to any human.
And it shouldn’t be surprising that a mechanical tool could do this sort of thing. One of the points I have long emphasized when teaching propositional logic is that this is exactly what a truth table does – you take a set of propositions, and it considers every possible logical combination of them. If you provided it with an unusual set of propositions, it may come up with a point in logical space that no human has ever explored before.
Now the problem is that I don’t have any good idea of how to tell the two kinds of responses apart. You can ask the model, but it won’t be a particularly reliable judge of this (unless there is enough meta-discussion about the familiar point that it is making).
And of course, parallel things happen with humans – you’re having a conversation with someone about some view, and they put together an objection, but if it’s in the vicinity of things they might have read, they’re often not going to be sure if they’ve put together a combination that has never been put together, or if they half-remembered something that is already out there.
I wouldn’t want a rule against ever using an objection that originates in this way, any more than I would want a rule against attributing an idea to someone if you can’t be sure that they came up with it originally.
Though I definitely agree that people who use objections that originate in either of these forms should do some amount of legwork to try to figure out if it has appeared in the literature before.
Assuming such things must be disclosed, when should they be—when submitting the original manuscript? When submitting a de-anonymized version after acceptance? Acknowledgements and footnotes thanking particular people should obviously be omitted for anonymous refereeing. Since there seems to be widespread animosity toward AI users in the profession, disclosing AI during during the review process risks unduly biasing referees who have negative attitudes toward AI. For more or less the same reason that name dropping famous people to signal prestige should be prohibited under anonymous review, I’m inclined to think AI disclosures should also wait—provided, of course, that the use of AI for a manuscript remains within the bounds otherwise allowed by the journal.
I think Nicolas is entirely right here, but, following the logic of his argument, there is an interesting implication for the injunction against co-authorship. (Note: as a card-carrying Luddite, I refuse to use AI at all – I don’t have a dog in this fight!)
To see the problem, it’s worth spelling out the starting point: presumably, referees should judge papers on the basis of quality, rather than whether they were formed solely through the author’s own brilliance; so, a paper which contains several footnotes along the lines “thanks to X for suggesting this objection” is no worse than a paper which (justifiably and correctly) doesn’t contain any such footnotes. In turn, it doesn’t seem to matter where those suggestions came from; there are good reasons to hide where they came from in some cases (say, a risk of “prestige bias”); so, authors don’t need to acknowledge an AI suggested an objection).
Consider, now, a slightly different case. Imagine that there are two papers: one is solo-authored, one is co-authored by two humans. Again, I assume that this fact is simply irrelevant to the refereeing process: what matters is whether the paper is any good, rather than whether it was the product of one person or more than one person. To put it another way, refereeing a paper is not like marking an exam, where you are assessing the author’s skills. So, while I don’t think there are any norms around this, it doesn’t seem like there’s any reason you have to acknowledge whether a paper was co-authored at the point of submission. But, if so, I don’t really understand why it would matter whether the co-author was a AI system.
Sure, there’s a claim that this “author” can’t take responsibility, but the very notion that authors must be humans who can take responsibility is not obvious. After all, the history of norms of authorship is really very interesting and complicated; it’s worth remembering the large number of influential articles and books which were published under psuedonyms. And, in this case, it looks as if there is a risk that enforcing the norm would work against publishing high-quality contributions (or, even weirder, incentivise hiding the true history of high-quality contributions).
So, I think there’s a pretty compelling argument which says “sure, co-author with an AI, submit anonymously, then acknowledge in the publication”. At least, I don’t really grasp why that wouldn’t be a good norm if we cared about quality. (As I said, it’s no good to me: I’m just genuinely confused!)
I’m not a Luddite but I agree with this!
This seems to me like the right attitude to have. A lot of discussions within the profession seem to me to start with a presumption that the point of publication is to be a credit to the author, and then try to work out how we should change the system to more fairly attribute credit.
But to me it seems that the point of publication is to provide certifications to the reader that this paper is worth reading (with whatever disciplinary norms there are about “worth reading”, whether it’s correctness or relevance to current debates or whatever). Credit to the author shouldn’t be driving the debate about whether journals should be publishing more or less, and whether the author should be one or more people, and whether the author should personally believe everything they argue for in the paper, and whether the author should even be a person.
I think I’m also convinced by my own points(!), but it’s worth noting something weird here: I suspect that any argument for ruling out AI co-authorship (while allowing human-human co-authorship) is going to have to make some hefty assumptions (say, about how ‘credit economies’ work or about the nature of responsibility for speech acts). Maybe those assumptions can be justified, but they look like the sort of thing philosophers love to discuss. So, it seems really odd to rule the topic out of court.
I bet Ethics would be happy to publish a paper defending really counter-intuitive first-order ethical conclusions, as long as the arguments were really compelling and intricate. But when it comes to publishing norms, the approach seems highly conservative. There’s something odd here, although I can’t quite put my finger on what
That’s a great ideal (that “the point of publication” is to index worthiness to the reader). It works for full professors. But if you’re anybody else, the point of publication really is “to be a credit to the author”. The author needs that credit for a job, for tenure, and for so much more. I wish our profession were more like your ideal, but, sadly, it ain’t.
I understand the concern that the “point” of publication is to be a credit to the author, but I’m not clear it is in tension with the idea that the point of publication is to index worthiness to the reader; I assume that the reason publication provides credit to the author is because it signals that people think this author is writing things which are worthy for the audience. Imagine a journal decided to accept papers on the basis of some random mechanism, rather than judgments of philosophical quality – it seems plausible that, in that system, there would be no credit to the author from being published in that journal.
I think that there’s a very interesting question about what would happen to the job market if AI systems started producing brilliant quality papers and these flooded the pages of top journals; you could imagine a (highly hypothetical) situation where the quality of published philosophy goes up, but no individuals are getting credit. And that would, I agree, be a real mess. On the other hand, it’s not clear that it makes sense to say that journals would be somehow not playing their proper role in that ecosystem; it’s just that we would have to rethink how we do things.
I just want to register how surprised I am there is debate about the application of extant norms–commenters suggesting they can take my confidential work product and upload it to an LLM! Commenters suggesting they can just feed it into an AI to “help” their understanding, instead of reading it like I expect a human ought to read my words (i.e., those beings I am trying to communicate with)! Commenters suggesting that we don’t have to cite AI for specific content provided by AI (e.g., objections)!
Adding exclamation points to these does not constitute an argument, so much as it registers genuine surprise that our norms are so flimsy so as to entirely collapse, whereby people think it perhaps reasonable to do all these things that (I thought) our norms implied were straightforwardly and highly objectionable. Maybe the norms should be changed–that’s a fine debate. But I’m surprised to read people interpret status quo norms in these uber-flexible “yeah just upload it to the LLM if it helps! Yeah just pull the objections from the LLM without attribution, why not?” ways.
Huh. Do you think our extant norms prohibit uploading a paper to a PDF word count website (or downloaded software that offers the same functionality offline) in order to check the word count? I’ve never heard of such a norm. So it doesn’t seem like there’s any general pre-existing norm against uploading papers into software tools per se. Nor could there be any pre-existing norm about LLMs specifically. So I honestly have no idea what extant norm you think is violated by uploading a paper to an LLM.
(We have extant norms about attribution that seem like they carry over straightforwardly enough—I agree with Brian Weatherson’s comment here. But I’m quite baffled by the suggestion that uploading a paper into *software tools* violates an extant norm.)
> “Commenters suggesting they can just feed it into an AI to “help” their understanding, instead of reading it like I expect a human ought to read my words”
This is an egregious misreading (not uncommon among referees!) that you might avoid by plugging the original comment into an LLM and asking whether your gloss is a fair and accurate characterization.
People should not upload (e.g.) a paper they are reviewing for a journal to a PDF word count website, at least without checking whether they are signing away rights to the paper (that they don’t have!) by uploading it etc. You may be right that there’s no norm against using these websites, but this is just because most philosophers, like most other people, do not understand what they are doing when they interact with these websites. If people knew what these websites in fact were, they would mostly not use them, or at least there would be norms about their use, rather than the existing situation, where there are no norms.
The same is true with respect to LLMs, except the confusions here are typically more extensive.
All this talk about uploads seems misleading. It is orthogonal to the original discussion, which was about LLMs. (Or perhaps it was about “AI” more generally, though since no one is saying we need to go back and check 2010s papers for the use of autocomplete, I think it is just about LLMs.)
Sending a paper to a GMail account, or receiving it at a GMail account, or storing it in a particular Dropbox folder or iCloud drive for papers to be refereed, all involve uploading a paper. They are also ways that put the contents of the paper more at risk of being detected by malicious actors than using the most unscrupulous LLM company. (I don’t know what the most unscrupulous LLM company is, but I assume either xAI or OpenAI.)
Asking DeepSeek to summarise a paper does not involve any uploading at all.
Norms about when it’s ok to upload something should be applied in the first instance to uploading practices that have become ubiquitous over the last two decades, and not to practices, like LLM use, that don’t always involve uploading.
I’m confused on one point. How would Deepseek summarize a non-published paper it has never read? I really don’t understand how you would get it to summarize a paper without providing it the paper (ie, uploading the file, or the substantial the text itself).
Open source models can be downloaded and run locally, so they’re just “opening” a file, like MS Word and other software.
Deepseek, Mistral, and LLaMa are three “open source” language models that people can download and run on their own computer in such a way that no data is ever uploaded.
(A) Yes, I think if you are uploading confidential peer review docs to third party sites is clearly against our norms, in the same way forwarding a confidential peer review docs to colleagues would be (or, for that matter, forwarding a colleague’s draft to strangers without explicit permission). Non-special LLM subscriptions use uploaded material as training data. Submitters do not consent to such sharing. FWIW this applies to third party AI checkers too. I am confident that many many faculty are violating student’s rightful expectations, when they take students’ academic work and just upload it to random sites under the guise of protecting integrity of their assignments.
(B) I did not mean to imply that reviewers would not read the paper. Only that AI would be used to “help” (ie, supplement) their understanding. Which fundamentally changes the mode of reading and digesting a text as intended. But thanks for the little ironic argumentative barb.
I don’t think there’s an extant norm that referees have to read papers using the specific method that their authors intended. I might intend my papers to be read from start to finish in one sitting. But I have no legitimate complaint against a referee who reads the conclusion first, or digests a single section per day, or strings together post-it notes on a board, feeds it all into text-to-speech software, or whatever other quirky approach to “reading” they may have, so long as they do a competent job of refereeing my paper. (If their quirks help them to do a better job than they would otherwise do, it would seem outright unreasonable for me to expect or demand that they adopt my “vanilla” reading style instead!)
So it rather seems like you have strong (and in my view, false) normative beliefs surrounding LLM use and what it signifies, and are mistakenly “reading back” your implicit preferred LLM-norms into the “extant norms” that existed prior to LLMs. This could explain your “surprise” that others have a different view of how best to extend our old norms: you somehow failed to realize that they need to be extended because our old norms didn’t actually speak to whether there is anything wrong with using AI tools (responsibly) as part of one’s reading process.
(In principle, it shouldn’t really matter: the important question is just what norms we should endorse. But academics are a conservative bunch, so I worry that AI-restrictivists get unearned rhetorical support by claiming that their view is already implicit in our existing norms.)
What is this mode of interlocution? You’re interpreting my own principled beliefs and arguments as resultant of some kind of epistemic vice you have taken it upon yourself to diagnose, peppered with scare quotes that imply some kind of disdain. Such a bizarre way of interacting with colleagues—ironically, against the norms of collegial scholarly discourse.
I do think your analogy to order-of-reading doesn’t work, as it misses the principle at play, but won’t engage further on your terms.
Sorry if you found my writing style weird, but it’s not implying disdain. A couple are literal quotes referring back to terms you used, and others are just indicating a kind of metaphorical use of the terms in question.
Since I disagree with you about what the extant norms imply, and moreover found your original comment extremely surprising (because the position you expressed seems so clearly mistaken from my perspective), I’m trying to reason through and diagnose our disagreement. Identifying a possible conceptual mistake underlying the opposing view seems like a pretty normal way to do philosophy to me?
But again, no need to continue if you’re not enjoying the exchange. I’ve no idea who you are, whereas my comments are signed, so I’m certainly not intending to make enemies.
If anyone else wants to pick up the case that our old norms already covered LLM use of the sort I’ve described, I’d be genuinely curious to hear the supporting argument.
I think the more relevant question is the extent to which our extant norms prohibit taking an article that you’re refereeing and sharing it with your weekly reading group to get help refereeing it, or sharing it with a grad student to help referee it.
I think everyone would think the former is quite problematic. I don’t exactly know what people think about the practice of “co-refereeing” with a grad student.
The question seems to me about what amount of sharing is ok for work that sent to you as a referee, and the established norm is that not much sharing is ok. (And I don’t understand the circumstances in which someone would send a paper they are refereeing to a word-count website.)
I’m also an early career faculty, and these comments terrify me. Theyre giving me yet another reason to fear the peer review process, as if there weren’t enough already!
think of all the time reader 2 will save.
chatgpt: how can i help you today?
reader 2: read the attached essay and write a rejection review based on an a misreading a clear point on page three. make sure to scold the author for not citing some tangential literature and add a comment about how you are amazed that a paper like this wasn’t desk rejected.
I really don’t see what is to be gained from this kind of pearl clutching. LLMs are here to stay, and they are only getting more impressive, and widespread in their use (significantly so) by the year. Richard Y Chappell in the comments below is just trying to clarify how we can navigate publishing norms against the background premise that LLMs are obviously now a widespread tool used by many as standard practice – and that we can assume LLMs are being trained on pretty much all our work; the whole ‘how dare an LLM be trained on my work’ worry is not going to age well as it’s a bit pointless to protest against. A better use of our time is (as per RYC) just nailing down realistic norms.
There is tension between the first and second point. If AI cannot be an author, then it doesn’t need or deserve credit. Similarly to others who have commented, I don’t see a difference between getting an objection from an audience member in a Q&A session and getting it from ChatGPT. The same arguments apply to both. The person deserves credit, or they might be recycling something they’ve read which should be referenced, but it’s not ethically required to credit them in the paper or track the original paper they’ve taken their objection from. Being a non-person, AI seems less deserving of credit than a person in a Q&A.
Citation needn’t be about giving credit. We surely want authors to be transparent about their methodology, and pre-existing citation practices are an easy way to achieve this. And this should be something LLM boosters can get behind, since it’s one way to track what the technology can do now and what it might be able to do as the technology evolves.
(A) Portmore: “As nonmoral entities, AI tools cannot, as authors must, take responsibility for the accuracy and originality of their products.”
(B) Portmore: “if an author’s submission borrows some text generated by an AI, that text needs to appear in quotes just as it would if it had been borrowed from a person.”
These two look contradictory. Maybe the wording leaves some way to escape from the contradiction, but it looks like AI both lacks and has authorial agency. I think Boaz (above) points to something similar.
There also appears to be a tension between people complaining that LLMs are incapable of any philosophical insights, are just pattern matching, wouldn’t get a C+ in their class, and so on, and people suggesting LLMs are good enough to deserve a citation in Ethics. Now, these two sets of people might be mutually exclusive, but it looks to me like they overlap quite a bit. If LLMs are truly unoriginal and just recycle widely available average content into average ‘slop’, we shouldn’t worry too much about authors claiming credit where it isn’t due, because there isn’t much to claim credit for. But again, maybe the people arguing for giving credit do think LLMs are capable of insight, so maybe there is no tension.
Yeah, I think you’ve pointed to another tension in this discussion.
I don’t see any straightforward contradiction.
The reason for (B) isn’t that AI is a person who deserves credit and would be wronged if we didn’t give it; the reason for (B) is that the author has a responsibility to disclose which of the ideas in the paper weren’t theirs. I thought Portmore was pretty clear about this in the quoted text above.
Before LLMs, this meant acknowledging the person who gave you the idea. Footnotes are filled with “thanks to so-and-so for bringing this to my attention,” “thanks to so-and-so for pressing this point,” “thanks to so-and-so for suggesting this objection.” Of course this all goes alongside the obvious case of citations to extant literature which amount to acknowledging the more indirect input of another person.
Now, with LLMs, there are sources of ideas which aren’t people. That doesn’t change the underlying norm to disclose when ideas in the paper aren’t your ideas, or so the thought from Portmore goes.
But why say “just as it would if it had been borrowed from a person”?
To indicate how it should be presented in the text.
Quote it as if it had been borrowed from a person, but not because it was borrowed from a person.
Presumably the reason we cite persons is that they have moral agency and moral responsibility, and thus some degree of ownership of their intellectual products, which we are obligated to respect. If an AI has no moral agency or responsibility (Portmore says AIs are “nonmoral”), then why should anybody cite it as if it were a person? The norms governing persons don’t apply.
Maybe the reasons we’d cite AI and the reasons we cite people are different, or maybe even overlapping.
For example: the norm that the origin of something is clarified. We quote and cite other people’s work not just out of respect for them, but because we are identifying what is ours and distinguishing what is not ours.
I don’t think Portmore would agree with your presumption (based on the text quoted in the post). If you want to argue with Portmore about why we should cite, that’s fine, but it’s besides the point of the contradiction you tried to establish (which is the only point at which I entered the discussion, I don’t think there is such a contradiction).
On Portmore’s view (in so much as I understand it from the brief text quoted in the post), we don’t cite just to satisfy our obligations regarding others’ ownership of their intellectual products, we cite to disclose the sources of our ideas. If the ideas aren’t ours, we should cite them.
One source of ideas which aren’t ours is other people, so we cite those other people when they’re the source of our ideas.
Another source of ideas which aren’t ours is an LLM, so we cite that source when it’s the source of our ideas.
We cite agents who are the sources of our ideas, but we don’t normally sight the nonagentic causal processes that lead to our ideas, even when they are partly external to us.
One of the original discoverers of DNA structure (I forget whom) is supposed to have been inspired by the sight of a spiral staircase. Nobody thinks he should have cited that staircase.
There is a story that in 1953 Watson had a dream of a spiral staircase, woke up and later told Crick about it. However, he doesn’t mention this dream in The Double Helix book, so it is unclear whether that part is a true story. At any rate, however he came up with the idea, Watson does use the spiral staircase as a metaphor to think about DNA’s structure and to explain Franklin’s data.
I think we wildly overestimate how much agency we have over where our ideas come from, so we tend to think we are where the buck stops. And we give credit to people because we assume they have agency over what they’re receiving credit for. For a bunch of institutional and practical reasons this makes a lot of sense. LLMs are kinda making it more painfully obvious how much of our thought doesn’t come from us as agents, and a sense of unease puts us in weird corners (e.g. cite LLMs even though only people get credit).
“If you make yourself really small, you can externalize virtually everything.” – Dennett.
That’s why he couldn’t figure out where he was!
I’m going to go with the others below wrt to the problems of identifying what is or isn’t a source of anybody’s ideas.
I use Google Scholar, and surely I get lots of my ideas from its underlying search algorithm (which by now is heavily powered by AI). It carefully and at least somewhat intelligently provided me with a curated collection of articles and books. Do I cite Google Scholar?
And my research often involves running complex searches (e.g. boolean proximity searches) on the texts I work on. I get ideas by running those searches. Do I cite those search tools? I’ve used AI to do translations of Ancient Greek. Do I cite that?
All our research by now depends so much on a vast and usually united digital infrastructure. Which ideas came from where?
I’ve had research-level philosophical conversations with GPT-5. Some of these at least had a wiff of godlike intelligence. I haven’t used GPT-5 to write any articles, but I might like to. And in those cases, where it really has informed my thinking, I’d like to be able to cite it as a co-author. I’d be treating it more like a person.
But I wouldn’t quote it. I’ve co-authored articles, and me and my co-authors don’t quote each other. We just list ourselves as co-authors.
Edit: “with you and the others”.
It’s not the only reason we cite persons. We also do it so as not to be falsely credited as having come up with an idea we were not necessarily capable of having.
As long as it remains an impressive achievement in this business to come up with a point, an argument, a clever example, etc., then it’s wrong to present oneself falsely as having done so — quite apart from whether it wrongs another agent.
If the LLMs that engaged these tasks were called “Virtual Super Philosopher”, this would be even more obvious. A clergymember can get sermon ideas from all sorts of wacky stimuli. But if they downloaded it from Sermons.com and didn’t say so, the congregation would be rightly pissed.
I don’t see why you have a responsibility to disclose which of the ideas in the paper “weren’t yours” – I can understand that if they belong to someone else, then it’s worth giving credit to that person. But why should the reader of a paper have *any* concern about whether the author of a paper actually came up with the ideas, or just threw random words on a page that happened to be the sort of thing a referee thought a member of the discipline would find worth reading?
There’s plenty of disagreement I have with Derrida, but his point that we have no particular reason to care about the author in order to get what we want out of a text seems right to me.
There is no contradiction nor even the appearance of a contradiction here. There would only be a contradiction if we assume that the only reason for putting a text in quotes is that it is ‘borrowed’ from a moral agent that ‘takes responsibility for the accuracy and originality of their products’. Presumably Portmore rejects this assumption.
Isn’t that a reasonable assumption though? Compare this case. I’m trying to come up with a word to describe something in a paper, and I’m completely stuck on coming up with it. Later that night, while playing Scrabble, the letters on my tray precisely spell out the word I need, and it’s quite obvious that’s the right word. Should I credit the bag of Scrabble letters? It might be amusing, even charming if phrased correctly, to do so. But it’s not obligatory.
I think the lesson is that credit is mandatory not when the writer doesn’t deserve the credit, but when someone else does. And I kind of think (though Harvey Lederman’s recent work might be making me rethink) that LLMs are no more the right kinds of ‘someone else’ than bags of Scrabble letters.
I don’t see the point of the Scrabble analogy. The Scrabble tiles didn’t tell you what word to use, they didn’t fit it into the context of your argument, they didn’t formulate your thesis, they didn’t provide you with counterexamples, they didn’t propose possible objections. The Scrabble tiles didn’t contribute to your paper at all.
That there’s a causal connection between the accidental arrangement of the Scrabble tiles and your remembering the word you were looking for (or your thinking for the first time that that would be a good word to use) doesn’t seem relevantly similar to using an LLM to draft your paper.
If it is important to give credit when someone else is the proper author of an idea or its expression, it seems also plausible to assume that it is important not to take credit when you are not the proper author of an idea or expression (at the very least because we do use publications as evidence for conference invitations, hiring, promotion, etc). Sure, if you get one word from the scrabble tiles, it would not be significant enough to bother with crediting it. But if you threw the tiles (probably more than one set…) and it miraculously spells out the entire main section of you paper, or even the perfect formulation of an argument that you were struggling to make, it is less clear to me that it should not appear in quotation marks (or marked in some other way to acknowledge that this is how the text was generated).
If you have the miraculous ability to throw Scrabble sets at the wall so that they form perfectly formulated philosophical arguments, I am perfectly happy to hire you on that basis provided you can keep doing it.
Professors used to have to teach courses at Pitt when I was there, but I guess this is no longer true!
Fair! I meant for the research side.
Although, frankly, if you have that miraculous ability it sounds cool enough that I might try to magic up a research position for you.
I think one of the deep disagreements between some people in this discussion is whether the purpose of publishing norms is to ensure that we are publishing things that readers want to read (while making sure that people can get some appropriate credit when possible) or whether the purpose of publishing norms is to ensure that we are giving appropriate credit to authors (while making sure that readers get something useful to read when possible).
I agree that there are these two sets of norms or, roughly something like that, but I am not sure that the source of disagreement is which one takes priority. I certainly think that, with some caveats, the first principle is correct (at least if we add “and that people do not get inappropriate credit when possible”). That’s why when someone is beat to a result, they don’t get to publish their version, even if they already had everything set up ready for publication and might be just as deserving of credit. But Doug’s proposed rule seems squarely within the first principle, as “quoting” LLM output the same way as we quote genuine papers does not seem to go in any way against the purpose of publishing things that readers want to read (but perhaps you, or others, think otherwise, and this is the real source of disagreement?)
I must say I am quite surprised by a lot of the comments which suggest philosophers are using tools like ChatGPT quite extensively, and seem to think these are unproblematic use cases: getting ChatGPT to adjust the technical terminology in a sentence or paragraph, provide objections, summarize and make a paper more understandable that one is reviewing (?!), among others. The comments on DN and other forums of academic philosophers over the last few months would have suggested a much more restrictive use of ChatGPT by people.
I personally use ChatGPT like a smarter google, ie if I forget the term for some idea I will ask it “what is the term for that fallacy where one assumes blah blah?” It seems like people are a lot more permissive? As a recent PhD, it’s hard to not wonder if I am “falling behind” by being overly strict where others are using these tools much more and therefore writing and publishing more?
I’m pretty skeptical that they’re writing and publishing more.
The publication bottleneck isn’t really about writing, it’s due to the review process. In terms of volume of writing, it’s pretty easy to produce loads without LLM assistance. But it all has to go through the same bottleneck.
The bafflement you express at the idea that we should want referees to double-check their understanding of papers they’re reviewing makes me wonder whether you’ve ever been on the receiving end of a referee report!
Among the many referee reports I’ve received that have displeased me, I’m not sure any of them would’ve been better off along the dimensions I care about had the referee used an LLM in any capacity. One of the displeasing reports was, as far as I can tell, written partially or entirely by an LLM, and I think this contributed to many of its displeasing factors. Maybe my experiences are unusual.
It should go without saying that one can think that it would be great if referees could or would double check their thoughts, provided there is a permissible means of doing so, while also acknowledging that there is, as of yet, no permissible means of doing so that doesn’t involve obtaining the informed consent of the author. But apparently that doesn’t go without saying. Double checking would be great — if you could do it permissibly. But you can’t. Two things can be true at once.
What exactly do you think requires the “informed consent of the author”? (Because I enjoy arguing, I argued elsewhere that exposing data to LLM training does not require anyone’s consent. But the issue is moot: one can use LLMs without allowing training on your user data.)
Although I agree that it’s often impermissible to upload papers to LLMs, my bigger concern is that a referee in a position to make effective use of an LLM’s feedback to in turn shape the referee’s own feedback is typically in a position such that they don’t need the LLM. The people who write bad referee reports are not going to be effective LLM users, for the same reason our students haven’t become better at writing papers now that they have LLMs to help them. The skills you need to use an LLM to make a good referee report include all the skills you need to make a good referee report without an LLM.
The norm, I’m afraid, is going to become “use LLMs well,” but of course the norm is already “write good referee reports,” and people violate that due to lack of skill, and referees are similarly going to use LLMs badly due to lack of skill. So I am not impressed with (e.g.) Richard Y Chappell listing good things LLMs can do, because such lists serve mostly to convince people who use LLMs badly that they are doing nothing wrong.
The results are going to be (are already?) even more disastrous than the crummy state of affairs that obtains when people write bad referee reports using solely their own capacities. I don’t see any great way to fight against this except to make the norm “do not use LLMs,” but I think the battle might be lost. To the extent it’s not, I encourage everyone, especially journal editors, to promulgate and (to the extent they can do so) enforce a “no LLM usage for refereeing” norm. Is this unfair to people who could use LLMs well? Yes, but it’s worth it. You can take the time to manually double-check your report for the sake of upholding a norm that will prevent bad actors from misusing LLMs.
You definitely shouldn’t take the comment threads as a representative sample of disciplinary opinion! In particular, my guess is that the majority of comments on the threads on AI consist of discussions between about 5-10 individuals who are very opposed to using them for anything, and 3-4 of us who think there’s a lot of value in some uses of them. You definitely shouldn’t generalize too strongly from these couple dozen individuals to the norms in the profession at large!
Using an LLM like a better Google is an easy use case to adopt without changing much more of your practice. I’ve adopted a few more uses, but still probably not as many as I eventually will as I get more used to having them around (even if there is zero future improvement).
The main additional use cases I’ve adapted into my workflow are asking Claude to give me LaTeX code for a diagram illustrating an idea (now that I’ve realized I can do this easily, I am much more willing to insert a diagram or table when it would be worth a few hundred words), and using it to re-write abstracts of my work for different audiences. Just in the past week, I was trying to find which of several papers made a particular point I could remember in the literature, and having Claude to quickly re-read the papers and let me know was very helpful. (I think I’ll also probably start using it to help me get quick summaries of interesting-looking papers to help me know which ones I should prioritize reading soon, rather than just adding to my folder that already has 50 or so papers I’ve told myself I’m going to read on my next plane flight.)
I could definitely imagine running paragraphs through an LLM for re-wording, and to get a second opinion on whether I’ve defined some technical terminology effectively.
Providing objections is one of those uses that sounds more scary to me – but at least in part because asking for objections to my views always sounds scary.
And I definitely think (contra Richard) that one shouldn’t upload papers one is reviewing to online LLMs (though I think it would be totally reasonable to get a second opinion from a local instance of an open source LLM that you are running on your own hardware).
But there are probably many more uses that even current systems are already good for, that I *should* be doing (in the same way that I *should* be using a consistent naming scheme for different drafts of a paper that I’m working on over an extended period of time).
“ideally, you would track down the original source of the objection and cite that, while acknowledging the help of ChatGPT in bringing it to your attention.”
When you track down the original source after a person brings that objection to your attention, surely you don’t acknowledge the person in the middle (apart perhaps from the more general thank you note at the beginning or end of the paper)? I can see no reason to proceed any differently in this regard when it comes to ChatGPT.
Unless we want journals to become even slower and for the number of papers in need of reviewers to increase, then there are strong reasons to oppose LLM usage in this context, whatever other concerns we might have around authorship and copyright.
That depends on how LLMs increase the number of papers. If they just lead to a lot of superficially polished rubbish being submitted that takes time to be rejected, then agreed. But LLMs double the number of papers while keeping paper quality constant, that sounds good: more excellent philosophy research being done.
Compare: suppose someone said: ‘LLMs will double the number of new cancer treatments being developed and tested”. That would put more strain on the medical journal system, but it would be a nice problem to have.
Sure, if LLMs did a thing they don’t do, then things would be different. But if we’re talking in practical terms, there’s no evidence they increase the quality of philosophy papers and plenty of evidence of increased slop put strain on an already creaking system. And if there’s too much noise in any field, signals get lost.
As an editor and referee, I haven’t seen any evidence yet of increased slop putting strain on the system.
I did get one submission that seemed to have heavy LLM use (very extensive use of bulleted lists at points where I would have wanted paragraphs of arguments), but that one was both a quick desk reject, and also a paper I really would like to read a good version of!
See: what happened to Clarkesworld Magazine a few years ago.
I guess “just don’t use it at all” is out of the picture?
Individuals certainly are free to not use it at all, just like I am free to not use Microsoft products at all. I’ve found a set of workflows that lets me do good work without using Microsoft, and if you have a set of workflows that lets you do good work without LLMs that’s great. (Probably everyone currently publishing has that right now, since the vast majority of us spent most of our time developing our workflows more than 3 years ago.)
But I don’t think that a general norm that no one should use LLMs at any stage in the development of their work is any more reasonable than a general norm that no one should use Microsoft products at any stage in the development of their work.
Should we not consider starting a movement of “I am only reading human-made articles”? I don’t have enough time in my life to invest it in reading what a LLM has to say on a given subject. If I wanted to read that, I would go to OpenAI directly.
The difficulty with this is in saying what’s a “human-made article”.
Lots of people are probably already submitting articles that are mostly written the old-fashioned way, but with a few of the paragraphs run through a language model to see if there’s an alternate phrasing that might be clearer or more concise. (I expect this will be especially common when someone is running right up against a word count limit, or when a non-native English speaker has a very clear sense of how to best phrase something in another language but is less certain what’s the best equivalent in English.)
Probably less common, but also probably already happening, will be cases where someone has a 90% draft of a paper, and then runs it through a language model to ask for objections to consider that the human then writes up, or takes an objection they already have in bullet point and asks the language model to expand it into a stylistically-consistent paragraph or two.
Somewhat more problematic to me would be cases where someone comes up with the full outline of their article and then asks a language model to flesh it out into paragraphs, or where someone starts with a disorganized mess of paragraphs and notes for several talks and earlier paper drafts, and asks the language model to convert it to a useful outline that the human then fleshes out.
I would be very surprised if there’s anyone meaningfully submitting papers where whole sections just originate from a high-level prompt without any provided outline or paragraphs, which is what you’re going to get if you go directly to OpenAI yourself without the details of a paper in mind. I agree that I don’t want to bother reading many, if any, of this category. But I don’t know which, if any, of the earlier categories, you would count as “human-made”.
Or if, as Kenny suggested at another spot, they use an LLM to get the code for their TiKZ diagram. Or if they use an LLM to get the code for a graph in ggplot. Those diagrams and graphs are important, so I guess if you are really anti-LLM you should not want to read papers where they built the code for them.
Speaking for myself, I don’t feel any more hostility to those uses of LLMs than I would feel towards someone using TiKZ or ggplot rather than drawing by hand. If others do, that’s interesting, and suggests a very deep disagreement.
If your time is scarce shouldn’t you allocate it according to quality rather than provenance? And if you do, then the question is whether LLM-assisted papers are worse. If they are, then pass, and you’re not wasting time. If they are as good or better as ‘human-made’ papers then you might be missing out. I’m assuming that if they were published in Ethics they must be quite good.
I would like to know how to quantize gravity. If I read a paper that answers that question, why on Earth should it matter to me whether AI was used to help write it?
Reading the discussion, I am thinking that there are radically different norms in different subdisciplines of philosophy. The long established norms of some subdisiplines (e.g., *very* strict citation practice and other writing rules) make the people there think their opinion on AI-assisted writing doesn’t need any more justification. I am wondering why there is such a dramatic norm difference, and whether your earlier explanation of process vs product fully explains the difference.
I wonder if the following is the real difference. Those people really care about belief states only while scientific-related philosophers really care about propositions only. The former don’t really care about propositions unless they are entertained by specific people or otherwise related to some belief states. Reading is somewhat like socialization. In that case, I can understand the repugnancy of ai use, since I have zero interest in (say) dating an AI.
Here is what a number of bioethics journal editors came up with: Editors’ Statement on the Responsible Use of Generative AI Technologies in Scholarly Journal Publishing – Kaebnick – 2023 – Hastings Center Report – Wiley Online Library
I am glad that norms on AI use in philosophy research are being discussed. Thanks to Douglas Portmore for floating these guidelines.
But as a non-native English speaker, I often use generative AI in the way I used Google Translate, Google Dictionary, or Grammerly: to find an adequate word/phrase, to correct grammar, to see suggestions about sentence composition, to enhance styles, etc. If “us[ing] an AI tool simply to edit their writing” (in the second paragraph in the quote) includes such use, the relevant part seems a bit too demanding to my eyes.
First, I don’t see a clear reason why we should acknowledge such use of Chat GPT if we didn’t have to acknowledge Google Translate, Google Dictionary, or Grammerly before.
Second, I am sure that many non-native English speakers use various AI systems in that way for many sentences almost every day. So it is almost impossible for them to state what specific AI system they used for what parts on what date in the way Portmore’s example sentence does. (I believe Douglas Portmore’s example is just an example and not something to be followed as-is, but just in case.)
Anyway I appreciate the discussion!
Right, a lot of AI uses are intractable. But even for those that can be tracked, I suspect that there will be a justifiable psychological resistance to reveal the history of back and forth with AI on editing sentences, coming up with section titles, and so on.
Maybe this is tangential to the original post, but I’d be interested in hearing from faculty in graduate programs: Are you seeing an increase in student use of AI? And if so, how is your program addressing it?
Given the intense pressure to publish and present at conferences, not to mention the pressure to impress one’s faculty with stellar writing, I imagine the incentive to use AI for research writing is strong, particularly among students who may have already used AI extensively in high school or undergrad, and who may have had faculty encouraging that use.
Graduate school is where many professional norms are formed or reinforced in practitioners, so I don’t think my questions here are particularly too far out of left field.
I thought we LIKED research and writing? And these are creative, humanistic enterprises? That we don’t WANT to outsource?
(And what about the carbon footprint of AI?)
Many of us value securing stable employment and professional status and esteem more than research and writing per se, which come to be mere means to the former ends once one endeavors to philosophize professionally. Anything that can speed up securing steady employment and/or rising through the ranks, then, will be quite attractive to many.Perhaps people aren’t often so honest/blunt about this, but yeah, I think it’s the case.
esteem for what? your 60k asst prof salary?
When I was a kid, we had a name for this sort of attitude (not that you’re endorsing it): selling out.
Making a living, aka “selling out”.
If it only works to secure employment and reputation if you keep your use of it secret, I dunno that it actually does those things in the relevant sense.
It’s awfully uncharitable to assume that the only possible use of LLMs is to “outsource” the entire “humanistic enterprise”. (My main comment outlined a couple of ways that some might reasonably use LLMs to improve their refereeing, for example.)
fyi, the carbon footprint of an LLM query is negligible (comparable to 1 second of microwave use, or 1 second of using a vacuum cleaner); people are widely misinformed on this topic. If one finds a positive-value use for LLMs (as with microwaves and vacuum cleaners), it would be wildly irrational to pass up this value for fear of the environmental costs. I discuss this issue more here.
On the contrary, it seems awfully uncharitable to label passing up the use of AI “wildly irrational”. There are plenty of plausible non-consequentialist moral reasons for doing so, even if one grants your empirical claim.
Claims about rationality and irrationality are normative claims, about which I have systematically-developed views. On my account, which I’m happy to defend against substantive objections or counterarguments, it is wildly irrational to pass up a noticeably *positive-value* action (note the condition) *because* of its carbon footprint, when the latter is the same as 1 second of vacuum-cleaning. Now, I don’t see how having a systematic normative view of this kind could conceivably be “uncharitable”: it’s not (mis)attributing thoughts or motivations to others. It’s just offering a principled normative verdict about which trade-offs are or are not sensible to make out of concern for the environment.
I’d be interested to hear a “plausible non-consequentialist” response to my linked post on thinking about collective impact, offering a comparably systematic account that yields the opposite verdict (without also implying that it’s wrong to vacuum your floor). Nobody has yet presented me with such an account, but the comments on my substack are always open!
Contrast that with the comment I was responding to, which did (at least implicitly) misattribute motivations to others, even in the face of prior comments on this page that presented alternative accounts of how and why some might find legitimate, academically valuable uses of AI.
The “wildly” modifier irrational is not a precise, substantive claim but rather an ad hominem in ordinary language. Either way, it seems to me that your post was either uncharitable or parochial. Uncharitable if you decided to assign a “wildly irrational” position to the OP, when there were more plausible alternatives by our own lights. Or parochial if you couldn’t think of any other more plausible alternatives.
There is a wide literature on individual responsibility for climate change, as i’m sure you know. I contribute to it in (Obst 2024, “moral reasoning in the climate crisis”). One important aspect of the ethics of the individual use of AI is many people have strong in-principle objections to AI on climate and humanistic grounds. If you believe that, like the OP clearly does, there are many non-consequentialist moral reasons to omit its use (complicity being the most obvious) that makes doing so non-irrational. It seems you don’t share that wider perspective, and it doesn’t seem productive to try to hash that out here.
No, I simply took the OP to be ignorant of the empirical details, as most people are. I haven’t attributed to anyone the specific normative view that I take to be wildly irrational. I’m just highlighting, with an intensifier, how irrational a position it would be. If you care about the climate, it’s worth being sensitive to empirical facts about the actual carbon-intensity of one’s actions, since the variation is immense!
Further, if you look at what I wrote, it was: “it would be wildly irrational to pass up this value for fear of the environmental costs“. This leaves open that there might be other (e.g. deontic rather than cost-related) reasons, even though I don’t personally think reasons of complicity disconnected from details of magnitude make much sense.
You may not wish to pursue this further here, but I do think it’s worth emphasizing that on any sensible view, concern about one’s individual responsibility for climate change should be sensitive to the magnitudes of the carbon costs of one’s various consumption activities.
To prioritize cutting some of one’s least carbon-intensive activities over other forms of consumption (driving, flying, etc.) that are many orders of magnitude more carbon-intensive would seem “non-consequentialist” in the radical sense of “completely insensitive to empirical consequences.” I trust that reasonable non-consequentialists do not intend to embrace a position of that form!
In addition to the examples Richard mentions, it’s useful to compare the carbon footprint of AI to the carbon footprint of Zoom calls. My understanding is that, per second of use, Zoom has a far greater carbon footprint than nearly any of the AI systems. (Some people worry that the carbon footprint of training is large, but everything I’ve seen suggests that training is much less of the total carbon footprint of AI use than the eventual use of the model by tens or hundreds of millions of users.)
(And regarding research and writing – I’m sure all of us like some parts of it more and some parts less. Most of us love the part of coming up with ideas. Some people also really love the part of getting a draft down on paper, but hate the part of editing and polishing it. Other people hate the part of getting a draft down on paper, but love the part of editing and polishing it. People who hate one part of the process might naturally outsource that part, either to a coauthor or to a machine, so that they can spend more time doing the part they love.)
I don’t know but just having a look at all these comments already puts me off the entire matter. I simply don’t believe that this AI business can be regulated properly and I also don’t believe that people will keep their side of the deal, i.e., follow the norms should there be any agreed upon eventually. I suppose I just soldier on without AI and, well, some people will just publish more (and perhaps better quality). I can’t be less bothered, to be honest.
It has indeed been a remarkably dull conversation. Not that I’ve helped!
Can we just require philosophers who lean on AI to self-identify so that I know which papers to ignore?
This sounds a lot like requiring philosophers who use translators and editors to assist with their use of English as a second language to self-identify so that you know which papers to ignore. Or requiring Kantians to self-identify so that you know which papers to ignore. I’m really surprised that people are so willing to openly talk of dismissing work from groups of people they dislike!
I mean, while we’re at it, people should disclose who their advisor was, where they live, and who they vote for.
People weren’t willing to openly talk about it; the person talking is anonymous! I find that the anonymous commenters on daily nous make dismissive remarks at a higher rate than the non-anonymous commenters.
That’s not my experience. Consider the irony that of the comments in this thread 2/4 (arguably 3/4) are dismissive remarks by named participants.
I’ve generally found that those on Daily Nous who use their own names are tenured professors. You have the luxury of making poor arguments in public without having to worry about a hiring committee spotting it. Then you mistake class advantage for virtue.
I think any department would be lucky to hire any of the named participants you have in mind (not including me, and I am not tenured). I’ll leave it as an exercise for the reader to decide whether they’d be lucky to hire the anonymous participants whose arguments are so good that they won’t take responsibility for them.
How? That’s just a transparently false equivalency.
The things you mention are perectly respectable writing strategies or positions. Using AI to write your paper instead of doing it yourself isn’t. I don’t believe in desk rejecting plagarized papers because “I dislike them,” I believe in doing so because they contribute nothing and are unethically produced. Likewise with AI.
Is anyone actually considering using AI to write the paper *instead of* doing it oneself? All the discussion here is about whether to use AI to help rephrase some paragraphs, or to put forward some ideas of objections, or something like that.
Do people not credit the person who translates their work?
I know it’s common not to acknowledge the grad student who compiles your index, but that’s always seemed wrong to me.
A related question. I’ve never seen anyone credit Lex Academic in a publication. But they used to promote their services a lot and more than a few people must have used their services to write articles, chapters, and books. I’m sure I’ve missed some people acknowledging them, but I suspect many people don’t think they needed to be cited or even acknowledged. I wonder what relevant difference there is between paid-for editing/consulting services and LLM assistance.
Why would you ignore papers that use AI? Would you ignore a revolutionary mathematical proof because it was made by AI? Or a cancer drug because it was invented by AI? Are you assuming AI can’t do philosophy at a research level yet? If so, what’s your evidence?
Maybe you have some other issue in mind. Maybe you have ethical issues with using AI. But do you really have a principled ethical system for when to ignore papers?
My evidence AI cannot do philosophy is every F-worthy paper my undergrads have submitted. The burden of proof it can is on you. You cannot argue something is in fact capable from an unproven hypothetical – where’s your evidence?
Also, yes. AI papers are slop and it’s a waste of time to read them. If we could identify and ignore them it would save our time to do actual work.
As I said in another thread, I don’t think anyone is considering “AI papers” here, if that means someone just gives a prompt, has the AI write something with the form of a paper, submit that, and then somehow it magically gets through the referee process.
The actual relevant issues are cases where someone uses an AI tool to polish the phrasing of a few paragraphs, or to help brainstorm some ideas that the human then develops and writes up, or maybe at the most extreme to turn the outline of a few paragraphs into actual paragraphs.
If AI is writing the paragraphs, it is writing the paper for you. You can’t insist that “nobody is saying X” and then clarify that you’re saying X.
If every AI paper students turned it was F-worthy we wouldn’t have to worry about cheating and the death of the take-home essay. So if you’re right, good news? (Or if the news are bad, you’re wrong?)
Steinhart’s argument rested on AI “doing philosophy at the research level.” Are you claiming that AI essays for 101 classes occasionally fooling underpaid graders is evidence that they have reached this level? If so, I’ll leave it as an exercise for the reader to decide whether they’d be lucky to hire someone who can’t distinguish a genuine philosophical contribution from AI generated slop.
I’m only addressing your claim that the papers are F-worthy. If this is irrelevant to Eric’s argument this is on you since you brought it up in response.
I didn’t make any claim about being able to distinguish AI slop from original research. Of course everyone can. The question is between good original AI research and good human research. You keep misunderstanding the point of this whole discussion.
I think it’s worth pausing to ask why we have the citation and credit norms we have.
I take it those norms are:
(a) If you discuss an idea or example that has been published elsewhere, you are obligated to cite that publication. That obligation applies even if causally that wasn’t where you got the idea from, though often it’s not blameworthy to fail to do so (if it was published decades ago in an obscure location, say).
(b) If you take a specific idea from a colleague, e.g. from an objection in a talk, written feedback, or a referee report, you are obligated to give appropriate credit. (Here of course causation does matter.)
(c) You have some minor obligation to acknowledge general sources of help but not doing so is more a social and personal failing than a scholarly failing.
(d) Otherwise, your own creative process is a black box. There is no obligation to document your approach to writing or philosophizing or the sequence of events that led to your paper being in the form it is; indeed, it would be unusual and odd to do so.
And I take it the point of these norms is that academia is a credit economy. We award the originators of good work and good ideas with jobs, promotions, grants, conference invites, and so on, partly because doing so incentivizes good work and partly because good work in the past is a good predictor of good work in the future. That means it’s important to accurately identify credit, and to have mechanisms (i.e., publication) where credit can be publicly logged so that others can’t take credit from you later.
But LLMs aren’t (yet!) agents. They can’t apply for grants; they can’t be appointed to tenure-track positions; they can’t keynote at fancy conferences in exotic locales. So depriving them of credit harms neither them nor the community’s scholarly goals. (Nor are they offended or harmed if you fail to credit them for more generalized help, e.g. critical feedback.)
What about inappropriately giving credit? It’s not obvious why it *is* inappropriate. What we care about is identifying people who in fact generate exceptional philosophical research writing. If the way they generate it in fact involves being good at LLM prompt engineering, what does it matter, provided that the work is good? Different cognitive skills have waxed and waned in importance as philosophy has evolved; it’s not obvious why this is different.
(It’s not obvious if you view philosophical research as akin to scientific research, at any rate. I do, but there is another conception of philosophical research where we mostly value the process. On that conception, things might be different.)
And what about the risk that your idea is actually someone else’s idea, regurgitated from the guts of an LLM’s training data? That risk is real, but it’s not qualitatively new, because (recall) you are obliged to cite the person who had the idea first *whether or not* they were causally responsible for you having the idea. You need to do literature due diligence even if you are 100% certain you came up with the idea yourself; probably if the idea comes from an LLM you should do more due diligence (and the LLM can probably help you with that), but there’s no fundamental change here either.
I agree with almost all of this, and I wish I’d put it this clearly. But there’s one point I partially disagree with.
I think sometimes we should cite tools. This might be a point (e) on the list, and it might be relevant to LLMs.
When I did stuff using topic modeling algorithms, I cited the particular sources of the algorithms I used. When I used a traveling salesman example in a paper, I cited the source of the data and again the algorithm I used to solve it.
Or, to be more precise, in all those cases I cited the particular people (mostly academics) who had collected the data and/or developed the algorithms. And I think it would be wrong not do so.
It’s not just me doing this. Here’s an old paper of Branden Fitelson’s from when automatic proof systems were just getting going where he includes some long automated proofs and is very clear about their origin.
https://fitelson.org/dist.pdf
And he cites both the tool and, I think more importantly, the person who made the tool. I think it would have been wrong to have included the 80-something line proofs without any attribution. That’s not because anyone else had the idea behind those token proofs.
I have no idea at all what kind of principle would deliver the verdict that these tools should be cited and not the further verdict that I should be citing the original writers of R or Excel or something for half the things I do. But these cases of tool citation make me think that (a)-(d) are not quite complete stories of the norms behind citations.
That’s interesting (and fairly persuasive).
I think *part* of citing tools is the credit economy again. Academia has a lot of open-source tools that people make and that get widely used, but don’t get you co-author credit; it’s a slightly vexed question just how to acknowledge that work, but at the very least it ought to be acknowledged. (It comes up a lot with open-source simulation software in astrophysics and cosmology.) From that point of view, the reason to cite those sources and not Excel might be that Excel is a commercial product (and not made available open-source). Still, I’m not sure that’s the whole story (and it’s not like I cite Donald Knuth and Leslie Lamport every time I typeset a paper in LaTeX either!)
I recently had a paper accepted that had a bunch of diagrams that I had drawn in Keynote and then included in the file. But when it came time for final submission, I realized that the images I had included weren’t vector images, so there was some visible pixelation.
I gave the images to Claude, and asked it to write up TikZ code to output the same diagram. For a few of them, it basically gave the code I wanted, but for several of them I then was able to take that code and tweak it until it looked right.
As David says, I definitely don’t cite Lamport and Knuth for LaTeX in all my papers, and I wouldn’t have considered citing Keynote if I had used the original images. I’ve assumed it doesn’t make sense to cite Claude in this case either, because it was just helping me come up with code to re-draw the diagram I had already come up with (and also because most instances required more tweaking from me too).
Does that sound right, or should I cite one or more of these systems?
Suppose you are judging a weaving competition. It’s time consuming inspecting the weaves. But there is at least a limit to how many pieces come forward for evaluation, as it takes time to train as a weaver and prepare a high quality entry. You can also be confident that each piece can be attributed to a human, and the significant labour invested by the weavers offers you, the reviewer, assurance that you are engaged in a cooperative activity highly valued by others.
Then the steam-powered loom is invented. Some weavers carry on with manual submissions. But others knock out a weave in a minute with their steam-powered loom, and submit it to the weaving competition.
Surely you, the judge, would want to know which weaves are manually prepared and which were mass produced. Some of the reasons for this are (1) proper attribution (i.e. to a human or not), (2) triage (as judges’ time is valuable and better spent on the manual weaves that have significant labour invested in them) and (3) demand management (so judges aren’t swamped with thousands of mass-produced weaves).
Note that, while (1) is a credit and citation norm, the point is not to fulfil the claims the machines have as moral agents (they aren’t). Rather, the point is to ensure that humans get credit for things they have done and don’t get credit for things (like manual weaving) they have not done. Furthermore, (2) and (3) are not credit and citation norms, but weighty nevertheless.
Of course, by design, the analogy allows that, at least in time, the mass produced weaves will be better in some technical sense than the manual weaves. But the three points would still apply as none of them turn on manual weaving being a superior manufacturing process.
Manual weaves and manual papers do of course use some traditional tools. The example above assumes these are clearly distinguished from steam looms and genAI. Matters are slightly complicated by the fact genAI is used for the likes of translation and referencing, things previously done by traditional tools. Perhaps these uses of gen AI can be treated as relevantly traditional and part of the manual process fully described. But as soon as you’re getting genAI to generate objections or rewrite English paragraphs into better English paragraphs (probably changing the argument a bit in the process), you’re firing up the steam loom, and shouldn’t be surprised that the weaving competition judges want to know this.
Sure, maybe essay prizes could reasonably have stricter rules for LLMs if they like. But I thought we were talking about journals.
If you’re running a competition, then the criteria should be whatever makes sense for your competition. But if you’re trying to advance knowledge through academic research, it doesn’t make sense to exclude tools that make research better.
A closer analogy: The advent of computer simulation and numerical modelling has transformed large parts of the experimental and theoretical sciences, with computers doing in seconds calculations that would take humans days or weeks. Should we ban scientists from using computers like this, on the grounds that (1) we can then properly attribute calculational prowess to humans when appropriate; (2) referees’ time is valuable and better spent on the papers which were hand-calculated; (3) demand management?
My suspicion is that people are importing norms from the classroom into academic research, conflating the constitutive aims of teaching, which may preclude unbridled AI use, with those of research.
Maybe some people are, but that could hardly explain away the norms I gave. Two of the three (triage and demand management) have no application in the classroom context, because markers are obliged to mark every assignment, and the number of submissions (“demand”) is unaffected by AI use.
The latter is a respect in which matters are much worse for academic research than they are in the classroom. The AI enthusiasts are only looking at this from the perspective of the submitter (Hey, I can write a paper in a fraction of the time!), not the poor editors and reviewers, who will look back on the “peer-review crisis” of the early 2020s with great fondness.
Fair enough. I was only addressing the credit point. I don’t have a view on triage and demand, though I’m skeptical.
No one is proposing to ban genAI – only to acknowledge it. If R gets acknowledged why not ChatGPT?
Your analogy effectively implies banning it. You want to triage the (in the analogy) mass-produced weaving so as to free up time for the judges to assess the handcrafted work.
And in science, generally papers do not acknowledge tools used to do calculations more effectively, like Mathematica. Generally you do acknowledge software used to do statistical analysis, but I think that’s because the analysis itself isn’t in the paper – it’s more analogous to describing your experimental setup.
There’s a big difference between a ban and the mere option to triage. Presumably that’s an option many editors and reviewers would decline, given the pro-AI responses here.
As Brian and David are suggesting, many of us see the point of journals as very different from the point of weaving competition. The core purpose of journals doesn’t rely on a human having produced the paper – it just relies on the paper being worth reading.
I think the more analogous situation for a journal would be someone who is not judging a weaving competition, but instead is purchasing woven goods to provide to guests at their hotel or in their cruise ship or whatever. The fact that a human spent a lot of effort to produce a woven good is a meaningful signal that at least one person considers it worthwhile. But this is only a defeasible signal, and it’s not obviously a good enough one to use to triage –
-unless it’s actually clear that the machine-woven goods tend to have difficult-to-detect flaws that would become apparent to guests after a few hours of use, but aren’t apparent to the evaluator in the couple minutes they usually have to look at things.
Unlike many commentators here, I am rather saddened by the lack of fundamental trust in the fellow academics expressed by *some* of the posts that express strong and ungrounded objections towards AI uses (and got many upvotes).
consider this first, when we receive a paper to referee, why don’t we go look up who the author is? The norm of course. But also because we *care* about writing an unbiased reports.
similarly, for places where ai fall short, I don’t use it because i care about the quality. And for places that Ai can help, I use it because i care about the quality. Or save time, which can then be used to improve quality.
why doubt all this? if we lack this trust for most (or many?) members of profession, how can we still be a community? Yes we can discuss whether ai helps or not, or about being vigilant, but not act as if who use ai are basically cheaters, no?
As many have noted, #1 and #2 are in tension, if we assume that the reason to cite an LLM is that it’s enough like a person that we ought to acknowledge its efforts — “credit where due”, as the op-ed says.
But it seems to me that it’s just a tool. I’m not stealing from ChatGPT. We don’t have norms of citing tool use in philosophy. I don’t cite Overleaf, or spell-check, or Zotero. I don’t think I should cite ChatGPT if I use it to format a bibliography to align with a journal’s idiosyncratic house style. If someone prompt-engineers an actual novel objection, why isn’t the prompt-engineering enough work for the objection to count as theirs? If you use ChatGPT to summarize lots of new articles so you can decide which ones to focus on, what would you cite?
So I think you’d have to make a case that philosophy’s *method* requires no LLMs; grounding citation in existing practices presuppose that it’s the one doing the thinking.
I don’t know if people are aware of this research earlier this year https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5255039:
I imagine the situation would be even more (perhaps much more) radical in philosophy. Even if AI-assisted papers pass the referees of Ethics, they would not have audience, and even if they do, the reputation of the authors might even hurt.