Are applicants to your graduate program using AI to create their admissions materials? Do you know? How? And what, if anything, should your department do about it?
The topic is suggested to me by Alex Guerrero (Rutgers), who, commenting on a social media post by L.A. Paul, writes: “What are people doing about AI and grad admissions? Writing samples, personal statements. Other fields use interviews; are any philosophy departments considering those as a check on AI assistance?”
Does your department (or college or university) have a policy or set of practices about this? Do you have thoughts about what such policies or practices should be?
Discussion welcome.
See also: “Does Your Department Have an AI Policy? Here’s Edinburgh’s“
Pitt HPS this year will be trialing Zoom discussions of writing samples for shortlisted candidates. There are various reasons, but certainly an important one is making sure we can be confident that candidates are in full control of the ideas in their writing samples.
That sounds like a good idea. Let us know how it goes.
This is a good idea, but being able to chat intelligently about a paper is a lower bar than producing it.
Just for example: give me a week to study, and I think I could chat intelligently about most of your papers.
Yes but presumably you were rightly admitted to grad school
Agreed, but (a) the best can’t be the enemy of the good, and (b) I think the gap is smaller for a grad admissions essay than for a published research paper. (And (c) Anya’s point below.)
For various other reasons, PhD programs at my institution have always been doing interviews as part of the application process. We are starting up our PhD program in Philosophy this year, and we followed the trend of the other departments and did interviews, in part due to the various other reasons and in part to help get around the AI concern.
Idea to help undergraduate students that want to go to grad school:
Last Spring I started offering an option in my upper-level classes for such students to write a term paper that will serve as the basis for their writing sample. (I switched to only in-class essays a few years ago to deal with AI-generated essays.)
It involves several meetings with me to talk through their ideas and get updates on their progress. They must also show me several drafts, and sign a written agreement acknowledging how AI can be used (ok for research such as help finding sources, not ok for generating ideas, any writing or editing, though basic grammar software is ok).
They must also acknowledge that if they violate this agreement, they would be harming me given that I would be unknowingly misrepresenting their work to my peers in any letters I write on their behalf.
The purpose of this process (aside from, obviously, mentorship) is that I can then include this information in my letter of recommendation. I can say that I have been involved in the process, seen them wrestle with the ideas etc.
I don’t know how much this will help the students that have signed on, and it is a labour-intensive process, but I think it is worth considering for those who have the time and resources to do so.
Just as academic job applicants are asked to submit statements about their teaching, research, diversity, etc., maybe grad-school applicants should be asked for a statement about their AI use.
This wouldn’t necessarily be used to rule anyone out automatically, but it signals how applicants would approach their PhD-level research and whether the department is prepared to supervise such an approach.
I can imagine that some departments may be open to AI-assisted research, so those departments would be a better match for some applicants. But an applicant who relies greatly on AI, even responsibly (if that’s even possible), may be a poor fit for a department that has decided to ban AI (as law schools are now doing, e.g., Berkeley and UChicago) or doesn’t have the AI expertise to supervise.
Also, an AI statement gives the applicant an opportunity to demonstrate their awareness of LLM ethics—this is now crucial to research ethics and academic integrity. It could even persuade the faculty admissions committee that such an approach is thoughtful and responsible.
And it can be a filter in that an applicant who says they don’t use AI, but whose materials were clearly AI-assisted, could be identified and rejected. That is, it’s a basic integrity check.
I’m just talking aloud here and don’t yet have a position: is this idea good or bad? If good, is there anything that could prohibit departments from asking for AI statements?
I think it’s a good idea if it were costless. In practice, it will mean more work for applicants and I am sceptical the upside is big. There will be an incentive for applicants to say whatever makes them sound best so we end up rewarding the best guessers or liars, and even if that were ruled out, it’s unlikely to make or break an application anyway. Plus I don’t think there is much agreement within departments about AI use (certainly mine covers all extremes) so something like this always ends up giving a veto to whoever happens to be doing grad admissions.
Fair enough, Anya. I guess I was thinking that there may be utility in getting applicants to confront the issue as they prepare their materials (e.g., to be careful in avoiding AI in their writing sample or research proposal, so to not contradict themselves in the AI statement) or to choose to lie or be deceptive, which may help push them to consider if they’re really cut out for grad school. That is, even if there’s not much value for the department, there may be value in nudging applicant behavior?
If you still don’t like that, what about this: instead of a separate statement about LLM ethics, grad schools can follow the norm in academic publishing to simply require applicants to disclose the extent of AI use, if any. This would apply to their writing samples, research plan, and other statements/work they submit.
Maybe grad depts are already doing that? I don’t know since I’m out of that loop. If so, what are they seeing?
Anyway, the revised idea really wouldn’t require more work from the applicant–just a few more sentences–and it’s nothing out of the ordinary, since it follows an established norm…
No more statements! Applicants have enough nonsense to wade through already, and admissions committees have enough stuff to read. Just ban AI usage on admission materials. No matter what anyone thinks about whether and how seasoned professionals should use AI, I think it’s clear that being completely reliant on it before even beginning your graduate education will leave you in a position where you lack the capacity to properly evaluate AI output, so I think it’s also reasonable to expect applicants to be able to produce at least their application materials without AI.
I’m totally fine with that, Daniel!
Taking it further, if it’s determined that the applicant used AI in the process, then their admission can be revoked. At any point during or even after their grad work is finished, if they’re found to have used AI in their PhD journey in proscribed ways, then their degree also can be revoked as a straight-forward academic dishonesty case, on par with plagiarism.
But it’s hard for me to imagine a department that’s brave enough to do that. Even just one would be a welcome beacon in this darkness…
If they say they didn’t use AI, and they did, you’ll reject them. And if they say they used AI, you’ll also reject them. So what’s the point of the statement?
Well, your first sentence contains a point: the AI statement can filter out dishonest students.
I wasn’t thinking about this as a “gotcha”, though I suppose it can be. If departments don’t want grad students who rely on AI, they should make that clear from the outset to save everyone time and effort in the application process. So, saying that you used AI–in ways that a department thinks is responsible–isn’t a deal-breaker but a transparency measure.
Some or many departments may be open to responsible AI use. For those applicants who rely on AI, an AI statement can also gauge their understanding of what responsible use looks like.
Anyway, if the only point of all this is that it deters applicants from using AI, that’s also a good outcome, given the implied concern in this DN post and thread.
P.S. I’m already abandoning the AI statement idea (as a separate document) in favor of an AI disclosure acknowledgement for the writing sample and other stuff. I think the latter can do the same work as the former, and it’s much easier to implement for everyone concerned.
Graduate Admissions in the AI Era? If your name shows up here in green…
https://certifiedaifreeskillsandknowledge.org/
…justly brag about it.
This is the only way we get out of this mess.
Perhaps grad programs should start weighting GRE scores more in decisions, especially the analytical writing portion, since those are taken in controlled environments with no access to AI. Perhaps consider inviting, if not requiring, LSAT scores as well.
This would be quite unfair to international applicants. The GRE writing part is a very special exam, and if one doesn’t know how to prep for it, one won’t do well even if one is a decent writer.
(Purely anecdotal, but I was applying from a country where no one had ever heard of the GRE, so had to fend for myself when I prepared. I did quite well in the other parts but didn’t prep for the writing much as I didn’t know how to and it was not great. Fortunately, my grad school didn’t care very much about it and since then they seem to have gotten rid of the GRE requirement entirely.)
Grad programs should definitely use Pangram to screen writing samples and personal statements.
Two obvious points:
1) More personable processes, such as a higher priority on interviews and/or visiting days.
2) A higher importance on detailed, personable letters of recommendation; this will obviously favor applicants doing to small schools, all else equal.