Graduate Admissions in the AI Era
Are applicants to your graduate program using AI to create their admissions materials? Do you know? How? And what, if anything, should your department do about it?
The topic is suggested to me by Alex Guerrero (Rutgers), who, commenting on a social media post by L.A. Paul, writes: “What are people doing about AI and grad admissions? Writing samples, personal statements. Other fields use interviews; are any philosophy departments considering those as a check on AI assistance?”
Does your department (or college or university) have a policy or set of practices about this? Do you have thoughts about what such policies or practices should be?
Discussion welcome.
See also: “Does Your Department Have an AI Policy? Here’s Edinburgh’s“
Pitt HPS this year will be trialing Zoom discussions of writing samples for shortlisted candidates. There are various reasons, but certainly an important one is making sure we can be confident that candidates are in full control of the ideas in their writing samples.
That sounds like a good idea. Let us know how it goes.
This is a good idea, but being able to chat intelligently about a paper is a lower bar than producing it.
Just for example: give me a week to study, and I think I could chat intelligently about most of your papers.
Yes but presumably you were rightly admitted to grad school
For various other reasons, PhD programs at my institution have always been doing interviews as part of the application process. We are starting up our PhD program in Philosophy this year, and we followed the trend of the other departments and did interviews, in part due to the various other reasons and in part to help get around the AI concern.
Idea to help undergraduate students that want to go to grad school:
Last Spring I started offering an option in my upper-level classes for such students to write a term paper that will serve as the basis for their writing sample. (I switched to only in-class essays a few years ago to deal with AI-generated essays.)
It involves several meetings with me to talk through their ideas and get updates on their progress. They must also show me several drafts, and sign a written agreement acknowledging how AI can be used (ok for research such as help finding sources, not ok for generating ideas, any writing or editing, though basic grammar software is ok).
They must also acknowledge that if they violate this agreement, they would be harming me given that I would be unknowingly misrepresenting their work to my peers in any letters I write on their behalf.
The purpose of this process (aside from, obviously, mentorship) is that I can then include this information in my letter of recommendation. I can say that I have been involved in the process, seen them wrestle with the ideas etc.
I don’t know how much this will help the students that have signed on, and it is a labour-intensive process, but I think it is worth considering for those who have the time and resources to do so.
Just as academic job applicants are asked to submit statements about their teaching, research, diversity, etc., maybe grad-school applicants should be asked for a statement about their AI use.
This wouldn’t necessarily be used to rule anyone out automatically, but it signals how applicants would approach their PhD-level research and whether the department is prepared to supervise such an approach.
I can imagine that some departments may be open to AI-assisted research, so those departments would be a better match for some applicants. But an applicant who relies greatly on AI, even responsibly (if that’s even possible), may be a poor fit for a department that has decided to ban AI (as law schools are now doing, e.g., Berkeley and UChicago) or doesn’t have the AI expertise to supervise.
Also, an AI statement gives the applicant an opportunity to demonstrate their awareness of LLM ethics—this is now crucial to research ethics and academic integrity. It could even persuade the faculty admissions committee that such an approach is thoughtful and responsible.
And it can be a filter in that an applicant who says they don’t use AI, but whose materials were clearly AI-assisted, could be identified and rejected. That is, it’s a basic integrity check.
I’m just talking aloud here and don’t yet have a position: is this idea good or bad? If good, is there anything that could prohibit departments from asking for AI statements?
I think it’s a good idea if it were costless. In practice, it will mean more work for applicants and I am sceptical the upside is big. There will be an incentive for applicants to say whatever makes them sound best so we end up rewarding the best guessers or liars, and even if that were ruled out, it’s unlikely to make or break an application anyway. Plus I don’t think there is much agreement within departments about AI use (certainly mine covers all extremes) so something like this always ends up giving a veto to whoever happens to be doing grad admissions.
Fair enough, Anya. I guess I was thinking that there may be utility in getting applicants to confront the issue as they prepare their materials (e.g., to be careful in avoiding AI in their writing sample or research proposal, so to not contradict themselves in the AI statement) or to choose to lie or be deceptive, which may help push them to consider if they’re really cut out for grad school. That is, even if there’s not much value for the department, there may be value in nudging applicant behavior?
If you still don’t like that, what about this: instead of a separate statement about LLM ethics, grad schools can follow the norm in academic publishing to simply require applicants to disclose the extent of AI use, if any. This would apply to their writing samples, research plan, and other statements/work they submit.
Maybe grad depts are already doing that? I don’t know since I’m out of that loop. If so, what are they seeing?
Anyway, the revised idea really wouldn’t require more work from the applicant–just a few more sentences–and it’s nothing out of the ordinary, since it follows an established norm…
No more statements! Applicants have enough nonsense to wade through already, and admissions committees have enough stuff to read. Just ban AI usage on admission materials. No matter what anyone thinks about whether and how seasoned professionals should use AI, I think it’s clear that being completely reliant on it before even beginning your graduate education will leave you in a position where you lack the capacity to properly evaluate AI output, so I think it’s also reasonable to expect applicants to be able to produce at least their application materials without AI.