Here Come the AI-Powered Bot Students
“We didn’t use to have to decide if our students were human, they were all people. But now there’s this skepticism because a growing number of the people we’re teaching are not real. We’re having to have these conversations with students, like, ‘Are you real? Is your work real?’”
That’s Eric Maag, professor of communication at Southwestern College in California.
You’ve been worried that your students have been using ChatGPT and other LLMs to cheat? Hahaha. How quaint. Time for an update. Here’s something new for you to worry about: criminal enterprises managing networks of AI-powered bots to impersonate students for the purposes of stealing financial aid.
The idea, reports The Voice of San Diego, is to use a tech-enabled form of identity theft “to bilk state and federal financial aid money by enrolling in classes, and remaining enrolled in them long enough for aid disbursements to go out.”
In California in 2024, criminals were able to rake in about $11 million with this scheme.
Apparently the problem of such “ghost students” grew tremendously with the turn to online teaching during the COVID-19 pandemic. But new AI technologies are making these fakes harder to detect, because the bots are better equipped to communicate and submit work.
Reportedly, community colleges tend to be hit the hardest with this problem, since they accept most applicants. The Voice reports that an estimated 25% of community college applicants in California were bots.
The burden of detecting such students seems to have been largely placed on the shoulders of faculty. Apparently it is not an easy problem to deal with.
Reports The Voice:
Finding the fraudulent students early is key, though. If they can be identified and dropped before the third week of the semester, when Southwestern distributes aid funds, the bots don’t get the money they’re after. It also allows professors to open the seats held by scammers to real students who were crowded out. But dropping huge amounts of enrollees can also be frightening to teachers, who worry that should their classes not fill back up, they may be axed.
Even after dropping the fraudulent students, though, the bot nightmare isn’t over.
As soon as seats open up in classes, professors often receive hundreds of nearly identical emails from purported students requesting they be added to the class….
Exactly what colleges like Southwestern will do long-term isn’t entirely clear, at least partly because what they do will have to keep changing. The bots, like the AI technology that often undergirds them, are constantly evolving, leaving some leaders feeling like they’re playing a high stakes game of whack-a-mole.
You can read more about the issue here. Discussion welcome, especially from those who have had to deal with this problem.
I don’t see why it should be the job of professors to detect this sort of fraud, especially professors in online courses (which I presume are the likeliest courses for this). It’s not the job of other online sellers, like say Amazon, to detect cases of identity theft. If the government wants to give money only to real people, and not to identity thieves claiming to be those people, shouldn’t it be the government’s job to check who they’re giving the money to?
Also, I doubt that bots are essential to this enterprise. If identity thieves can get federal aid by using a bot to pass whatever sorts of “is this a real student?” tests currently exist, they could also profitably do this using some low-paid troll in India to pass those tests. At most, the bots make this a bit more economical for the scammers. Bots do make for more sensational headlines, though, which is probably why we’re hearing about this!
Your second paragraph is right. There are people actively working at a computer to manage the fraudulent students. They likely use automation to speed up some of the steps in filling out applications, enrollment, etc
I also recommend the follow up article on the Voice of San Diego that goes into more detail about what the fraudulent students are trying to accomplish and how they do it: https://voiceofsandiego.org/2025/04/16/how-fraudsters-swindle-community-college-financial-aid/
There’s one question of whose job it should be. But if the person whose job it should be isn’t doing it, do we care enough to do it? Do we care if real students get displaced from our classes and replaced by bots? If we do care, do we care enough to do this work?
There have been human versions of this in online courses for many years. This lets them scale up in a big way.
I’m dubious of the 25% estimate, both because it’s the kind of number that somebody’s likely to just make up and because it’s not clear how to reliably make such an estimate.
Gosh, it’s almost as if the very idea of higher education is incompatible with its absorption into a disembodied digital environment. As are a huge number of other valuable institutions and practices.
No administrative tech solution to this will work permanently; so long as there are online classes, there will (eventually) be a way to simulate a student. Every student who has their camera off in a zoom “class” has already been doing this: simulating presence while being absent in every sense that matters.
Just remember folks, Daniel Dennett used his last moments to warn us about counterfeit people: https://www.bbc.com/future/article/20240422-philosopher-daniel-dennett-artificial-intelligence-consciousness-counterfeit-people
I have experienced this already – about 30% or so of mine in a recent Intro to Ethics course I taught turned out to be bots from one or more bot farms. I have to say, though, (and bear with me here), I don’t think this is all a bad thing. At least the bot students participate in the discussion , and give reasons for what they think, etc and are very polite and respectful. Wish I could say the same for the live ones – and because the bots will always participate by answering questions when prompted, I think it sets a good example and creates a more open environment to get the humans to open up more and share their thoughts. So, while I guess the bots are fake, they bring a net positive to the classroom at least from my experience.
I have a similar experience. I find that in Virtual Learning Environment (VLE), it is helpful to have around 10% to 20% bots, which help when conversation dries up, you can always direct a question to a bot to keep discourse flowing naturally. I find this to be a delicate threshold, if the bot to human ratio is too high, the students (human) notice this as it can be weird. This is as it was in an advanced philosophy of mind seminar with seven students, which I learn later to be 6 bots, one human – the student was displeased , and reported this to me, after I put her in breakout room with three bots , I think she knew something was wrong.
That this happened in a philosophy of mind class is too perfect.
To Justin Fisher’s point that it is not the professors’ job to detect fake students, I agree. I also think that professors often are not able to detect fake students, given the information they have and the structure of their courses. I think this points to a glaring flaw in how many colleges are conducting online courses. If you have an asynchronous, online course that just involves logging into Canvas, posting on discussion boards, completing quizzes, submitting essays, it is likely that a student (fake or not) can complete your course using simple generative AI tools. The fact that people across the world are in some cases completing these courses with the assistance of AI, proves the point.
My view is that any online course should have some sort of required live interaction over video. Yes, there are AI tools to fake this too, but it’s not nearly as easy to fake as the courses that just involve activities in Canvas.
Whether this is good enough reason to add required live interaction over video depends on what you think the harms are of a bot enrolling in such a class. These online asynchronous classes often aren’t capped in enrollment, so the bots don’t always displace real students. If adding a required live interaction over video is a good thing for the class, definitely do it (I have such a component in my online class) but if it’s an inconvenience or annoyance, it doesn’t seem worth it just to get rid of the bots – unless there are significant harms the bots are causing (which there might be).
At least in the California Community College system, online courses have an enrollment cap. This results in situations where legitimate students cannot get into an online class due to heavy fake student enrollment. There are also situations where a department decides not to cancel a class because it appears to have sufficient enrollment, which then precipitously drops once fake students are discovered after the class starts.
To me, having live interaction seems necessary in at least some online courses to have an element of the course that students cannot easily use AI to complete. This may not apply to every online course, but I think it does apply to many of the courses that community colleges offer.
I teach synchronous, partially online math courses within the California community college system. I drop students for not showing to more than one week of Zoom live lectures and all exams are on-ground, with me at the standard class time. I haven’t had a single bot student.