Last week I posted about PhilLit, a new AI research tool for philosophers that finds and summarizes philosophical writing. The post generated a lot of comments, which prompted one reader to run a little experiment.

The experiment consisted in having ChatGPT read the post, predict the comments, read the actual comments, and explain why its prediction did not match reality. It was conducted by Kelly Truelove and can be read in its entirety here.

The result of the experiment included some observations about philosophers and AI right now, which I think are worth sharing. It’s not that they’re earth-shattering insights. Rather they’re like a mirror—you may not know exactly how you look before you glance at it, but you’ll recognize yourself when you do.

So what did ChatGPT predict philosophers would say about PhilLit? Well, it’s a purpose-built tool and the AI assumed we’d largely assess it as such:

Among professional philosophers who read Daily Nous, I would expect a mix of curiosity, cautious optimism, and skepticism. Many will find the idea appealing, since keeping up with adjacent literatures is genuinely difficult, and a tool that promises verified citations and structured overviews targets a real pain point in research practice. Some, especially those who already use AI tools experimentally, will likely be eager to test it and compare it to their own knowledge of a field.

At the same time, there will likely be strong skepticism about reliability, philosophical sensitivity, and hidden biases. Philosophers are trained to worry about subtle mischaracterizations of positions, selective framing of debates, and the normative assumptions built into any system that “maps” a literature. Even if citations are verified, critics may question whether the tool captures the dialectical structure of debates accurately or whether it privileges certain journals, traditions, or methodologies.

Finally, I would expect meta-level concerns about professional norms and incentives. Some may worry that such tools could deskill graduate students, flatten intellectual diversity, or encourage superficial engagement with literatures. Others will see it as analogous to earlier research technologies—like PhilPapers or online databases—arguing that, if used responsibly, it could simply become another standard aid in serious scholarship.

But that wasn’t quite the whole story. After reading the actual comments, ChatGPT says “The comments reveal a deeply divided community, with reactions ranging from enthusiastic support to existential alarm” and provides a useful summary. Some highlights:

A prominent strand of criticism frames the very idea of AI-assisted literature review as a symptom of disciplinary decay. For these commenters, philosophy is intrinsically bound up with reading, writing, and intellectual community; outsourcing these tasks threatens not just professional norms but the meaning of philosophical life itself…

[D]efenders argue that the dispute hinges on a deeper disagreement about philosophy’s purpose: whether it is primarily self-cultivation or the production of collective knowledge…

Some argue that refusing AI will be professionally self-defeating in a hypercompetitive environment, while others worry that AI use will further distort already perverse incentives.

ChatGPT concludes that, “Overall, the thread is not merely about a software tool but about the identity, purpose, and future structure of professional philosophy.”

What explains the gap between ChatGPT’s predictions and reality? The AI says:

The main gap stems from underestimating how symbolically charged AI has become within philosophy. I treated PhilLit primarily as a research tool likely to provoke methodological debate, but many commenters interpreted it as a proxy for much larger anxieties about academic identity, neoliberal incentives, technological domination, and even the meaning of intellectual life. In other words, the tool functioned less as a discrete proposal and more as a lightning rod for preexisting tensions.

A second gap reflects the culture of Daily Nous itself. It is not just a venue for technical evaluation but a semi-public space where professional philosophers perform and contest metaphilosophical self-understandings. That makes it especially likely that practical questions (cost, accuracy, workflow) will quickly escalate into normative and existential ones.

Pushed a bit on this, it adds:

on Daily Nous, AI isn’t just “new infrastructure”; it’s already a contested symbol tied to authorship, legitimacy, disciplinary self-image, and the fear that the activity philosophers most prize (careful reading and writing) is being hollowed out. When a topic has that kind of identity-threat valence, people don’t stay at the level of “does it work?”—they jump to “what does this mean about us?”

I also underestimated it because “symbolic charge” is highly time- and context-sensitive, and I answered without checking the local conversational temperature of that specific thread. The comment section wasn’t starting from neutral; it was building on prior DN AI debates, on widespread stories about hallucinated citations and academic misuse, and on a background sense that the profession’s incentive structure is already strained. In that setting, PhilLit reads less like a helpful hack and more like a referendum on whether philosophy is becoming a production line.

Finally, I defaulted to assuming that professional philosophers would normatively separate “tool use for drudgery” from “doing philosophy proper” more cleanly than they did. The comments show that many readers reject that partition: for them, literature searching, reading, and community-facing uptake are not peripheral chores but constitutive of the practice. If you don’t share that boundary, then AI assistance isn’t marginal—it’s symbolically central, and my expectation model missed that.

You can read the whole discussion here.