A team of researchers conducted a three-year study of philosophers’ judgments about the philosophical capabilities of generative artificial intelligence. Their finding: “As participants became more familiar with generative AI and increasingly recognized its formal competence, they simultaneously sharpened the boundaries they drew around what they regarded as genuinely philosophical thinking.”

The study, “Can Intelligent User Interfaces Engage in Philosophical Discussions?: A Longitudinal Mixed-Methods Study of Philosophers’ Judgments” was published earlier this year in the Joint Proceedings of the ACM Intelligent User Interfaces (IUI) Workshops 2026.
It was a small study in which the participants, 16 philosophy scholars, were asked to “assess anonymized philosophical responses and to reflect on the role of generative AI in philosophical discussion.” They were interviewed over the three year period about whether and how their attitudes about generative AI in philosophical discussion changed. That was the qualitative part of the study. The quantitative part involved the subjects rating their familiarity with generative AI and registering their evaluation of the philosophical capabilities of generative AI.
The authors, Yibo Meng (Tsinghua University), Bingyi Liu (University of Michigan), Lyumanshan Ye (Shanghai Jiao Tong University), and Yan Guan (Tsinghua University), summarize and discuss what they learned:
The qualitative findings reveal a longitudinal shift that is not merely attitudinal, but philosophical in nature. Over three years of sustained engagement, participants moved from initial resistance, through instrumental acceptance, to a principled critique of generative AI grounded in questions of understanding, embodiment, and responsibility. Importantly, this trajectory does not reflect growing pessimism about the technology’s performance. Rather, it reflects a deepening reflection on the nature of philosophical activity itself. As participants became more familiar with generative AI and increasingly recognized its formal competence, they simultaneously sharpened the boundaries they drew around what they regarded as genuinely philosophical thinking.
At the same time, the quantitative findings complicate this trajectory. Although participants consistently denied generative AI philosophical standing in their interviews, blind evaluations indicate that they nonetheless attributed to AI forms of originality typically associated with human philosophical work. From a human–AI interaction perspective, this divergence reflects a failure of expectation calibration, whereby stable normative beliefs are not updated in line with evaluative performance as system capability improves.
One possible interpretation of this: what’s going on is defensive goalpost-moving and stubborn inconsistency. In other words: philosophers are people, too. The authors suggest a less cynical alternative: that the study “prompted renewed attention to those dimensions of philosophy that participants regarded as irreducibly human—such as embodied experience, value-laden judgment, historical situatedness, and the willingness to question one’s own conceptual frameworks,” and thus served as “an occasion for [the] rearticulation” of philosophy.
Perhaps a little from column A and a little from column B?
The full study is here.
(via David Strohmaier)


I’d be curious to hear more about who the evaluators were in the blind evaluation tasks. The human contributors are described as “philosophy PhDs and professors”, whereas the human evaluators are merely described as “philosophy students and scholars”. It would be an odd choice to have evaluators with less training than contributors.
The quantitative data presented in section 4.2 is also remarkably sparse. There is, as far as I can see, no data presented on human performance on the evaluation tasks besides the single statement that the mean ratings across all six dimensions were lower in 2025 than that of the AI responses. Is the performance data presented in some companion piece somewhere?
It will be interesting to see how these results compare when the ongoing PhilosophyBench project finishes.
The excerpt read like Claude to me so I put the full paper into Pangram, which thinks it’s 59% AI written (qualitatively matching my own sense on a quick read).
No particular implication intended about the substance of the research.
This is a case where both based on the subject matter, and based on the fact that the authors mostly seem to have Chinese names and be based at Chinese universities, there could well be AI use either on the more substantive side or on translation and language polishing.