How to Show that Studying Philosophy Improves Thinking Skills
How would you determine whether studying philosophy improves critical thinking?
[Max Ernst, “The Gramineous Bicycle Garnished with Bells, the Dappled Fire Damps, and the Echinoderms Bending the Spine to Look for Caresses”
Prinzing and Vazquez incorporated these findings with further research they conducted and earlier this summer and published an article about them in the Journal of the American Philosophical Association, discussed here back in July.
They’ve now, smartly, written an op-ed about their research for The Conversation. Many news venues republish articles from The Conversation, and the piece by Prinzing and Vazquez is being widely reprinted, usually with a headline that includes some variant on the phrase “studying philosophy makes people better thinkers”.
This is great publicity for philosophy. Philosophy departments should be sharing this article far and wide—posting it on their websites, emailing it to their students, printing out copies for major fairs, etc., not to mention showing it to their deans, provosts, and presidents.
Philosophers and philosophy departments have long claimed that studying philosophy improves one’s critical thinking skills. Yet apart from inconclusive correlations between majoring in philosophy and standardized test scores, there has been precious little offered in the way of empirical evidence. (One exception to this, from a few years ago, is “Philosophy Majors & High Standardized Test Scores: Not Just Correlation” by Thomas Metcalf of Spring Hill College.)
As valuable as this kind of work can be—especially when philosophy departments are worried about cuts or elimination—some people have expressed some dissatisfaction with the way Prinzing and Vazquez go about making their case. (A brief explanation of their approach and findings is below.)
In a conversation on social media about that, Chris Surprenant (University of New Orleans) asked the following question, which seemed to me worth putting to all of you:
Let’s say you had an unlimited budget and access to students. How would you design a project that tested whether or not taking philosophy courses makes people better thinkers?
A brainstorming session on this could be useful. With luck, there will be some worthwhile, practicable ideas that, with further luck, could be funded and tried. Let’s hear your suggestions!
Information about Prinzing and Vazquez’s research:
Prinzing and Vazquez look at traits indexed by the Habits of Mind scale (a measure of intellectual dispositions) and performance on the GRE and LSAT. For the Habits of Mind scale, they rely on student self-reports. Here are the results as reported in their October 2024 report:
For the standardized tests, they control for selection effects by comparing SAT scores (taken before college) and GRE and LSAT scores (taken during or after having majored in a topic) and seeing which majors showed the greatest improvement:
Baseline-adjusted average scores on self-report measures for specific majors. Points and error bars indicate estimated marginal means with 95% confidence intervals derived from mixed-effects regression models. Philosophy is highlighted with red. From Prinzing & Vazquez “Studying Philosophy Does Make People Better Thinkers” (2025).
More information here and here.
A massive twin study
I’m not sure where this link should go, but it seems to be broken.
My department has been tracking this kind of thing internally for years in our Intro to Ethics course. If anyone is interested, I’m glad to correspond.
[email protected]
What are you guys tracking and how are you tracking it?
Essentially, growth of critical thinking skills in relation to moral problems.
I feel the title to this post is in some danger of begging the question…
Sometimes there’s a fine line between hopeful and question-begging, I’ll admit.
Anecdotally I’ve had a ton of former students report that they benefitted greatly from philosophy classes. None that I know of have said they didn’t. I don’t think this this begs the question at all.
I am pretty inclined to think it’s true. But you can’t make a serious attempt to scientifically study whether X causes Y if you are presupposing that X causes Y. (If you already know that X causes Y, why are you studying it in the first place? And if the answer is: to persuade skeptics, then why should the skeptics find your evidence persuasive if your methodology assumed the conclusion?)
Also: some students benefit from classes and some don’t, whatever the subject; and the ones who benefit are probably more likely to tell the professor than the ones who don’t; and professors plausibly are more likely to remember some student comments than others. The plural of ‘anecdote’ is not ‘data’.
I didn’t say scientifically–I said by my own experience. And anonymous student evals tend to draw out negative comments. I had few out of 7000 students say that the classes were useless, and a majority said that they were good or useful. I’m just saying.
Or, at the very least, you need to have everything reviewed by people who are neutral to the discussion.
There’s nothing wrong with someone saying, “I believe X causes Y, let’s design an experiment that proves that,” and then have the protocols signed off on by various 3rd parties with none of these hopes or biases.
But, yes, in practice an article coming out in the APA’s journal that philosophy is really useful isn’t going to persuade skeptics.
That’s quite a fascinating response to the post.
However, I think that most of us can vouch for there being a connection between studying philosophy (X) and thinking critically across a range of areas, beyond philosophy (Y). A charitable reading of the post is that it’s just calling for us to find ways of studying this relationship, so that we can find more robust evidence for communicating it to people who haven’t already been exposed to philosophy in a way that personally reveals the link.
If it turned out that some of the studies confirmed the null hypothesis, we should adjust the way we speak about the connection. But should we abandon the hypothesis that most of us intuitively support? If some studies showed the philosophy made you dumber, what would we make of that? Would we say to our deans that they’re silly for keeping us employed?
Perhaps the way to conceive of this research, and its value to philosophy, is two-fold
(1) Can help us refine our pedagogies, by clarifying which techniques of teaching improve critical thinking.
(2) Can help us to advocate for philosophy
“If it turned out that some of the studies confirmed the null hypothesis, we should adjust the way we speak about the connection. But should we abandon the hypothesis that most of us intuitively support?”
Yes, I think we should abandon intuitively plausible hypotheses as and when they are clearly contradicted by scientific evidence (and, more plausibly, we should reduce our confidence in hypotheses insofar as good-quality science fails to support them). Is this controversial?
“If some studies showed the philosophy made you dumber, what would we make of that?
Depends on the quality of the studies, but if good-quality scientific research supports the claim that philosophy makes you dumber, we should we worried, and should urgently try to replicate it.
Would we say to our deans that they’re silly for keeping us employed?”
I’m not sure saying “scientific evidence supports the claim that studying philosophy makes you dumber, but our intuitions say otherwise, so we are going to ignore the evidence” is a very good look.
I am reasonably confident that study of philosophy is good for critical thinking – at least in the context where I’ve worked extensively enough with students to have enough to go on. (Various grad programs; UGs at Oxford). But the evidence for this is shakier than we might like, and we should keep an open mind.
“Yes, I think we should abandon intuitively plausible hypotheses as and when they are clearly contradicted by scientific evidence (and, more plausibly, we should reduce our confidence in hypotheses insofar as good-quality science fails to support them). Is this controversial?”
No, it’s not controversial. In fact, it reveals the power of critical thinking (and hence, philosophical teaching and research being done well)! But there’s room for disagreement about what kind of evidence would be sufficient for abandoning the hypothesis. Keeping an open mind is crucial. But if we’re so open-minded to the idea that philosophy is bad for critical thinking (as opposed to not as good as we hope), why would we be doing it? Is “the power of critical thinking” just a collective mantra we use, so that we’re not overcome by guilt for not putting our labour into something else? There’s definitely something self-serving about some of it. But that’s not the story you get from philosophy in schools and prisons, and among disadvantaged groups.
Overall, I guess what I’m saying is that it’s not unreasonable to hold a high degree of confidence in the relationship between X and Y for cases where philosophy is being taught well. Therefore, one way to read “bad results” is just as guidance about how we should teach philosophy, or at least the aspects of it that are purely devoted to critical thinking.
The APA is hosting an online panel on this paper with the authors and a couple of commentators later this week: https://www.apaonline.org/events/EventDetails.aspx?id=1975213
(Full disclosure, I’m one of the commentators.)
Some older empirical evidence, together with some inconclusive causal modeling: Livengood, J., Sytsma, J., Feltz, A., Scheines, R., & Machery, E. (2010). Philosophical temperament. Philosophical Psychology, 23(3), 313-330.
replicated by Nick Byrd. See also Byrd, N. (2021). Reflective reasoning & philosophy. Philosophy Compass, 16(11), e12786.
Do you choose philsophy because you are attracted to that way of thinking? As a child I was considering the world around me, and those in it, long before I knew the term philosophy. I am not sure that Thales used the term philosophy when he posed the basic question: What is it? He did go on to say that in seeking answers we should use our rational capacity alone, and not resort to mythical or supernatural explanations. Piaget considered children little scientists. I think of them little philosophers, voicing the non metaphysical Why question pretty much as soon as they speak. Sadly, the world in general, and more particularly the education system discourages questioning, which sadly gives the lie to the opening sentence to Aristotle’s ‘Metaphysics”.
I am unconvinced studying philosophy at university will necessarily make you a better thinker. Much more effective would be discussion at home around the table, and out in the world at large. It is there where children can be encourged to question and think. Leaving it to late teens at university is far too late.
Why would that be “much more effective”? Isn’t “discussion at home around the table” often narrow and parochial? When do children have discussions “out in the world at large”? What evidence is there that late teens at university can’t become “better thinkers”? Piaget never said any such thing–in fact, just the opposite!
There’s some good, although probative, evidence for the positive impact of philosophical instruction in primary and secondary schooling. But it’s been awhile since I’ve looked into this, so I’m not sure what the state-of-the-art is.
https://dailynous.com/2018/01/17/evidence-supporting-pre-college-instruction-philosophy/
Thanks, Justin, for putting this up. I wanted to give it a day before I chimed in.
So part of the challenge is when the inferential data is good enough to overcome any concerns that correlation is not causation or that there is some problem with the sample. So, for example, a fairly straightforward concern about using LSAT scores is that people from certain majors aren’t likely to take the exam as seriously because they have other job prospects. But I’m not sure that’s too much of a concern here — do English majors really have more job prospects than philosophy majors? Seems unlikely.
Another concern that was raised was that the data was self-reported. I’ve been engaged in very large projects over the past 5 years that have all focused on sampling undergraduate and high school students. One of the things we can confirm fairly conclusively is that students will lie about their academic performance. A class of 30 will tell you that they’re all A and B students, but the transcripts will say otherwise. So, with the self reporting of scores, we could be concerned that the students are misreporting their numbers, but this would be a concern only if we also think that philosophy students misreport more (either more frequently or misreport higher than accurate scores) than students from other majors. I’m not sure there’s any reason to think this.
With all of that said, there does seem to be something not quite right about the conclusions being drawn here, but I also can’t put my finger on exactly what that is (and others seem not to be able to either, so maybe the problem is with us and not the conclusions being drawn). One thing I’m fairly certain of is that the APA journal was probably the worst outlet to publish this article in, at least from the standpoint of optics. *Of course* the APA would want to show that taking philosophy classes makes people better in all sorts of ways.
I’m still looking to think through what kind of study could be done to conclusively prove or disprove this claim if you had access to an unlimited budget, an unlimited number of college students, and access to grade and whatever other records you think you’d need. I don’t know how you could attribute any gains (especially anything long-term or retained) specifically to taking philosophy courses, but it would be nice if this could be done. Open to ideas here.
If the budget is really unlimited, something like the following would possibly work:
A variant of this proposal:
This might work if you’re not worried about impact over time and also think that one course is enough to have this kind of impact. The suggestion below seems better.
Ex hypothesi I have unlimited money, so I may as well do both. You might well be right that single courses don’t have much impact, but that too is presumably something we’d like concrete evidence for.
(And: If single courses do have no impact, that has some implications for how to ethically market gen ed offerings.)
I asked a social science faculty member what she thought of the study cited elsewhere, and given her response below, I’m not ready to cite it as a set of reasons one should study philosophy:
Skimming it, my first reaction is always, “That’s cute; non-scientists trying to do science,” so that should set the stage. I tried accessing the article itself, but it’s behind a paywall. I would highlight this conclusion passage:
All told, we looked at test and survey data from over 600,000 students. Our analysis found that philosophy majors scored higher than students in all other majors on standardized tests of verbal and logical reasoning, as well as on self-reports of good habits of mind, even after accounting for freshman-year differences. This suggests that their intellectual abilities and traits are due, in part, to what they learned in college.
First, until I see something like a “Methods” section, I’m going to be leery. Second, until I see something in that Methods section along the lines of “students were randomly assigned to conditions…” I’m going to be leery. Third, when I read stuff like “This suggests that their intellectual abilities and traits are due, in part, to what they learned in college.” I’m really leery, because if you call something a trait, kinda-by-almost-universal-consensus most professionals would take that to be a shorthand for “personality characteristic” or “thing about the person” or “something inside rather than acquired.” So if you’re making the argument that learning how to reason, learning about logic, learning how to do a Venn diagram, being called upon consistently to support arguments, etc., makes you a better reasoner…sure. why wouldn’t it? As my undergraduate mentor might have said, “You drop a penny, it hits the floor.” You know? Could have guessed that one all on my own. So, does instruction improve “intellectual abilities” (such as being a better test taker, scoring higher, applying knowledge, whatever they measured)? I’d sure as *#@$##@& hope so, otherwise we’re in the wrong business! Does the acquisition of knowledge/skills via education change “intellectual traits?” Doubt it. That’s sloppy language (see (“cute when non-scientists try to…” part). And more to the point: Despite controlling for “Freshmen whatever” that they mentioned, how do we know that smarter people aren’t both A) attracted to majors that involve rigorous thinking (e.g., philosophy) and/or B) are better at taking tests….because they’re smarter people to begin with (see “until I see something about random assignment…” part).
Now, again, I haven’t read the whole article, so maybe this lot were the philosophers who really, really, really understand research design and statistical analysis….the inheritors of Dan Dennett’s beard, shall we say, or the new breed of Patrick Suppes or Ilya Prigogine. (God help us if they’re acolytes of that goofball Thomas Kuhn!) But I doubt it. I’d wager it’s some philosophers who know a little about science, found a seeming gap in the literature, got access to a whole bunch of data, and thought they were being clever.
A final methodological note. When you just measure people — measurement without manipulation — you leave yourself open to a whole host of confounding variables. That’s what’s being done here; they measured pre-existing characteristics, such as the choice of major. There are ways of statistically controlling for some, perhaps most, of those confounds, but you never know for sure. That’s what I think we have here: Students self-select into being philosophy majors, they learn philosophy stuff, then they do well on measures that should be related to having learned that philosophy stuff. That’s science…kind of; the sociologists would giggle with glee and maybe even pee their pants a little! But real science, the cause-N-effect type, requires manipulation of an independent variable, control of confounding variables, and random assignment to conditions. We call that “an experiment.” So if you want to show that being a certain major in college makes you better at reasoning, you’d randomly assign students to one of various majors, control for…oh, let’s say 1,000,000 things (such as prior educational background, baseline intellect, motivation, family income, quality of instruction, mental health, etc., etc., etc.), expose one group to philosophy instruction (and not expose any of the other groups to the same), then measure a lot of things — lots of dependent variables — after some specified time (and, ideally, at several points along the way in this longitudinal design I’m sketching). “But…but…that seems really difficult, if not impossible, to do!” Yeah. So instead, you’re left with studies like these; cute finding, probably I don’t doubt it, but not sure it explains anything…which is what actual science is really after.