The Benefits of Studying Philosophy: Not Just Selection Effects?
Philosophy majors do well according to various measures of critical reasoning and intellectual habits. But is this an effect of studying philosophy, or an effect of students already talented in these ways choosing to major in philosophy?
A new study suggests that while there are indeed selection effects, there is some “initial and suggestive evidence” in favor of the claim that majoring in philosophy makes students, in general, better thinkers.
The study was conducted by Michael Prinzing (Baylor) & Michael Vazquez (UNC) as an informal report for the Executive Committee of the American Philosophical Association (APA). The research was funded by an APA grant.
They write:
We find clear evidence that philosophy majors enter college with more open-mindedness, curiosity, and a stronger tendency to be intellectually rigorous and reflective than non-philosophy majors. In addition, although students generally show some growth in these traits, philosophy majors tend to [show] more growth than non-philosophy majors in at least some of them (indexed by the Habits of Mind scale). Although this difference in growth is not drastic, it remains noteworthy and suggests that philosophical education may uniquely enhance certain intellectual virtues.
Here is a breakdown by specific metrics tracked by the Habits of Mind scale of how philosophy majors and other students change between their first year and senior year of college:
And here is the breakdown for specific metrics that fall under “Pluralistic Orientation”:
The authors cover other topics in their report, including common subjects chosen by students double-majoring in philosophy, and the effect of demographics on the tendency of students to add or drop the philosophy major.
Commenting on the value of this research, they write:
The case for philosophy’s place in the core curriculum could be substantially strengthened by finding ways to articulate its value in terms of quantifiable and communicable outcomes. Yet, even if the data are not so boosterish, it is nonetheless important for us to understand exactly how philosophy coursework impacts students. Indeed, perhaps we should be more interested in the empirical data if these metrics are not so favorable for philosophy, as this would help to to know what kinds of claims we are entitled to make on its behalf and how we might improve.
The whole report is here.
Thanks to the authors for undertaking this impressive empirical research!
Maybe the figures can be posterized for the propaganda arm of Daily Nous ( https://dailynous.com/value-of-philosophy/ ) with attribution?
I pored over the graphs struggling to find evidence for the claim you bold-faced “philosophy majors tend to [show] more growth than non-philosophy majors“. Your bold-facing left out the crucial delimiter “in at least some of the [measured traits]”, which completely alters the content of the claim.
By my assessment of the graphs:
Overall, this looks to me like mostly just a selection effect, with a mixed bag of departures that mostly have to do with interactive discussion, revision, and disconnection from scientific evidence. I guess it’s technically true that this means more growth than non-majors in a few cherry-picked areas, though leaving out the cherry-picked delimiter in one’s bold-facing makes this quite misleading, and the overall pattern looks to me much more like random variation around a mere “selection effect” than it does like philosophy majors tending to show more growth than others.
If you look at Figure 1 in the paper, it reports that philosophy majors showed more growth in habits of mind than non-philosophy majors. This is not mere selection effect since we are comparing change in growth. Are the cherry-picked? I don’t know much about the measurements used, but there are reasons to believe the results were not cherry-picked because 1) habits of mind is an existing scale–not something the authors cooked up ad hoc for their study, and 2) the authors reported null results (pluralistic orientation measure) which they would not do if they were seriously cherry-picking.
As far as I can tell, Figure 1 in the paper is the same thing as Figure 4 shown in the original post above, just averaged together into a single graph. You’re right that it shows a slight overall average gain for philosophy majors across all the “habits of mind” traits, but that is driven by the specific factors I noted above, with the magnitudes of philosophy majors’ relative gains in (a-c) interactive/revisionary skills slightly outweighing their relative loss in (d) scientific literacy.
It may be worth emphasizing that a large portion of this difference is driven by (a) non-philosophy majors stopping asking questions in class as they age much more than philosophy majors do, which is really hard to construe as the philosophers experiencing more “growth”. An honest advertisement would say, “Come study philosophy: we offer much the same growth as other majors, but we won’t stifle your interest in asking questions as much as they do.” Yay philosophy!!!
I see your point that it seems misleading to claim that X improved Y when “Y” is operationalized in a very specific way that can be quite far from lay definitions. In this case, “growth” is operationalized as specific measures like habits of mind and its constituents like asking questions, etc–which is probably not the first thing people think of when thinking of “growth”.
Nevertheless, niche operationalizations is par for the course in social science research, so the current paper is not being uniquely misleading in this aspect.
Furthermore, and more importantly, I think that we should not be expecting philosophy to provide a holistic growth advantage over other disciplines to begin with. After all, why should philosophy have such special powers? Much more believable and reasonable is that philosophy has specific growth advantages, which is precisely what the study found
Isn’t it also possible that philosophy tends to attract students who are more likely to continue asking questions (grow) more than other majors, in which case the major itself may not have any causal impact?
PS: others made similar points in the thread.
You should just read the article because this comment misunderstands several items.
The article is quite clear that the data show selection effects among these measures, but they also show treatment effects because they show that on several metrics philosophy majors increase their self-report of these virtuous habits more than non-philosophy majors.
The conclusion is simply not valid because it overlooks the scenario that the same self-selected people would have made the same progress had they changed their major to something else. Therefore, the control (other non-Phil majors) cannot count as real control since we are comparing apples with oranges. To establish causation, you would look at those who started out as philosophy majors and changed their major. You would also need to eliminate other confounds (eg., grades, which is a loose reflection of intelligence, conscientiousness, neuroticism, etc. since some people left philosophy because they are not cut out for it). Only then are you comparing apples with apples.
Interesting contribution! I like the focus on intellectual virtues, rather than simply economic outcomes. I’m glad this work has sparked interesting conversations and think it will be useful to build on. Hopefully in follow-up work it will be possible to
1) isolate the effect of philosophy over and above selection by exploiting some instrument or random assignment (such as lotteries for freshman seminars or majors) — currently, as other commenters have argued, these results could be explained by philosophy majors having a higher propensity for growing, as well as initially starting with, those habits.
2) where feasible, use measures that reflect observed behavior (recognizing that this is much more difficult to do!)
3) most importantly, make fair comparisons across disciplines. These data are consistent with the claim that the habits of mind questionnaire measures the degree to which one does challenging university-level writing. The item-level analyses were striking: the biggest interaction effects (i.e., the ones that the authors argue are less susceptible to selection) can be found in the items concerning writing and editing habits, skills that philosophers would be expected to use often and engineering students much less so. The lollipop chart that separates by specific major supports this interpretation, philosophy/english majors are at the top and engineering at the bottom. The slopes look pretty similar by discipline.
I wonder as well what the practical implications are of around a 0.1 to 0.2 sd increase in habits of mind. Are there critical thresholds that students cross (e.g., by senior year, we can be sure philosophy equips students to do their intellectual due diligence)?
Also, I haven’t thought deeply about this, but I’d love to see the results of a multilevel GLM: time-points within students within institutions split by philosophy vs non. The degree of class imbalance (~1,600 philosophers in a sample of ~123,000) coupled with heterogeneity by university could pose issues for estimating interactions with a simple 2×2 ANOVA that may not be resolved by the relatively large sample size.
This is an important addition to the conversation and I hope it sparks further research to help identify how exposure to different disciplines and teaching methods shapes the development of intellectual habits!
Would any of this impact why philosophy departments are being cut? You need a better argument than this.