Does Philosophy Improve Critical Thinking?
Following on the heels of last week’s discussion of non-philosophers teaching critical thinking, the Chronicle of Higher Education drew attention to a meta-analysis of studies about whether colleges succeed in teaching critical thinking at all. As it turns out, they do:
Students’ critical-thinking skills do improve in college. The difference is comparable to a student whose critical-thinking skills start at the 50th percentile and, after four years in college, move up to the 72nd.
The Chronicle also reported:
The study’s authors found no differences in the critical-thinking skills of students in different majors.
Wait. What? How could philosophy majors not be leaving most others in the dust? So I looked at the study, “Does College Teach Critical Thinking? A Meta-Analysis,” by Christopher Huber and Nathan Kuncel (Minnesota), in Review of Educational Research. Here’s the relevant passage on philosophy:
The current evidence on differences between majors is inconclusive. Pascarella and Terenzini’s (2005) review failed to find strong evidence for differential gains across majors. By contrast, Ortiz’s (2007) meta-analysis suggests that philosophy students may learn more critical thinking than other students. Ortiz estimated gains of 0.26 SDs per semester for philosophy students compared to only 0.12 SDs per semester for other majors. However, the number of pure philosophy samples in her analysis is small (k = 6), and the samples appear to be from unpublished studies. In addition, the confidence intervals for the philosophy and nonphilosophy effect sizes show substantial overlap. Ortiz herself suggested that the observed difference may simply be statistical noise. (pp. 3-4)
Assuming the meta-analysis was relatively thorough, it appears that philosophers lack good empirical evidence for what I take to be the widespread belief that majoring in philosophy is a superior way for a student to develop critical thinking skills. Was a study overlooked? If not, someone, get on this!
Of course, part of the issue may be disagreement as to what counts as “critical thinking,” which Huber and Kuncel discuss. There is some controversy over whether “critical thinking” should be conceptualized as
a broad ability to interpret information and approach problems correctly that can be applied across a wide variety of domains
or whether, since
the ability to reason and think critically is required for a broad range of tasks beyond analysis of logical arguments, such as “finding one’s way home, investing money, fishing, driving a car, doing sums, shopping, playing hopscotch, intelligent voting, building math models, writing poems, and countless other classes of activities” … the ability to think critically about such a broad array of domains is not well represented by any general skill (e.g., analyzing arguments), and therefore critical thinking ability is best conceptualized as domain-specific.
It appears that more work needs to be done before philosophers can claim the bragging rights to which we think we’re entitled.
(“Question Mark” by Kumi Yamashita)
The reason the “philosophy majors do well on the GRE and LSAT” line is suspicious is because the high performance may be due to the fact that philosophy attracts bright students. Isn’t the same thing potentially going on w/ critical thinking? In other words, the greater critical thinking skills of our students may just be due to the fact that they were critical thinkers to begin with and not because we made them better thinkers.
Can someone school me on what ‘k’ is? Is it just ‘n’? If so, the result is hopelessly underpowered and doesn’t even address the question of whether “pure” philosophy majors would outperform others on critical thinking.
Also, let’s say this metanalysis does find no significant differences in critical thinking between majors (with respectable power, etc.). This would be surprising given the LSAT, GRE, and GMAT data. So to say that this metanalysis is the end of the story would be myopic. The proper response might be to try to account for the difference between the well-powered, robust findings from graduate testing — on the one hand — and the metanalysis — on the other. Maybe the findings are in conflict, but one set carries more weight than the other. Or maybe the findings are actually orthogonal.
k is the number of studies in the metaanalysis—not subjects! 🙂
Right. I realized that moments after I posted my first comment (see my later comment(s)). Thanks.
I don’t know where to find it anymore, but wasn’t there some evidence that a course in logic boosts students’ critical thinking, while there’s no clear evidence of other philosophy courses doing so? I think it’s been mentioned a few times in comments here, but I can’t for the life of me find it.
Do you mean this on argument mapping? http://philpapers.org/rec/TWAAMI
You’re probably thinking of Ortiz, C. M. A. (2007). Does philosophy improve critical thinking skills? (Unpublished master’s thesis). The University of Melbourne, Victoria, Australia
https://sites.google.com/site/timvangelder/publications-1/does-philosophy-improve-reasoning-skills
It’s the Ortiz 2007 discussed in the paper.
Nevermind about ‘k’. It’s obviously not ‘n.’ I should have realized that.
More on Ortiz 2007: the authors provide more statistical description of the philosophy v. non-philosophy difference from Ortiz 2007 than what is above:
“Ortiz reported semester-long gains of 0.78 SDs in philosophy courses that incorporate large amounts of argument mapping practice.”
That description and the ones in the original post are consistent with a significant result. In fact, Ortiz does find that the gains in critical thinking (CT) are significantly better for people who take courses in philosophy v. those who do not (ES ≈ 0.3, p < 0.01). And Ortiz finds marginally significant differences in CT between people who've taken philosophy courses v. CT courses (ES ≈ 0.15, p < 0.05) — philosophy courses result in better CT. Interestingly, Ortiz also finds that philosophy courses aimed at CT produce better CT than tradition analytic philosophy courses (ES ≈ 0.25, p < 0.5). And philosophy aimed at CT that use argument-mapping produce the largest do much better than traditional courses in analytic philosophy (ES ≈ .45, p < 0.01). So courses featuring (i) analytic philosophy, (ii) CT, and (iii) argument mapping produce better gains in CT than courses featuring only one of two of these.
And sure, these results might be statistical noise (especially considering that the sample size on any given study in the meta-analysis was 26-77 — not nearly up to 50/cell minimums recommended by many social psychologists [and social psychology journals]), but the effect sizes and significance from Ortiz 2007 were easily up to the standards of social psychology at the time). Perhaps this is why Huber and Kuncel 2015 are not impressed by Ortiz 2007 findings. But I do not see anything in Huber and Kuncel 2015 that suggests we should disregard Ortiz findings — which address the effect of philosophical training on CT far better than Huber and Kuncel 2015 — altogether. Feel free to point me to something in Huber and Kuncel 2015 that conflicts with this.
And even if these results are significant, they don’t necessarily rule out the selection effect raised by recent grad. Until we randomly assign people to majors [which we probably shouldn’t do] and then control for the number of philosophy-ish classes from non-philosophy departments (like Ortiz does, i.e., code CT classes that employ the tools of philosophy differently than courses in analytic philosophy and differently than courses in both CT and philosophy, featuring argument mapping, etc.), we are in a poor position to understand the precise effect of training in philosophy on CT.
Some other potentially relevant studies: Livengood et al 2010 and Byrd 2014. I’d be interested in hearing about others relevant studies as well.
Thanks for posting this!
(sorry for typos)
Is it that k is the number of studies, while n would be the size of the population within an individual study?
That’s what I have inferred about k. And ‘n’ can be what you have stated. I have also seen people use it as the number of observations for a given statistical analysis (which can much higher than the number of participants in the population if each participant does multiple trials).
It’s unfortunate that such a claim is only justifiable via the gathering of statistical information.
Ever try talking politics with a student is business or engineering programs? There are generally (and I’m perfectly okay with generalizing for the sake of this argument) fallacies abound and a clear lack of thoughtfulness.
In the course of everyday interactions it seems to simply be pretty clear, without any gathering of statistics, that those with an education in philosophy are better at critical thinking. Seeking justification? Go to a social and political philosophy class during a discussion of Marxism and then go to a business class during a discussion of Marxism. I don’t think I need to say any more.
“Go to a social and political philosophy class during a discussion of Marxism and then go to a business class during a discussion of Marxism.”
That observation (even if true) is entirely consistent with critical thinking being domain specific. Students in the philosophy class are actually taught to think critically about Marxism. The business students are (perhaps) taught to think critically about something else.
It does seem likely, though, that some forms of critical thinking emphasize abstraction more and so are more portable across domains.
Also, there’s the selection effect problem with the business vs. philosophy majors. Spending a lot of time with the 2 groups only increases one’s suspicion that this is part of the story. Ever spent much quality time with business majors?
The purpose of this post is to enable me to receive notifications of follow-up comments by email.
I hate to see this discussion thread peter out so quickly. How about some thoughts as to ways of collecting data from one’s Intro Logic/ Critical Thinking courses that *might* contribute toward confirming or disconfirming the thesis that philosophy improves critical thinking? Here’s one that would increase an instructor’s workload only slightly. On the first day of class, ask the students to write one anonymized (identification by I.D. number only) paragraph defending or criticizing a particular view; collect, grade and keep the answers; and compare them with those they give to the same assignment, under the same conditions, on the last day of class. Repeat this data-gathering assignment each time the course is taught, and note any statistical patterns that might emerge. Just a thought.
Hi Adrian,
I know Michael Bishop teaches a philosophy slash critical thinking course at FSU and his students (as well as students in similar courses at FSU) are part of ongoing research about the outcomes of critical thinking courses. When the results are available, I’d be happy to send them to you if you email me: http://www.byrdnick.com/contact
Nick, great. I’ve just sent you the following message at your website:
Hi Nick,
Many thanks for your encouraging reply and kind offer. I’d be very interested in seeing the results of Michael Bishop’s research. Thank you so much for suggesting this.
Best,
Adrian
Another option is to administer a critical thinking test at the beginning and end of each semester. Two versions of the test are needed. Half the students receive test A at the beginning and test B at the end. The other half receive them in the opposite order. I’m collecting data on this now in my intro course, which is based on argument mapping.
Sounds neat, Charles. I’d be interested in the results (if the results can be shared, of course) when they’re in.
The immediate inclination of many people is to think philosophy improves critical thinking skills. But why should we think that? Take the first definition of critical thinking suggested above:
“a broad ability to interpret information and approach problems correctly that can be applied across a wide variety of domains”
Why should anyone think philosophy students should be able to correctly approach problems across a variety of domains outside any they have studied? If I have to figure out how to fix my car, or what is the most efficient way to exercise, I don’t see any reason to think that I should be better at approaching such problems because of my philosophy background. Philosophy isn’t *just* a set of skills; it’s a set of skills applied to a set of subject matters. Of course you *can* philosophize about other subject matters, but doing so well requires gaining a familiarity with them.
Take thinking clearly about politics, as Anonymous does above. Most students, and most people in general, are not very good at thinking clearly. On the other hand, some are. I, at least, do not have good (even anecdotal) evidence for thinking that those capable of doing so are overwhelmingly current or former philosophy majors. My experience, in fact, suggests the opposite. My Facebook feed is full of political commentary from educated philosophers and non-philosophers. In general, philosophers as such are no better at thinking clearly about politics than are other equally educated people; philosophers are just as prone to knee-jerk reactions, straw person arguments, and misinterpretation as anyone else. Those with an education relevant to politics, on the other hand, do tend to have more interesting takes.
An “ability to interpret information and approach problems correctly [applicable] across a wide variety of domains” doesn’t imply its application to *any* domain. Maybe auto repair and efficient exercise needn’t among them, although of course they might be for those individuals who gain familiarity with them.
As for clear thinking about politics, that is, indeed, in very short supply across the board. But whether philosophers are more *capable* of doing so is not the same as whether they in fact do so, or deliberately try to do so. Many moral and political philosophers have devoted quite a lot to attention to spelling out the conditions under which clear political thinking can reasonably be expected to take place. Motivation to do so usually plays a quite central role in these accounts.
Right. My question then would be: if there is an “ability to interpret information and approach problems correctly [applicable] across a wide variety of domains,” why should we expect that philosophers would be any better at doing so in *any* domain than non-philosophers who have been trained to do it? Or, for that matter, if a philosopher and a political scientist approach a domain that neither has familiarity with, why should we expect the philosopher to be perform better? What I find strange is that so many philosophers just *assume* that philosophers are better at this. But what in our experience would lead us to think so other than that when we talk about philosophy, we generally find that philosophers give us the most discipline-appropriate feedback?
Your first question asks for at least one domain in which philosophers might reasonably be expected to have a better “ability to interpret information and approach problems correctly [applicable]” in that one domain “than non-philosophers who have been trained to do it.” In line with my point above, I’d like to rephrase that question as follows: Is there at least one domain in which philosophers might reasonably be expected to *in fact* better “interpret information and approach problems correctly” than non-philosophers who have been trained to do it? Here’s one, assuming the philosopher has taken Intro Logic: making the correct inferences from the provisions of a business or insurance contract. But philosophers of science also frequently have interesting data about how scientists think about their work that might be relevant here.
Your second question asks why we should expect a philosopher to perform better than a political scientist in a domain with which neither has familiarity. Some unfamiliar domains are probably more philosophy-friendly whereas others are probably more political science-friendly. So I wouldn’t have this expectation without knowing enough about the domain to justify it.
I am sympathetic to Roman’s points. If critical thinking is defined broadly as skilled reasoning (or similar), then the claim that philosophers are better at critical thinking than those in other disciplines is implausible. Here is a more modest idea: (i) there is some cognitive skill that professional philosophers have to a greater degree than those trained in other disciplines (ii) that skill is at least slightly more general than the ability to produce academic philosophy, and (iii) that skill is less general that whatever is usually meant by the term “critical thinking.” I don’t know if this is correct, but if it is, an interesting question is how to characterize that skill, and how to distinguish it from critical thinking in general. We could then say with more clarity what benefit the study of philosophy offers. However, because there is substantial disagreement about what critical thinking is, the project I suggest is quite difficult. Davies and Barnett provide a useful overview in the introduction to their anthology “The Palgrave Handbook of Critical Thinking.” Some theories of critical thinking emphasize skepticism in the face of political and intellectual authority. Some theories emphasize the ability to distinguish appeals to emotion from appeals to evidence. Some theories emphasize the ability to parse arguments and detect fallacies. Some other theories emphasize metacognition and the ability to articulate one’s own cognitive processes. Each discipline probably contributes in varying degrees to each of these skills. Perhaps the contribution that philosophy makes to this distribution of skills has a distinctive profile. One could argue that the distinctive contribution of philosophy to this set of skills is attractive even if you deny the existence of a distinctive cognitive skill at which philosophers in particular tend to excel.
If philosophy at best is doctrine and at worst a dogmatic approach to doctrine, then of course critical thinking will be confined by those parameters. Critical thinking helps interpret the first order. Philosophy helps interpret the second order and explore the relationship between first and second orders. The objects of study are different, and hence the methods of thinking are different. If second order understanding contradicts first order understanding, then second order understanding must be reexamined. The purpose of second order understanding is to complement first order understanding, and when this is so, dogma and even doctrine are counter balanced by the rational and the objective empirical. Poor philosophy leads to poor critical thinking. Good philosophy leads to better critical thinking. Good philosophy distinguishes between first and second order and respects the differences, while bad philosophy, that is, dogmatic approaches to doctrine either overvalues one order and undervalues the other. And when this is so critical thinking and rhetorical fallacy are combined.
People might also be interested in this paper: https://dspace.lboro.ac.uk/dspace-jspui/handle/2134/21445
Does studying logic improve logical reasoning?
Abstract
There has long been debate over whether studying mathematics improves one’s logical reasoning skills. In fact, it is even unclear whether studying logic improves one’s logical reasoning skills. A previous study found no improvement in conditional reasoning behaviour in students taking a semester long course in logic. However, the reasoning task employed in that study has since been criticised, and may not be a valid measure of reasoning. Here, we investigated the development of abstract conditional reasoning skills in students taking a course in formal logic, using a more sophisticated measure. Students who had previous experience of logic improved significantly, while students with no previous experience did not improve. Our results suggest that it is possible to teach logical thinking, given a certain degree of exposure.
Another paper to add to this: “The effects of argument mapping-infused critical thinking instruction on reflective judgement performance” (2014).
Highlights
• While results revealed no effect of argument mapping (AM) on reflective judgement (RJ) performance, RJ performance was moderated by critical thinking (CT) disposition.
• Results revealed a main effect of CT disposition on RJ performance.
• Results revealed that CT was significantly correlated with RJ at all three testing times.
• Students scoring low on CT disposition, trained through AM, showed a significant increase in RJ performance from pre-to-post-testing.
• Students scoring high on CT dispositions, in both the traditional hierarchical outlining (HO) and control groups, showed a significant increase in RJ performance from pre-to-post-testing.