A New Kind of Critical Thinking Text (guest post by David Manley)
“What would it look like if we taught only the most useful skills from the toolkits of philosophy, cognitive psychology, and behavioral economics?”
That’s a question David Manley, associate professor of philosophy at the University of Michigan, asked himself after coming to think that the standard approach to critical thinking courses in philosophy departments was not as effective as it could or should be. His answer is in the form of a new, online textbook that brings philosophy together with other disciplinary approaches to take aim at more common reasoning errors using “the tools best suited to fix them.” In my opinion, this is an important, needed step in the development of critical thinking pedagogy.
In the following guest post*, Professor Manley describes the book, its motivation, and how it has worked out in the classroom.
Corinne Wasmuht, “Pehoé P”
A New Kind of Critical Thinking Text
by David Manley
- emphasizes acquiring a mindset that avoids systematic error, rather than persuading others.
- focuses on the logic of probability and decisions more than on the logic of deductive arguments.
- offers a unified picture of how evidence works in statistical, causal, and best-explanation inferences—rather than treating them as unrelated.
- There are embedded questions in each section that are auto-graded and ensure the students are doing the readings.
- It offers a really nice UI for students with search and note-taking capabilities, and they can read the text and answer questions on any device.
- It’s pretty cheap: TopHat charges $35 plus a $10 platform fee if the student isn’t using TopHat already.
- Most importantly, any prof who assigns the text can change it however they like. Want the students to skip a section? Just cut it out. Don’t like the wording of a question? Just change it. It’s hard to overestimate how useful this is in a text.
The text is ready for use right now, but I’ll be continuing to improve it, so I’d be very happy to get any feedback. For the next month or so I’ll be working on an additional chapter called “Sources”, about social epistemology in a world of information overload: navigating science reporting, expertise, consensus, conformity, polarization, and conditions for skilled intuition.
I love most of this, I think it’s a great idea. That said, I’m a little nervous about the chapter on decisions. Here’s why: models of inference and evidence are one thing, but teaching students highly contested models of decision-making are another entirely. There is a pretty profound difference between “de Morgan’s laws preserve truth” and (quote) “the best choice is the one with the highest expected utility.”
I don’t believe that the standard models given to us by economics and decision-theory represent the best way to make real decisions; in some cases, they are downright incoherent or impossible to use consistently, for others, their use will famously promote unhappiness or dissatisfaction. Now, the author does footnote these kinds of controversies, and I’m not here accusing anyone of scholarly neglect! But “how to make decisions” is, in fact, the basic Socratic or ethical question concerning how we ought to live our lives. And it seems slightly problematic to present one particular kind of answer–especially one so fixated on that chimerical ideal of homo economicus–as though it were obviously correct.Report
I thought at the end of the first paragraph that you were going to tell us the profound difference is that the expected utility claim is at least approximately true, and the DeMorgan claim is only true in weird logics (like classical logic).Report
Thanks Brian! Happily, the text entirely avoids discussing DeMorgan! (And repudiates homo economicus: see the comments below.)Report
To be honest, I’m a little worried about the idea that enriching the value function can solve all the problems here. Campbell Brown has a very good paper (“Consequentialize This”, Ethics, 2011) on why we should be suspicious that the correct moral theory can be modeled as requiring the maximization of a suitably enriched value function.
Maybe the right theory will turn out to have this kind of consequentialist form. (As, e.g., Seth Lazar argues.) But it’s a really substantive claim, and not one that can be defended merely by noting that we are trying to maximize more than hedons or preferences.Report
Hey Brian- we don’t disagree! On the philosophical point- my remarks in the comments below about agent-neutrality and incommensurability below were aimed at the very two issues Brown is most concerned about in “Consequentialize This”. The book’s framework rejects the assumption of agent-neutrality, so doesn’t have that problem, and Brown seems to endorse a partial-ordering fix for the incommensurability problem.
More importantly, my text doesn’t actually make the maximization claim that Avalonian quotes: it describes it as “one of the central ideas of decision theory” and says that it’s useful in understanding why some choices are better than others, and in helping us to avoid errors in our decision making.
So can we even teach the idea of comparing the goodness of outcomes in an intro text? Yes, I think we can and should. I liked your earlier point because the situation really is somewhat analogous to teaching classical logic to intro students despite knowing that’s not the whole story.
In many other (mostly non-philosophy) texts that present the idea of maximizing utility, it really is expressed in terms of rational self-interest. This gives the impression that there is the cold and calculating world of self-interest on the one hand, where one weighs the probability and goodness of outcomes, and the warm and moral world on the other hand, where one does not. That’s a deeply confused sensibility, shared by many students I encounter. The idea was to push back by presenting the powerful framework of considering probabilities and goodness while rejecting the ideal of the homo economicus. Report
Thanks, Avalonian! I really appreciate the comment. I wanted to clarify that in the sentence you quote (which I don’t fully endorse in the text), “utility” is not being a unit of happiness or well-being, but represents anything that can make an outcome better. “For instance, many of us care about acquiring knowledge and moral goodness, even beyond the happiness that these things can provide. And we also care about other people’s well-being.” And later I stress that the framework does not commit us to “the ends justify the means” since the utility of an outcome should also include “the negative value of any immoral means that were used to bring it about.”
So the text isn’t assuming agent-neutral consequentialism, though I’m not sure if you were including that in the “ideal of the homo economicus.” But the framework presented does assume that in the relevant decisions we are able to compare the value of outcomes, taking into account everything we care about. And there are important ideas that I do regret having to gloss over in a text like this, such as whether risk aversion is really irrational, and what happens if/when values are really incommensurable.Report
The textbook is a wonderful and brilliant project. I also happen to think the point that Avalonian makes is an excellent one. So long as the decision theory presented in the book compares only the value of outcomes, it is consequentialist in form. It excludes (for lack of a better term) “deontological” forms of decision-making that place value on rules and rule-following, independent of their contribution to outcomes. Likewise, it rules out what we might call “character”-based forms of decision-making. These alternative forms are usually discussed in normative moral theory, but I don’t see why they should be restricted only to moral decision-making.
When I play a game, for instance, I am bound to observe certain rules that are constitutive of the game, and believe I ought to follow other rules that are regulative for playing the game. Within the game, I can focus on making moves that are calculated to maximize the probability that I will win. And I might refrain from making certain types of moves because that would be “unlike me” or “beneath me”. Report
Hi Boram- thanks for the kind words! I think part of the issue here is that we’re using “outcome” a bit differently: the framework in the book includes the wrongness of the act itself in the evaluation of the outcome. (The idea is that, when you act, you change the world in a certain way, but part of that way includes the fact that you committed that act.) As the chapter says, we can model a duty not to steal regardless of the consequences as infinite negative value on outcomes in which I myself steal. Likewise for acts performed out of one kind of character or another. So, use your metaphor, the fact that a type of move is “beneath you” is included in the outcome of the game.
So this is a model for thinking about decisions, intended more to illuminate some prominent decision theoretic pitfalls. It’s explicitly neutral on the moral rightness or wrongness of actions or what we should value in outcomes.Report
Thank you for your response David! I really should not have commented until I’ve read your book. I will stop here because I do not want to detract any further from the great effort and contribution you have made without sufficient justification.Report
“…what happens if/when values are really incommensurable.” – This is one way of putting my concern. The concern is that there is no homogeneous value, utility, that we ought to maximize, allowing trade-offs among different kinds of items. Report
Thanks! I can make it more explicit in the text that there doesn’t need to exist, in any deep sense, a unified value corresponding to ‘utility’ for us to compare two outcomes, given all the things we value. And that there are cases–as I think there are–where no determinate comparison exists, if only for reasons of vagueness.
Still, in many cases we really can compare outcomes. And we really do want to avoid the many common decision pitfalls, e.g. sensitivity to outcome framing, incoherent risk heuristics, honoring sunk costs, time-inconsistent utilities. Thinking about decisions as integrating the goodness and probabilities of outcomes (in the inclusive sense of “outcome”) is a useful framework for diagnosing these mistakes.
Also, I wanted to emphasize that if anyone is interested in using the text, they can modify it for their students however they like! (For example, leaving off the Decisions chapter, cutting out bits of it, or adding a paragraph spelling out any caveats they have.)Report
oops, forgot to include my last name.
I wanted to add that if anyone’s hesitant to leave a piece of feedback in this form, I’ve included a feedback function in the “Notes to Instructors” at the beginning of the text. It’s completely anonymous and no one will see it but me. (The link is below.)
https://app.tophat.com/e/455176/assigned/Report
PS. if the “ideal of the homo economicus” is an individual who behaves according to their own rational self-interest, then not only do I not assume this as an ideal, but I explicitly reject it in various places. For example: “It is often assumed that at some fundamental level, we care only about ourselves, and that we help others only as a way to show off or make ourselves feel good. But the best and most recent science does not support this idea. Instead, like at least some other animals, it seems we often have genuinely altruistic motives. The fact that helping others can make us feel good or increase our status—or even that it confers some evolutionary advantage—doesn’t mean that these things must be our goals when we help others. On the contrary: in part, at least, our aim is often simply to keep others from suffering.” Report
I think this is a great class if the audience is undergrads who may leave motivated to take an upper level classes that get more in depth on these topics. I hope it is marketed outside the philosophy department as well.
Criticisms made in earlier comments about humans not being hyper-rational seem misguided. The whole point of the book seems to break out of the academic silos and acknowledge the work done in other disciplines that should interest philosophers.Report
The text looks great and easily accessible. I’m wondering if you have seen a recent text similar to yours called, The New Critical Thinking: An Empirically Informed Introduction, by Jack Lyons and Barry Ward, (2018). Here is the link: https://www.routledge.com/The-New-Critical-Thinking-An-Empirically-Informed-Introduction/Lyons-Ward/p/book/9781138687486.
It does some of what yours does. It talks about how Cognitive Biases effect our thinking and spends most of the text going over Inductive Reasoning. I’ve taught through the text once now and really thought it did a good job presenting how to reason better according to how we are more inclined to think (i.e. less deductively and more inductively).
The fact that yours is all online and very accessible is a nice feature that I do think will resonate with students.Report
Hi Mike- thanks for bringing this new text to my attention! I hadn’t seen it before and it looks wonderful. Just looking at the table of contents it’s clear that there’s a lot of overlap with what I’m doing. The more, the merrier! I hope this kind of course becomes a standard staple, both in philosophy and psychology departments.Report
Hi David,
Sure thing. I’m glad it is an added contribution to the Critical Thinking literature.Report
I have a what might be a fairly simple question. While behavioural sciences have identified all sorts of heuristics and biases at play in how we reason, how does the science stand on being able to learn to reason better? Can we be taught to reason better, and if so, does this book take an evidenced based approach to pedagogy?
ThanksReport
That’s a great question. I’d put that into two categories: (1) work on the effectiveness of techniques to improve reasoning generally, and (2) work on techniques for successfully overcoming heuristics and biases. Despite insufficient empirical work, on both points there is good reason to be optimistic, and the positive strategies in my textbook are based on some of the excellent work that does exist.
Here are some works addressing (1):
Nisbett et al. (1987). Teaching reasoning. Science, 238(4827), 625-631.
Abrami et al (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275-314.
Abrami, et al (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134.
Fong, et al. The effects of statistical training on thinking about everyday problems. Cognitive Psychology, 18(3), 253-292.
Kosonen, P., & Winne, P. H. (1995). Effects of teaching statistical laws on reasoning about everyday problems. Journal of Educational Psychology, 87(1), 33-46.
Larrick, et al. (1990). Teaching the use of cost-benefit reasoning in everyday life. Psychological Science, 1(6), 362-370.
There is much less work demonstrating successful de-biasing than one would hope for, at least in contrast to demonstrations of what doesn’t work! However, there is some solid work backing up the efficacy of the techniques that presented in the book. For example:
Budesheim, T. L., & Lundquist, A. R. (1999). Consider the opposite: Opening minds through in-class debates on course-related controversies. Teaching of Psychology, 26(2), 106-110.
Lord, C. G. et al. (1984). Considering the opposite: a corrective strategy for social judgment. Journal of personality and social psychology, 47(6), 1231.
Koriat, A., et al. (1980). Reasons for confidence. Journal of Experimental Psychology: Human learning and memory, 6(2), 107.
Stanovich et al. (2013). Myside bias, rational thinking, and intelligence. Current Directions in Psychological Science, 22(4), 259-264.
Haran, et al (2013). The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision Making, 8(3), 188.
Hirt, E. R. et al (1995). Multiple explanation: A consider-an-alternative strategy for debiasing judgments. Journal of Personality and Social Psychology, 69(6), 1069.
Babcock, et al (1997). Creating convergence: Debiasing biased litigants. Law & Social Inquiry, 22(4), 913-925.
Green et al. (1990). Teaching critical thinking and writing through debates: An experimental evaluation. Teaching Sociology, 462-471.Report
PS. given the importance of this issue I consider it one of the most tragically underfunded and understudied areas in the sciencesReport
… but even more tragic is to ignore the research that has been done and teach critical thinking without an eye to the empirical data!Report
Thanks! I’m really grateful for all the feedback I’ve had from instructors and students in the dozens of colleges and universities that have now used the text. The positive feedback has been inspiring, but critical feedback is very helpful as I continue to improve the text and accompanying materials. Report
Hi all- the anonymous guest access link here now requires a password; please email me for it!Report
PS- actually I don’t see why I can’t share the password here: it’s “cognition” (for now).Report