Teaching Students Logic Improves Their Logical Reasoning Skills


Newflash: teaching students logic improves their logical reasoning skills—at least according to some new research. You may be thinking, “duh,” but that would be a mistake. After all, “teach” isn’t a success term. And as it turns out, “there is little evidence that studying logic itself improves one’s logical thinking.”

You may recall that there have been some rather dispiriting empirical studies questioning the effectiveness of philosophy courses in improving students’ critical thinking skills. There’s this meta-analysis which we discussed here last fall. And then there’s this one:

Cheng, Holyoak, Nisbett and Oliver (1986) investigated the development of conditional reasoning skills in undergraduates taking a semester-long course in logic. The students completed four Wason Selection Tasks (with a mixture of conditional and biconditional statements and abstract and thematic content, see Figure 1 for an example) at the beginning and end of the course, which contained 40 hours of teaching, including the definition of the conditional. It seems reasonable to expect that after such training students should be fairly competent at dealing with conditional statements; it is difficult to imagine a more promising way to improve a student’s logical thinking competency. Nonetheless, there was a non-significant decrease in errors of only 3%.

That description is courtesy of Nina Attridge (University of Bath), Andrew Aberdein (Florida Institute of Technology), and Matthew Inglis (Loughboro University). They raise some concerns about that study before describing their own perhaps slightly more reassuring one in “Does Studying Logic Improve Logical Reasoning?” (forthcoming in Proceedings of the 40th Conference of the International Group for the Psychology of Mathematics Education).

To investigate the question of whether and how students’ logic skills could be improved, Attridge, Aberdein, and Inglis tested around 80 students from various majors on conditional reasoning tasks prior to and following a semester-long course in logic. The student data was split into two groups—those whose majors typically involved some logic instruction, and those whose majors did not. Only those in the first group showed a significant improvement in their logical reasoning skills.

Here’s how they put it:

We investigated the development of conditional reasoning skills in undergraduates taking a course in logic. Overall, our results suggest that studying formal logic improves students’ ability to deal with conditional statements, but only if they have had some experience with logic previously. While conditional inference scores did improve over time for the whole sample, when we examined the role of previous experience with logic, it became apparent that only those who had studied logic previously actually showed any gains in reasoning skills during the course. For those students who had not studied logic before, there was not a significant improvement in conditional inference scores over time. Interestingly, the students who had taken a logic course previously did not outperform those who had not at Time 1 [at the beginning of the course]. This suggests that the amount of logic training the students had received previously was not sufficient to give them an advantage on our conditional inference task, but that it was sufficient to make the logic course in question more effective.

Our findings suggest that it is possible to teach logical thinking, but that a certain level of exposure may be necessary before students’ skills begin to develop. We do not have data on the number of hours of previous study that participants had, but the fact that students without prior experience did not improve during the 37.5 hours of lectures involved in the current course suggests that a greater number of hours is required for development. Future research should systematically investigate the number of hours of exposure necessary for students’ logical reasoning skills to improve.

Still, this study does not make a strong case for instruction in philosophy or logic, over, say, mathematics:

Our findings suggest that, contrary to previous research, it is possible to improve students’ logical reasoning through instruction. Nevertheless, the level of improvement we found was comparable to that seen in A level mathematics students, who received no explicit logic tuition.

(Thanks to Nick Byrd for linking to this study in a recent comment.)

Mark Reynolds, "Phi Series: Root 5 Grouping, 1.15.15" (detail)

Mark Reynolds, “Phi Series: Root 5 Grouping, 1.15.15” (detail)

 

guest
25 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
PeterJ
PeterJ
4 years ago

I know nothing about this but it seems very interesting and surprising. If there really is little evidence that studying logic improves one’s logical thinking then it would seem to call into question what is being taught. Otherwise I cannot make sense of a study of logic failing to improve one’s logical thinking. Few people even use Aristotle’s rules properly so this result makes no sense to me. Anecdotally I can say that a study of logic transformed my reasoning processes and I cannot grasp how this cannot be the case for most people. Or, at least, most people that I know.

So many fascinating titbits on DN. Report

highfive_ghost
highfive_ghost
4 years ago

“Still, this study does not make a strong case for instruction in philosophy or logic, over, say, mathematics”

Okay, that’s fine. But some students hate math, or think that math is not for them. Logic can be an appropriate alternative for those students. Report

Tom
Tom
4 years ago

ITS BECAUSE WE’RE TEACHING THE MATERIAL CONDITIONAL RAWRAWRAWRAWR!!

Sorry. Had to get that out of me. But seriously, large fragments of classical logic are deeply unintuitive, which I don’t think I realized until I noticed how many of my students were *memorizing* truth tables. Report

Daniel
Daniel
Reply to  Tom
4 years ago

As Tom points out students find many elements of logic to be not at all intuitive. Two points:
1.) we have to teach logic as a part of philosophy: it can’t be a course that could just as easily taught by the math department. This means (a) showing the student some of the history of logic (Aristotle, Stoics, Ockham), and (b) giving the student plenty of time to apply logical rules in the evaluation of specific philosophical texts (eg, Descartes’ third meditation, or sections from Plato’s dialogues, such as arguments for the immortality of the soul). Otherwise they will have a jumble of memorized rules but won’t be able to apply this to arguments made in real time.
2.) they need to see the roots of logic, in the same way that one would understand algebraic formulas not by merely memorizing them but by seeing the geometric roots. A more wholistic approach is needed.Report

Alan White
Alan White
Reply to  Tom
4 years ago

First, show students that bivalent classical logic–propositional and first-order predicate–serves as the basis of two very relevant parts of our lives: the practical skeletal support for the basic semantics of much of natural language, and the formal basis for most of the marvelous devices constantly in their hands. (An aside–shouldn’t the younger generation now be dubbed “The Devotionals”, heads constantly bowed in worship?)

Second, stick to those themes in further explication. Negation and the conjunction are rather intuitive and straightforward for both natural language and machine language, but the disjunction introduces the complexity of semantics and truth value, resolved by making the inclusive sense basic and the exclusive defined by it along with the tilde and dot (or whatever). Then you face the open-set semantics of conditionals and face the disjunction problem all over again, but much worse. Explain that the material conditional is a compromise solution: it by definition only captures by negation all cases where conditionals (of asserted dependence) fail–thus the two crucial lines (F/T; F/F) that result where conditionals are “held-harmless” for assertion as true (by negating that they are not the cases of T/T “good” or T/F “bad”). So the material conditional just does a basic job of showing when asserted conditionals succeed, fail, or are held harmless. I do not see that at all counterintuitive, but a conservative approach to respect all such assertions with respect to truth value. As far as computer coding is concerned, it’s enough to find when conditionals definitely succeed or definitely fail. (I then introduce the biconditional as only a further truth-functional convenience that can be used to show logical equivalence for the purposes of flexing the muscles of proving validity–otherwise known as good coding.)

If you do those first and second parts well, then the predicate calculus is just an extension with sets of individuals and properties that can show how our natural language has additional complexity that needs to be reckoned with.

Well anyway, this has generally worked for me as a nobody at a nowhere campus. Which is most of us.Report

Mackerel
Mackerel
4 years ago

At my university the director of the academic center worked at Kaplan testing for several years before. He told me that in his experience the single, best way for students to improve their LSAT score was to take a Logic class, which he regularly recommended. Hard to see how this worked if students didn’t learning anything.Report

Eric Baum
4 years ago

TruthSift.com allows you to diagram debates and settle them point by point on the basis of whether they can be rationally demonstrated. Users post proofs and refutations (of any topic a user cares to propose) onto diagrams, and TruthSift keeps track of what has been logically established. I predict it will be a great way to learn critical thinking and reasoning skills, as well as to figure out what is demonstrable and what illusion.Report

Eric Baum
4 years ago

In Mathematics, people publish proofs, which are equivalent to trees. Someone might publish a refutation. The proof might be fixed and/or a refutation to the refutation published. When the dust settles, the proof is established or not. TruthSift supports exactly that logical process on public diagrams, for any topic. I predict it will be a great way to learn critical thinking and reasoning skills, as well as to figure out what is demonstrable and what illusion. Report

Merely Possible Philosopher
Merely Possible Philosopher
Reply to  Eric Baum
4 years ago

It is worth pointing out — full disclosure and all that — that Eric Baum appears to be the founder of TruthSift (or otherwise affiliated: http://truthsift.com/aboutus).Report

Eric Baum
Reply to  Merely Possible Philosopher
4 years ago

I am indeed the inventor and founder of TruthSift.
However, its worth pointing out the design of TruthSift is that everything is to be transparently justified on its merits, rather than by appeal to authority, and anything not so demonstrated is subject to challenge on that basis. I’m not asking you to use TruthSift because I said so and I am smart and disinterested, I’m hoping you will use it because you are smart and understand its utility from first principles yourself.Report

Tom
Tom
Reply to  Eric Baum
4 years ago

“In mathematics, people publish proofs, which are equivalent to trees”. Really! I read mathematics all the time. Very few trees. Care to elaborate?Report

Eric Baum
Reply to  Tom
4 years ago

Every proof in mathematics is supposed to be equivalent to a proof tree (or proof DAG). It starts at leaves that are axioms or previously established statements, and proceeds by steps that are supposed to be simply syntactic applications of rules of logic (or previously established transformations) such as modus ponens to previously established nodes. (A step may typically be based on or combine several previously established nodes.) The modern field of computer-verified formal proofs, eg in programs written in Coq or HOL-light or etc. makes this explicit. TruthSift works in the same way, only it allows any steps that users believe are justified, subject to challenge.Report

WP
WP
4 years ago

The methodology unfortunately seems a little iffy—the lack of a control group makes the results hard to interpret. They found that students majoring in computer science, software engineering, or electrical engineering (the “prior logic” group) improved on a conditional inference task over the course of a term where they also took a logic class, and students in other majors did not. So was it the logic course, or was it the rest of their major?

In case anyone doesn’t have a chance to look at the paper, the task: Students were presented with arguments like
Conditional rule: If the letter is M then the number is 5.
Premise: The letter is M.
Conclusion: The number is 5.
and had to say if they were valid or invalid. This is exactly the subject matter of a logic course! It could be a question on a final exam! It’s shocking to me that there wasn’t more improvement. It’s not just that the non-CS/engineering students didn’t apply what they learned to their everyday lives; they seem not to have learned much at all. (The task was administered in class, so it’s not a context effect.)Report

Arthur Greeves
Arthur Greeves
4 years ago

I have a theory that logical reasoning skills are pretty much fixed at an early age, and don’t improve much after that. Parents are the central teachers of logic. If they use conditional statements deceptively, children simply cannot recover. A child who is constantly given empty threats will have difficulty understanding the material conditional. A child whose parents use language with precision will adjust to formal logic quite well.

This is all just a crazy theory of mine, but I would love to see it tested. I have anecdotal evidence to back it up with, but not much else.Report

WP
WP
Reply to  Arthur Greeves
4 years ago

If the theory is that logical reasoning skills literally don’t improve much after an early age, there’s already enough evidence to rule that out. There are enormous gains between late grade school and late high school—see for example Ward and Overton’s 1990 ‘Semantic Familiarity, Relevance, and the Development of Deductive Reasoning.’ Few 6th graders correctly answered at least 3/5 questions with the material conditional, ~80% of 12th graders did. Report

Arthur Greeves
Arthur Greeves
Reply to  WP
4 years ago

To say that something is fixed at an early age does not entail that it manifests itself at an early age. Musical talent may be fixed at an early age, but no two-year-old can play a five-part fugue.Report

PeterJ
PeterJ
4 years ago

Makes sense to me, Arthur. Not fixed, maybe, but pretty well entrenched. Report

Hey Nonny Mouse
Hey Nonny Mouse
4 years ago

In my personal experience, students have always seemed to get a lot less out of my introductory formal logic courses than my introductory informal reasoning courses. They always found it hard to apply the formal stuff to real life. I’ve certainly had a lot more say on end of semester evaluations that they found the informal course of practical use.Report

Georg
Georg
4 years ago

Readers may find this edifying. Report

Nick Byrd
Reply to  Georg
4 years ago

I get a 404 error when I use that link. Can you say something about it so that I can search for it?Report

Georg
Georg
Reply to  Nick Byrd
4 years ago

Strange. Wonder what happened between July 16 (last accessed) and now. Here is an archived version: https://archive.is/0xbIO#selection-217.76-217.268Report

Nick Byrd
Reply to  Georg
4 years ago

Thanks Georg!Report

Javier
Javier
4 years ago

It’s been impressive to me to see how many people dedicated to philosophy in the US have always tried to make it an esoteric cult thing, distorting the process of conveyance of ideas with the apparent purpose of excluding anyone from understanding it. This, instead of lovingly passing along this mother of all sciences. The books on the subject I have come upon here are undecipherable.

The next interesting fact I have observed is that those same people have also tried to ideologically manage objective realities of philosophical thoughts or facts to fit their own capricious worldviews. An example of this is a basic introductory book on philosophy that I found, which -for example- addressed the problem of God by analyzing the different proofs given for God’s existence, and then went on to attack these proofs, yet the author didn’t even once mention the ontological argument of saint Anselm. You may take either position on the ontological argument, but what you cannot do is ignore it when every philosopher, from saint Thomas Aquinas, including Leibniz, Kant and Hegel, dealt with it!!

A refreshing exception to all this is a discovery I happened upon -at Barnes and Noble, of all places- which showed a beautiful editorial decision by the bookstore of selling an excellent translation of Julián Marías’ “History of Philosophy”. A paradigmatic opus of philosophical pedagogy, much needed here.

At any rate, on the subject of this article I say that if you concentrate in Aristotle you’ll have most of what you need to understand of Logic and be able to use it in your daily life.Report

Tom
Tom
Reply to  Javier
4 years ago

Can someone explain what Javier is saying? I lost his point in all the ranting about how bad philosophers are at philosophy. Report

PeterJ
PeterJ
4 years ago

The points about the value of logic made in this little article seem to be undeniable.

https://philosophynow.org/issues/115/Bad_Arguments_That_Make_You_Smarter
Report