Formal Methods Training for Philosophy Graduate Students (guest post by Joshua Knobe)


The following is a guest post* by Joshua Knobe, professor of philosophy at Yale University.


Formal Methods Training for Philosophy Graduate Students
by Joshua Knobe

It is widely agreed that graduate students in philosophy can benefit from training in formal methods. In our present system, we do at least a relatively good job of providing training in (certain areas of) symbolic logic, but I worry that we sometimes aren’t helping students to learn the methods that will prove most helpful to them in their actual philosophical work.

To give just one obvious example, people working in numerous areas of philosophy need to read and understand papers that make use of statistics. This sort of work has become increasingly important in philosophy of mind, moral psychology, experimental philosophy, feminist philosophy, global justice, and numerous other areas. But there is sometimes a curious mismatch between the training people in these areas receive and the actual work they go on to do. As a result, many philosophers are in the difficult position of doing research that relies very heavily on empirical findings while having a background that gave them knowledge of the Löwenheim–Skolem theorem but no understanding of what a correlation coefficient is.

I have dwelt on this one example just because it is the one with which I am most familiar, but similar issues arise in numerous other areas of philosophy. Philosophy of economics, philosophy of physics, decision theory—each requires a serious background in certain formal methods that might not be used by philosophers working in other areas.

I was curious to hear whether people had any helpful suggestions about how we might be able to organize graduate education in a way that would better prepare students to master the formal methods that will actually be used in their subsequent research. Just to clarify, I don’t have any definite solution myself—I am just genuinely curious to hear what other people might suggest.

Laurie Frick - Pieces Fall Apart and Get Organized (detail)

Laurie Frick – Pieces Fall Apart and Get Organized (detail)

guest
18 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Helen De Cruz
4 years ago

Could you outsource it? As a graduate student, I followed two courses in statistics, organised at Ghent University (not my institution) – one general intro, another one on ANOVA and related methods. http://www.ugent.be/we/nl/diensten/ipvw-ices/aanbod/statistiek2015-2016
It costed about 300 eur per course, and was paid for by my institution. The professor who taught it was a statistician who worked primarily on analysing biochemical data. She was very mathematically-minded and let us calculate t-tests and the like by hand. Report

Fritz Allhoff
Fritz Allhoff
4 years ago

One time I was working on an empirical study and had my friend Josh Knobe help with the statistics. He was pretty good. Another time, I was at an NEH summer seminar for x-phi (hosted at Utah by Ron Mallon and Shaun Nichols) and the aforementioned Josh Knobe showed up and taught everyone basic statistics, including how to use SPSS. I then taught a graduate seminar on x-phi and did something similar in hiring the stats lab to teach it. I was actually surprised how quickly the students caught on. So yeah, I’d say outsourcing to more talented philosophers or statisticians isn’t the worst idea ever.Report

Joshua Knobe
4 years ago

Thanks for these helpful comments! I definitely agree that the solution will in some way involve outsourcing to people in other departments, but It still seems that we need to think about how best to make it possible for students to take these classes.

For example, it is widely recognized that students in ancient philosophy need to learn ancient languages and that these are best taught in other departments. There are then rules in the graduate program that make this possible.

For topics like statistics, things seem very different. Philosophers like Helen and Fritz have taken the time to learn stats, but this shows considerable initiative and independence on their parts. It is not at all an ordinary part of graduate training in the way that, say, Latin would be for an ancient philosopher.Report

ejrd
ejrd
Reply to  Joshua Knobe
4 years ago

I agree that the relevance of interdisciplinary skills is different than the perception of language skills when it comes to philosophy graduate work. I was fortunate enough to go through a graduate program that encouraged interdisciplinary training (and even allowed it to count in place of the language requirement). On a purely selfish note, I was wondering if any of you knew of any programs aimed at philosophers like myself who have enough background in psychology and statistics to read (and understand) research in psychology, neuroscience, and cognitive science but for whom it’s been years since they’ve actually done any research and for whom SPSS feels (once again) foreign? Fritz Allhoff mentions the NEH summer seminar though that is not taking place this year. Report

Fritz Allhoff
Fritz Allhoff
Reply to  ejrd
4 years ago

Duke has a philosophy and neuroscience one going this summer that’s a similar model: half philosophers, half neuroscientists. Those things take a lot of funding to pull off, but they’re a great collaborative idea. So, agreeing with Josh that not everything is like statistics, maybe neuroscience is more like Latin and the Duke model is a good one. Another idea is that we could set up virtual seminars online; that fixes (at least some of) the budgetary stuff.Report

Eddy Nahmias
4 years ago

One barrier, perhaps the biggest, to students learning relevant formal methods (and more generally, sciences or languages or anything else relevant to their philosophical work) is if they have to find/make the time to do it on their own, given all the other demands of a graduate program. So, one way to help make it happen is to remove that barrier. The simplest way to do that is to allow relevant coursework to count towards required coursework for the grad program. That’s what we do for our “Neurophilosophy Track” in the MA program at Georgia State, requiring students on that track to take 2 (of 9) required courses in relevant sciences (our students typically take neuroscience or psychology courses, but it’s very flexible, and some have taken statistics or math courses relevant to their research).

I’d be surprised if most universities (at least in the U.S.) would not allow such adjustments in PhD requirements and allow students to take (for free) coursework in other departments (or nearby universities with reciprocal relationships, such as we have with Emory). The main disadvantage, of course, is that students typically take fewer philosophy courses (though many of our students end up taking more than 9 courses anyway). Another disadvantage is the potential for bad or unhelpful courses or professors, so it’s good for advisors to keep track of which courses and professors are useful for (and inviting to) philosophy students.

Two other advantages, beyond learning the relevant content and methods (and sorry for going beyond Josh’s useful call for info):
1. Philosophy students typically do quite well in such courses and ask questions that remind profs and students in other departments why philosophy is relevant.
2. The philosophy students are exposed to a different academic culture, including the way students in those fields approach the subject. Doing so can give them useful pragmatic information once they enter academia and can occasionally spur new ways of thinking about how to do philosophy.Report

Nick Byrd
4 years ago

A few ideas:
(1) Require it: it would seem that some formal methods like statistics are at least as important as other formal requirements for philosophical research. E.g., I was required to learn some modal logic, set theory, and modern logic. I personally use my understanding of statistics in most of my research. I almost never use (explicitly, anyway) modal logic, set theory, or modern logic. I’d have been thrilled if statistics training were required.
(2) Let it fulfull existing requirements (e.g., language, logic, etc.). Learning statistics requires learning both the basics of mathematical modeling and a coding language (R, SAS, etc.). So there is a real sense in which statistics training could fulfill a logic-like or language-like requirement.
(3) Let it fulfull elective-style requirements. In the two grad programs I have experienced, I have been required to take at least one course outside of my own department. A stats course should be able to fulfill this.
(4) Partner with other departments to create mutually beneficial certificate programs. The Institute of Cognitive Science at CU Boulder offered a graduate certificate in cognitive science. It involved taking 15 or so credits of coursework from at least two departments: psychology, neuroscience, computer science, philosophy, linguistics, and maybe a couple others. I am pretty sure that every course in the certificate counted, somehow, towards the MA portion of a PhD in philosophy. Among many benefits of this certificate was an official avenue by which philosophy grad students could learn formal methods from other departments (e.g., statistics, computational corpus linguistics, programming courses, neural modeling, etc.). Details about the certificate: http://www.colorado.edu/ics/graduate-programs/graduate-certificate-programs
(Obviously, these ideas are at least implicit in the comments above, so I am not saying anything new so much as I am just organizing what is already being said).
I’d be curious to (a) hear about other departments that offer 1, 2, 3, and/or 4 and (b) hear specifics about both faculty and grad students have successfully implemented 1, 2, 3, or 4.
(Apologies for typographical errors). Report

Nick Byrd
4 years ago

In case this wasn’t clear, all of the coursework options I mention above were covered by the standard assistantship tuition waiver so there was no out-of-pocket cost associated with taking courses outside one’s department as long a graduate student had an assistantship.Report

Cathy Legg
Cathy Legg
4 years ago

How about a MOOC?Report

mhl
mhl
4 years ago

As an economist, I can say that economics PhD programs have better luck with undergrad math majors than undergrad straight economics majors. Math majors also do well on the LSAT and in law school, which surprises a lot of people. In other words, I think the solution is to not wait until after people get to grad school to think about this problem.

On the other hand, many people are upset with what is perceived as the excessive formalization of political science. Certain institutions are turning out PhDs who can teach game theory and stats methods, but have trouble applying their tools in the real world because they don’t know anything about the real world.Report

ben
ben
Reply to  mhl
4 years ago

But the most plausible explanation for why math majors do well on the LSAT and in law school is that math selects for people who have a high IQ, not that math training somehow prepares people for the LSAT or law. Report

JBR
JBR
4 years ago

At my PH.D program there was (for a brief time) a Phil of Sci/Phil of Cog Sci track, which allowed me to take neuroscience, psycholinguistics, and statistics towards my phil course requirements. It was ultimately dissolved because few took advantage of it (or at least, I think that was one of the reasons). Two observations: (1) that these counted towards my course requirements gave me extra incentive not to mess them up; (2) these courses helped me get my foot in the door as an experimentalist, and a (basic) foundation in statistics and (most important!) coding. I think that this sort of set up is something other departments could readily implement.

Oh and: there are excellent free resources for learning basic statistics, like “Learning Statistics through R” from Dan Navarro (https://health.adelaide.edu.au/psychology/ccs/teaching/lsr/). It teaches you stats and R at the same time, and does not really suppose a background in coding. Report

Jonathan Livengood
Jonathan Livengood
4 years ago

I think Nick basically has it here. We should let probability and statistics courses substitute for logic and/or language requirements.

But there is maybe a tiny bit more to say. I think that the discipline of philosophy would be well-served to come back to a more 19th century conception of logic on which logic is the (normative) study of reasoning, not just the study of *deductive* reasoning. If the standard undergraduate logic requirement were increased and included some probability and statistics (understood as a standard approach to inductive reasoning), then students at the graduate level would be better prepared to take statistics courses — regardless of whether they are offered in philosophy departments or in statistics departments or in psychology departments or wherever. A few years ago, Andy Arana and I worked up a proposal (which you can see here: http://jonathanlivengood.net/teaching/Logic%20curriculum%20proposal.pdf) for a minor that would treat logic in this way, but two things got in the way of implementing it: (1) it’s hard to change course offerings — lots of paperwork and related hassle; and (2) we have had so much faculty turnover that it’s unclear how we would staff the courses we would need to teach. In sum, the basic idea is to replace what we now think of as mathematical (deductive) logic with formal methods more generally.

Two further tangentially-related observations. First, many departments that work with data have resident experts who teach how to deal with data in that field. Hence, there are statisticians or statistically-inclined researchers in departments of education, psychology, sociology, political science, bioinformatics, library science, and so on. Not to mention statistical and machine learning experts in computer science. We could, presumably, do the same in philosophy provided we had funding to do so. (Similar concerns are going to show up here as show up in discussions of expanding the canon to include non-Western philosophical traditions, I’m sure. And not all of those concerns are ill-founded or unreasonable.) Second, it’s a shame that philosophy didn’t hold on to a closer association with statistics. Here at UIUC, more than 3,000 students take introductory statistics *every year*. (Here’s a great recent data visualization of courses at UIUC, focused on “course GPA”: https://courses.engr.illinois.edu/cs199205/sp2016/discovery/gpa_of_every_course_at_illinois/?a=re). I think that intro statistics sees more students each year than philosophy sees in all of its courses each year — if not, it’s really close. What would our discipline look like today if statistics were understood as part of philosophy in the same way that logic is understood as part of philosophy? I’m not sure there is anything to be done about it at this point in history, but it would be nice if we philosophers could get a slice of the intro stats pie.Report

Gregory Wheeler
Gregory Wheeler
4 years ago

This is an issue we’ve thought about in Munich in terms of our MA program in Logic and Philosophy of Science. The things in our curriculum that speak to Josh’s post are particular to our program, so I should explain a bit about how that works in order to get across what we are up to. Although our experience is within an LPS program with students who come to us already with some background, I believe a lot of what we are doing will work in a philosophy graduate program that has a broader scope than ours. In particular, I’d like to highlight three points.

First, however, some background. Our MA program is a two-year program that started in 2012, and we have done well so far in placing our students in very good PhD programs, even though data privacy laws make it difficult for us to responsibly brag about it. The program is designed to expose students to a range of formal methods, with a mix of required courses and electives that span the following areas: logic & computation, philosophy of logic & mathematics, formal epistemology & rational choice, philosophy of science, philosophy of special sciences (including a two-semester series in philosophy of physics), and analytic philosophy.

By requiring our students to work with a variety of formal methods — in logic, in probability, in computer simulation modeling, and some rudimentary statistical analysis — our first goal is to sensitize our students to a trap that one can easily fall into by narrowly focusing on a single method. For philosophy, that trap has been logic (including rudimentary probability theory). Basic limitations of the applicability of logic ought to be common knowledge to philosophers, just as sensitivity to scale-type is common knowledge to psychologists.

So, the first point about formal methods training for graduate students in philosophy is that the curriculum ought to effect a shift in focus in how students view formal methods. Rather than focus on determining what something means, the question now ought to be what does a thing allow me to do.

As for hands-on experience with statistics, we approach this objective (mainly) through our courses in computational methods. One of our required courses is a models and simulations course where students program their own agent-based models (ABM) in order to address some or another problem in philosophy: e.g., the emergence of social norms, signaling systems, the structure of epistemic communities, or some such topic. The course does not presuppose any programming experience, but instead takes students through a half-dozen programming exercises and examples of working models so that, by semester’s end, students are able to design their own model, program it, and run simulations with that model to generate data which addresses a philosophical question in this new but growing area.

One of those assignments involves rudimentary data analysis, where students are asked to manipulate some data from their simulations using a numerical computing environment, such as R or Octave/MATLAB.

I say all of this because we are introducing a new elective next term, an introduction to machine learning for philosophers, which is designed to next course following the mandatory ABM course.

The idea behind the machine learning course for philosophers, which is called Machine Epistemology, is to walk students through the basic probabilistic approaches to supervised and unsupervised learning through a half-dozen programming exercises; so, what is asked from the students will resemble what we require in the ABM course.  But rather than operationalize parts of evolutionary game theory and some different learning algorithms, Machine Epistemology will operationalize the rudimentary Bayesian statistics that underpins the core of contemporary ML. Also, the philosophical component for Machine Epistemology will be different. The idea will be to re-assess chestnuts in the philosophy of science — confirmation, curve fitting, reference class reasoning, Duhem’s problem, concept learning, to name a few — through the lens of machine learning; in many cases, there is a manageable ML problem that resembles the original. Is it a reasonable solution? Do the differences matter? Is there progress? But, the second part of the course will deal with some of the limits of machine learning methods, which will again reinforce the point about the limits of any formal method. 

I should emphasize that this course, like the models and simulations course, is not designed to replace stand-alone courses one might find in a sociology or computer science department. Instead, each is designed for an LPS MA student who has no prior experience, yet will prepare them for (say) the ML sequence in computer science if she so chooses.  I think that similar courses could be run in a more general philosophy graduate program. Moreover, I think they should be run there.

So, the second point I would argue is that there is a unique place within a philosophy graduate curriculum for courses that resemble Machine Epistemology and agent-based modeling; I would argue against (just) outsourcing students directly to the ML or statistics department. The reason why is that knowing how these methods work, and when they do not work, is integral (as Josh mentions in his post) to an ever growing list of areas in philosophy. What used to be a competitive advantage is fast becoming a necessity for making a contribution. Also, when such courses are done well, they are attended by students from neighboring sciences who are interested in foundations.

Finally, the third point is that methods courses in philosophy ought to have two aims. The first is to give students hands-on experience; in our case, that mainly involves programming and exercise sets. The second aim should be to connect formal methods to philosophy, and there are two ways to approach this second aim. One, as is done in the agent-based modeling course, is to show how a particular method advances philosophical inquiry. Here is a question we are interested in, and here is a method that is well-suited to advancing our philosophical understanding of the issues raised by that line of inquiry. Another, as (I hope) will be done in machine epistemology, is to show how the workings of a formal method raise interesting philosophical questions about that method. In the case of machine learning, in addition to the connections to the philosophy of science, there are ethical and policy considerations but a clear assessment of those considerations depends on sorting genuine risk from confused fantasy.

**
I apologize for the length of this note. I am very thankful that Josh opened this discussion. I would be very grateful for feedback and suggestions. For example, we do not spend any substantial time (in the MA program) on experimental design, except in the context of designing simulations to run. I currently don’t see a workable and profitable way to include this, however. Report

Joshua Knobe
4 years ago

Thanks so much for all of these extraordinarily helpful comments! Though the different suggestions differ in detail, I think that just about any of them would do a great job of addressing this problem.

Overall, this discussion leaves me in one way optimistic and in another pessimistic. Optimistic because everyone seems to agree, at least in principle, that we need to shift to a system in which students get more training in the formal methods that they will actually use in their work, and not a single person has voiced any dissent with this basic view. Pessimistic because even if everyone agrees that this is the direction we should move in, it will still take a lot of initiative to actually make changes in the rules of graduate programs, and I’m not sure whether we will be able to overcome the usual institutional inertia.

For example, my sense is that very few philosophers would genuinely conclude on reflection that a student working in empirical moral psychology should be required to study non-standard models of arithmetic but should not get credit for a course in basic statistics. Yet many existing programs are set up in precisely this way, and it might be difficult to figure out how to change them. I guess what we face now is a question in non-ideal theory. We seem to agree about which direction we need to move in, but how do we actually get there from here? Report

Current PhD Student
Current PhD Student
Reply to  Joshua Knobe
4 years ago

Instead of promoting training in formal methods as a benefit to philosophical research, why not come from a different angle and suggest instead that graduate students in philosophy need formal skills in order to be competitive in a non-academic job market. Given how bad the academic job market is, department chairs, administrators, and cooperating departments should be open to changes that will help prepare MA and PhD philosophy students for a career outside of the academy. Should there be any doubt about the career benefits that formal methods provide, I urge skeptics to go look around on LinkedIn, Indeed, Glassdoor, etc. and notice the type of backgrounds that are in high demand. They will find that skills in computer programming and formal training in statistics, math, engineering, econometrics, etc., are often required for “analytical” jobs, which now dominate the non-academic job market.

A major problem I see with many academic philosophers is that they are behind the times. The way that philosophers tend to market the benefits of studying philosophy are either outdated or so ambiguous that students are often misled.

In regard to being outdated, ‘analytical skills’ means something different to HR folk than to philosophers. Given the tech era that we live in, the meaning of ‘analytical skills’ pertains to computer programming and quantitative abilities. Sure, critical thinking and argument evaluation are important in every job. But such abilities are not specific to philosophy—there are a lot of smart people outside of philosophy, too. So one cannot just rely on the ability to analyze arguments and even formalize them in logic. Additional practical skills are needed to compete in a non-academic job market.

In regard to the marketing tactics being ambiguous and misleading, I often hear philosophers tell students (mostly undergrad) that they could pursue a career in business. First, I have no idea what that means. Business is comprised of so many specialized areas and related areas, e.g. entrepreneurship, advertising and marketing, sales, finance, accounting, etc. Most specialized areas require some specialized knowledge that philosophers lack. So an MBA may be required, which is a possible route for undergrads, but not so much for recent PhDs. Does a pile of debt adding on to an already existing pile of debt plus (for grad students) more time studying after finishing years in grad school sound attractive? I can’t speak for everyone else, but I would say no. Similar worries apply to the suggestion of going into law.

Ultimately, I think that philosophers need to get with the times by learning what is actually in demand outside of academia. Right now, training in formal methods is in demand and will make philosophy students a little more competitive. Moreover, since it is certain that newly minted and seasoned PhDs will be left out of the academic market at the end of each academic year, departments may even have an obligation to prepare their graduate students to take on non-academic careers.

This sort of line might be more compelling and get the ball rolling, but in the end, your goal will be achieved through this strategy also.
Report

Dan Hicks
4 years ago

For a few reasons that I won’t go into here (except to note that they’re things like the replication crisis, the open data movement, and the increased use of Big Data and computational modeling), there are similar discussions happening in biology and psychology. I highly recommend these two papers:
http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002430
http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002128

In particular, in terms of the content of “stats for philosophers,” I think Box 2 in the first paper provides a good starting point.

Unfortunately, I don’t have much to say about Joshua Knobe’s practical question of how to reform grad program requirements. I will say that it’s common for philosophers of physics, biology, and neuroscience to take several grad courses in the respective STEM field. It might be useful to survey grad directors at programs like Pitt, Notre Dame, Irvine, Madison, etc., on how they manage the requirements. Report

Gregory Wheeler
Gregory Wheeler
4 years ago

A two-semester course composed of modules that span logic, set theory, measurement theory, choice, games, and an operationalization-by-programming component could go a long way. The theme for such a course might be the question ‘What’s all the fuss about numerical representations?’, which would tie together the modules, but each module on its own could serve double-duty as foundations for follow-on courses offered in the department. So, to take Josh’s example, a department might offer a follow-on course in the foundations of mathematics that took up non-standard models of arithmetic in some detail; but a department might also have an active decision theory group, and therefore have follow-on courses in decision theory. In either case, all students in such a department would have walked through a couple of axiomatic systems in the common formal methods sequence, so such follow-on courses could spend far less time on set-up and preliminaries and more time on axioms for arithmetic, say, or show how the basic strategy from axiomatizing extensive quantities in measurement theory was used to supply numerical representations of one’s commitments and values, and so on. On the other hand, all students will have also confronted questions to do with calibration and external validity in such a formal methods course, so they will come to advanced courses with the wherewithal to consider the scope applicability of different formal methods.

I am displaying my bias for theory here; but, the idea is to show students how seemingly disparate branches of formal methods are connected to one another *and* show them how knowing how those parts fit together (and where they sort of fit together and sort of don’t!) can be put to good use.

I am cautiously optimistic about change. The forces prompting the adaptations we are discussing will be increasingly difficult to avoid. Some see this as a threat; others, a great opportunity. Fortunately, it only takes a couple of people in the right place at the right time to change a grad program.
Report