Assessment and Philosophy Courses
Most of the discussions regarding “assessment” are fine examples of exactly what we do not want to see college producing: vague and uniform truisms, hooked up with measures so meaningless as to guarantee that nothing will ever change. It is the deadened life of the bureaucratic mind. But imagine, as an alternative, academics charting the careers of students who have turned out to be really interesting, and trying to figure out what really happened, and to what extent their own courses or programs can take any credit for it. Undoubtedly, there never will be any sure-fire formula. But we might be able to collect a range of good practices, interesting ideas, experiments to try, as well as some solid critiques of what can stultify a college career.
That’s Charlie Huenemann (Utah State), who, like many of us, has to deal with “learning outcomes” and “assessment”: figuring out what students who’ve attended the school should be able to do, and how to measure how well the school is doing at helping students achieve these goals. In his view, the aim of college is for students to develop “interesting minds.” I am sympathetic with that — I remind my students to take advantage of college because it is probably the last time they will be surrounded by people whose job it is to make them more interesting people. However, since the interestingness of graduates is not readily quantifiable, and since the demand for quantitative assessment is not likely to go away, there remains the question of how colleges and universities should go about assessing courses.
A current trend is for schools to require that certain “learning outcomes” orient courses. Courses are then assessed by how well students are able to demonstrate that they’ve met the learning outcomes. Yet both the specification of the outcomes and their assessment seem, for various reasons, problematic. What should the learning outcomes of a philosophy course be? And by what means, realistically, should colleges and universities check to see whether they’ve been met? Have you had good experiences with assessment systems that might serve, in part or whole, as models for the rest of us? Or are there better ways to handle the demand for accountability that seems to be motivating the drive towards assessment?
This isn’t exactly the question that you asked, but one thing that I would like to see considered for evaluating students who receive bachelor’s degrees in the humanities, and especially philosophy, would be an independent written (or other) project completed near the end of the program of studies. If you think (as I do) that education in the humanities primarily ought to teach critical thinking, argumentation, and communication, it seems pretty clear that this is better evaluated on a more holistic basis than the individual course.
The model I would want to see would be something like this: in the last year of study towards a humanities degree, students write a 20-30 page independent research paper that is evaluated by a panel of 3 professors from 3 different humanities disciplines (perhaps one of which should be required to be the students’ own discipline). I’d be concerned about bias towards one’s own students here, so ideally I’d like to see institutions group together here and arrange to have students’ work evaluated by professors from different schools (although of course this raises its own questions). Evaluators should be instructed to base their grades 90% on clarity of writing and effectiveness of argumentation. I could be wrong about this, but I think any humanities PhD holder should be able to do this for the work of any undergraduate, regardless of subject (it’s like scoring the writing section of the GRE, but longer). Each student then receives a grade for this project which is attached to his or her educational record.
I think the advantage of a system like this is that it acknowledges that we do believe there are standards that some students meet better than others, but refuses to accept that they have to be “objective,” in the sense of not based on some professor’s evaluation of the work, or that they have to be analyzable into minute data points that are simply aggregated together. A weakness of this system might be that it doesn’t allow us to pinpoint the contribution of any individual teacher or course that a student had (this seems to me like a dubious goal, but it’s in the background of a lot of the push for assessments). However, if we had enough years of data like this on enough students, we could indirectly do this using statistical methods.
I have struggled with this at my own institution, but I have found (and perhaps my institution is in the minority) that departments are being given a lot of latitude to develop their own learning objectives, and I think that we should take advantage of that offer. There are, after all, things that we want our students to *get* from our courses. I certainly hope that my students will be more knowledgeable about the topics that I teach, but I also hope they will become better writers, clearer thinkers, more intellectually charitable to views different from their own, better readers of dense philosophical prose. And I suspect many of us have similar goals. So then our challenge is to figure out how best to determine whether our students have achieved those outcomes. That is a challenging task, but not an impossible one. Most importantly, it seems, we should want to know whether our students have achieved those goals even if administrators and accreditors weren’t asking. If I want my students to be better writers and it turns out they are not better writers by the end of the course, then I should modify my teaching, right? Where I have struggled is that I often want students to change affectively or with respect to their values. I want them to appreciate and value, say, reasoned debate, or appreciate and value metaphysics, Existentialism, or Stoicism. Figuring out how to determine whether they have achieved those sorts of outcomes is more difficult, but, I would argue, not impossible.
“What should the learning outcomes of a philosophy course be?” (a) Knowledge of the literature; (b) Ability to formulate in one’s own words the key questions, problems, and aporia discussed in the literature; (c) Ability to formulate a plausible argument, valid in point of form, to one or more of the questions or problems.
“And by what means, realistically, should colleges and universities check to see whether they’ve been met?” As Andrew Mills suggests above, we should keep the internal and external measures compartmentalized. What goes up to the adminosphere will be whatever they require, and the unit should not regard these measures as meaningful as an internal check on meeting learning outcomes. At the unit level the challenge is captured by the word ‘realistically’: assessing outcomes without further burdening faculty service loads is probably not something that can be shoehorned into existing governance structures and expectations. This, to my mind, is the most difficult challenge and one which probably requires a significant change in institutional culture. Philosophy students should be able to demonstrate their competence in the course both verbally and in writing. Paul Hammond’s suggestion for the written component seems reasonable to me, though the close evaluation of those theses is going to be labor-intensive. If faculty from other units are going to be involved, they will need to have in place a similar assessment system to make the burdens of operating it equitable. I am in favor of viva voce exams for philosophy students. Faculty, graduate students (if there is a graduate program), and undergraduates can be enpaneled to jury the proceedings.
The adminosphere does not care if your unit is meeting qualitative measures through written capstones, oral exams, or trial by fire. They have a different set of interests driving their absolute demand for quantitative measures, often at odds with the goals of education per se, and so they will measure the unit’s performance with different criteria. The demands on the unit to satisfy the requirements from above leave little energy left to design and implement internally meaningful learning outcome assessments.
I’m in administration now — I don’t want to bury that too much — but even when I was in the faculty ranks, I didn’t see why specification of learning outcomes for philosophy classes and their assessment were even prima facie problematic. For example, putting those terms in scare quotes seems silly to me. Assessments don’t require numerical measurement, though even if they did, developing a scale that was suited for the purpose seems like the kind of thing philosophers should not have much difficulty doing. Further, I don’t think there’s anything intellectually uncomfortable about that. For institutional purposes, there are all kinds of arbitrary scales that we’re comfortable using, so long as we think they track as much as possible the important directions and clusters of ability that we need to judge — grading, seminar enrollment limits, teaching loads, SAT scores, etc. A good assessment plan for philosophy learning outcomes can seek substantive answers to questions like: Are students able to spot errors of reasoning in an argument? Are students able to identify clearly empirical questions from clearly non-empirical ones? Are students able to apply general metaphysical or ethical principles to questions involving particular facts and situations? et cetera. Assessing students’ abilities with respect to these outcomes can be as simple as establishing particular course assignments in various sections from which to draw a random sampling of work — specific exams or papers that the sections all assign for assessment purposes — and marking them on a simple numerical scale to indicate whether it is clear that a particular artifact displays the ability in question. I’ve participated in these things, both as a philosophy professor and as a dean.
I suspect that people who regard outcomes and assessment with disdain aren’t really baffled by the process. They are against lots of other things that aren’t necessarily connected to it: wasteful administrative overhead, punitive use of assessment results, and the general commodification of higher education. Those things are bad but they don’t render assessment per se problematic. More interesting is the idea that it is absurd to think we can talk about learning outcomes and their assessment with things that are thought to be very difficult to assess but are some of the most important things that we try to instill in students of philosophy: having interesting minds, for example, or a sense of wonder (something one of my former departments included in our program objectives). I don’t know why we can’t assess students on such outcomes, personally, since I’ve seen philosophers do it all the time with each other, especially when assessing potential colleagues in a hiring situation. It would encourage us as philosophers to be more reflective and clearer, in fact, about what those judgments are based on.
I believe my department has done an admirable job with our assessment program and published a paper that describes our efforts in Teaching Philosophy a few years ago. That paper, “The Transition from Studying Philosophy to Doing Philosophy,” is available without subscription here: https://www.pdcnet.org/teachphil/free (scroll down to volume 34). It was reprinted this summer, in an excellent collection of Lenssen prize winning essays (together with multiple commentaries and replies… book description here: https://www.pdcnet.org/pdc/bvdb.nsf/item?openform&product=publications&item=aapt-lenssen ). I encourage your readers to read the whole collection of essays, but for those interested in a model for assessment in their own departments, I happily recommend our own as at least a place to start.
“They are against lots of other things that aren’t necessarily connected to it:”
Yes, but in most cases those things are actually connected to it. That there can be reasonable and worthwhile forms of assessment should do nothing to assuage those who are worried about the form demands for assessment are likely to take. Similarly, some of us are opposed to U.S. military intervention abroad not because such intervention is in principle objectionable, but because we’re confident that, as a matter of fact, it will be carried out in unjust and immoral ways.