An Impressively Detailed Philosophy Paper Grading Rubric
Micah T. Lewin, a recent PhD from Stanford who is currently an adjunct professor of philosophy at Perimeter College, Georgia State University, has created an impressively detailed and helpful rubric for grading philosophy papers.
Check it out:
Here is a PDF of it, and you can find links to variously formatted versions of it as his teaching page.
(Thanks to Nathan Nobis for bringing this to my attention, and to Micah Lewin for permission to post it here.)
It’s clear that this rubric is the work of someone who is laudably committed to excellence in teaching and I get that there are many people who could find it useful (including, of course, outcomes assessment coordinators), but from my perspective as a teacher this thing is too detailed to be usable and it leaves no room for finding a single damn good argument or philosophically interesting claim that trumps all the student’s other mistakes and makes and otherwise mediocre paper into a fine piece of philosophy deserving of the highest possible grade.Report
Yes, because informing students explicitly in a grading rubric that “a single damn good argument or philosophically interesting claim” will trump and wipe out all their other mistakes doesn’t seem like a recipe for receiving terrible papers, does it? And someone using this rubric obviously cannot make any exceptions ever, at anytime–why exactly?Report
Why use a rubric if one is prepared to throw it out when one finds something one is really looking for in the first place?
And one would not need to tell one’s students that a single impressively good argument or claim will trump their other paper-writing mistakes, but, even if one were to tell them that, why would one think that one is likely to get worse papers from undergrads in that kind of case than in a case where one lays out the “color by numbers” rulebook that most rubrics seem to provide?
I honestly do not find explicit rubrics to be helpful when trying to teach students how to write philosophy, but I realize that some of my colleagues do. In spite of the tone that might seem to be apparent in the above questions, I get that there are many good ways to teach. If you use explicit rubrics and is works for you, then good for you.
I worry about what I see as the increasingly common assumption–by students, faculty, outcomes assessment coordinators, etc.–that the *only* way to do good teaching is to use explicit rubrics.Report
Good thing no one is using a rubric to grade my comments for typos.Report
Ah, I see — you are not a fan of rubrics at all. Your first post’s criticisms aren’t really that fair to this rubric then *qua rubric*.
I guess I just don’t subscribe to the idea that the best way to grade philosophy papers is following the–to caricature–“I know good philosophy when I see it” school of thought: i.e. that undergraduate philosophy papers are to be held up to some relatively inchoate standard of ineffable philosophical genius. Of course there is plenty of room on the spectrum in between this proposal and using detailed rubrics–for instance, supplying some explicit standards, but not as many as this rubric. But I would think that the rubric posted here would not be supplied to students in a vacuum; it would be used along with other materials to help with philosophical writing. There need not be a “one size fits all” document to guide students in writing their papers: different intellectual profiles learn from different materials.
And as for making exceptions to a rubric, there is nothing wrong or inconsistent about relying on a rubric for most of your grading, but allowing for exceptions in truly exceptional cases. That’s just part of life: learning from extraordinary cases. In such cases, modifications to the rubric might even be made if a general lesson can be extracted.
Personally, I find that if I do not spell out where all parts of my own assessments of papers come from, there is a degree of unsatisfying arbitrariness in grading: am I being consistent in my evaluation of papers from student 1 to student 77? Rubrics can help maintain consistency in grading and evaluation. Prior to reflection on these matters, we are subject to all kinds of implicit biases. I find that rubrics can help keep professors, including myself, honest.Report
This strikes me as anal-retentive, OCD compulsiveness. You cannot reduce philosophy to rubrics, formulas, or whatever. Nonetheless, I do admire the dedication to trying to get things objective. But I remain, after 35 years of teaching, convinced philosophy is not an objectively measurable discipline. I say this with respect for a young philosopher trying to get it all together as a teacher.Report
I’m a history professor. Using something like this even once would send me home, screaming and tearing my hair out. I have tried simpler rubrics and they are far too confining because they assume a certain stable proportion among different categories but categorical quality should be treated as relative to other categories and not absolute.Report
Unfortunately, I don’t think rubric will help most students, if the goal is to help them write good papers from the outset. I would predict this would be most useful to the sort of student who is focused on details and doing things right, but these are the same students who would otherwise predictably get good paper grades. A much larger group of students assume they know what good papers look like and would predictably ignore this lengthy and very detailed breakdown. For the latter, instructions must be much more concise if they are to pay any attention to them. Alternatively, if the goal of the rubric is to justify a grade, I would think a large number of students would find reviewing this material tedious, and a professor who uses it, pedantic.Report
I agree. The problem I have with rubrics is they focus too much on justifying grades and not enough on helping students. I doubt it will be helpful for students to learn they received an A- rather than an A because their arguments were “somewhat plausible” rather than “highly plausible”. Is the student supposed to learn, “Great next time I will give an argument which is highly plausible rather than somewhat plausible”. However,if a professor wants to do both a rubric and detailed comments than I guess that’s fine, just seems like too much complex work for not much reward.Report
I don’t see how this streamlines grading. Instead of just checking a box, one still has to assign a number. For example, one cell of the rubric covers a range of 10.5-11.9 possible points. I think that fractional points are a kind of mock-precision which is hard to tally and which I can’t assign consistently.Report
I think rubrics are extremely important. They provide an intelligible structure that allows you to grade your students fairly and consistently. I prefer less detailed rubrics, however. Also, creative novelty is a bonus, but I don’t require students to break new ground to get an A.
Philosophy has the resources to allow evaluate students on fairly objective measures (via standards of logic, argument and critical thinking), so we should employ them, but philosophy often has the unfortunate reputation for subjective grading. I think this is because there is the impression that a student will pay the price for endorsing the “wrong view”. This is unfortunate and something we should be very careful to guard against. I think rubrics are one way, not necessarily the only way, for achieving that end.
If for nothing else, it helps me to know that I can clearly explain to a student in my office hours where they went wrong and what they did right and how they could do better, and I can justify their grade clearly and systematically.Report
Thank you for all the constructive criticism. This is my first attempt at a full-fledged paper grading rubric, so I appreciate the feedback.
I do agree with P.D. Magnus that perhaps one area of improvement would be to have a definite point value per cell to streamline grading: I envisioned using it in something like this way at any rate, with a default value for each cell, and then perhaps a standardized + or – value for each cell (e.g. B+ or B-) for standouts on either side of the default. That should help grading go more smoothly.
As for Izzy’s point about creative novelty, I agree: undergraduates shouldn’t have to break new ground to get an A. But if you look at the point values on the rubric, this category (# 5) only counts for 5 points. So not being novel/creative won’t knock you down but half a letter-grade. A student could still get an A (95) while completely missing out on originality (unlikely given the standards) according to the rubric.
I also appreciate the number of voices calling for including less detail in the rubric. I can see how giving this rubric without additional instruction or other guidance might be off-putting to many and how it might just end up being ignored by students. But it was never my plan to use this rubric without first explaining it to students in a class meeting specifically devoted to Philosophical Writing. I also provide students with a plain-text outline “Guidelines for Philosophical Writing” which many would find more useful: it simply discusses what you *should* do in writing a Philosophy papers, not the many ways in which you might fall short, like the rubric. I envision the rubric would be useful to a certain type of learner: the type of student who wants to know where each and every point of her or his paper grade comes from. Others might not find it as useful, and I fully anticipate that. Different people have different learning profiles and that is okay: that is why, when I teach, I strive to incorporate similar information into different modes of presentation in order to make it maximally accessible.
At any rate, too much detail ain’t all that bad: I would rather err on this side than the other. I always also like to point my students to the very detailed, though quite approachable, web resources of my undergraduate teacher, Jim Pryor: e.g. http://www.jimpryor.net/teaching/guidelines/writing.html
Again, thanks for having a look and for your comments. And many thanks to Justin for posting it here on Daily Nous.Report
I recently used a modified version of this rubric for a section of Philosophical Ethics, and I found it extremely helpful. Thank you for your contribution.Report
This post reminds me of a challenge that I don’t often see addressed in conversations about pedagogy: how to set effective assignments and exams. I was lucky to work with and study under professors that set fair assignments. The assignments’ instructions were clear, the goals were explicit, and students felt that the projects were motivated by the structure of the course. Exams were also well-set: students did well only when they studied regularly and understood the material.
But not all instructors are so well-equipped. Many instructors leave vague language on assignments “for the students to interpret as they will”, or they’ll reward memorization more than the ability to engage philosophical questions. I’ve met students who wrote twenty-page philosophy essays in their second year. From what I could tell the students learned nothing more than what I learned writing 750-word papers. Instructors who foresee the challenges that students will face on assignments save themselves and their students from unnecessary grief.
I hope the next big movement in professional development foregrounds the professor’s role in setting assignments and exams.Report
To the OP,
Using the rubric, how long does it take you to grade a typical paper? My worry is that it would add a significant amount of time to grading (though perhaps that is as it should be). There are, after all, 60 sections in the rubric. It is so detailed that, after giving the paper a read through, I’d have to go back to the paper several times to confirm my recollections. Jotting down notes as one goes will help, but several of the criteria are holistic.
No doubt that speed increases as familiarity with the rubric does, but at 200+ papers to grade at the end of each term, even a 5 minute addition will make the lifting heavy (when factoring the same number of exams and other end-of-semester duties). This of course isn’t reason by itself to ignore the rubric, but the rubric has to produce the kinds of improved results (from students and from grading accuracy) to justify the added time.Report
I haven’t tested the rubric out yet on anything but a small sample of papers from previous terms, so I can’t really give you a fair answer. I would say that, anecdotally and personally, grading without a rubric, I do find myself wasting a fair amount of time worrying about diachronic consistency of evaluation over the whole stack of papers, and for any given paper, whether I might be forgetting about important dimensions of evaluation in grading it. Scoring papers with a rubric sure does help me get over that second-guessing time-suck.Report
This strikes me as exactly the sort of thing I might have done, when I was a very young and somewhat unrealistic new professor.
Little did I know then that I would be routinely confronted with well over 300 final exams/papers that needed to be graded in less than a week’s time (as I just finished doing this past week). Once those sorts of realizations set in, I had to come up with much quicker — and by necessity much more impressionistic — methods of grading papers.
In truth, then, I find the rubric much more interesting as a view into a newly minted professor’s mind than as anything that would be of much use in grading in the proverbial trenches.Report
I’m not a teacher and it’s a while since I’ve been a student, so I won’t comment on the practicality of this. It’s certainly very comprehensive.
It’s amusing, though, to think how some of the canonical works we study might fare under this schema… e.g. Wittgenstein’s diligence in providing expositions of the arguments of others in the Tractatus, or Kant and Hegel’s concern for readability (in most of their full length works!)Report
In terms of helping students, I have found it much more useful to read them excerpts from other students’ papers — pointing to especially strong parts and maybe indicating places where things could have been done better — then to try and give them very complex instructions, either before or after the fact. There are so many different ways to construct a good paper/essay/short answer that no set of instructions could possibly capture them all. In that sense, perhaps, how one does good work of this sort is something — as Wittgenstein put it in the Tractatus — that can be shown but not said.Report
I suppose I don’t see how these two modes of instruction are mutually exclusive. As an instructor I provide examples of good papers, but I also think providing more explicit instructions/guidelines can be useful. I think it’s true that some students might only learn from being shown, not told; but others might not fit that profile. Why not do both, if you can? Different students learn in different ways.Report
Because in 25 years of teaching, now, in a number of very different settings, I have not found rubrics useful. This has been my experience, and that’s all I have spoken to — *my* experience. Yours may be different.Report
In my limited experience as a TA (and instructor once so far), I think I get much more improvement in student writing if I focus my comments on two things. First, I give some specific examples of where I thought the paper was weak, with some ideas about what I would have tried instead. Second, I give one or two priorities for what the student should work on more in the next paper (as well as mentioning the one or two things I thought they did well and should try to continue doing well). This is purely anecdotal, but I find a long list of negative evaluations both intimidating and unhelpful for teaching how to do better. I wonder if this fits other people’s experience (or the results of studies which I’m sure have been done but of which I’m entirely ignorant)?Report
Kudos on spelling these things out so impressively. I’ve tried to construct rubrics like this myself, and I know how tough it is.
One unfortunate feature of most such rubrics is that they present the virtues of essays as additive, when in fact there many virtues that are essential for a good essay. It seems to me that a rubric can be most effective at helping students write well when they make clear that certain vices can utterly doom a paper.
Here’s an example of what I mean, using your rubric. Suppose I’m a student in your course. I came into the course thinking that philosophy is just a matter of learning a bunch of terms and theories and names and trotting them out, or even just finding a Great Philosopher to mine quotes from. I have failed throughout the course to learn anything about the practice of doing philosophy. I now have to write my major essay for the course, and I get the librarian to help me find an article by a Great Philosopher on the topic I’m meant to address in your course. I then go to office hours with the TA or a tutor and learn to make sense of what’s going on in the article.
Armed with this knowledge, I present to you an ‘essay’ consisting of little more than a string of quotes and close paraphrases from the article. I cite everything perfectly, so I have not committed plagiarism.
It seems that, according to your rubric, the ‘essay’ I submit (if the article I’m working with is sufficiently great) only loses points on the originality/novelty component (Item 5). Let’s say you give me the lowsest score on that: 0/5. If, as it seems, you have to give me full points on everything else, I end up with 95%, or a very solid A, for the essay.
For these reasons, I’ve changed the structure of my rubrics. They now take the form, “Your essay should have virtue X. An essay that fails to have virtue X will receive _at most_ this many points. Your essay should also not commit error Y. On each occasion that your essay commits error Y, I will deduct this many points, and you may only commit an error of this sort five times in the essay or else I stop grading it at that point.” Etc. I list these wrong turns after explaining what a good essay will look like.
Some might want to change the structure of this, naturally. The point is to stop students from thinking that an essay is just a collection of different sub-projects that can be isolated. It’s a unified whole, and needs to stand together as a unified whole; and failures along a certain dimension can make successes along different axes practically worthless (I have visions of students walking away thinking that all life is like this (“Yes, boss, I admit that I didn’t do any of the research properly. But my presentation did have several clear pie graphs in it, and I rehearsed my talk so well that I spoke my lines like a professional actor. I think I deserve some credit for that. Why am I being fired?”)Report
This is a very interesting point. I’ll have to think about this, and your alternative structure — it seems intriguing and even promising. I haven’t tested this rubric in an actual course yet, so I’ll have to see. I don’t envision my students will be as creative as the example you cook up, but I may be underestimating their potential for mischief. Anyway, I still countenance the example you detail here as a case of plagiarism & academic dishonesty–you cannot present as your own work something that merely copies/paraphrases another paper, even if you cite appropriately. (Moreover, given my essay assignments, its unlikely they’ll find a paper on *just exactly* that topic.) Furthermore, elsewhere (in my “Guidelines for Philosophical Writing” among other places) I make it clear to students that academic honesty and appropriate citation/attribution are *preconditions* of a passing grade, not elements thereof.Report
Why does it have to be gender specific with the words “In her paper, the author…”?
Can it not be used for male students? Political bandwagon? I see so often in presentations and papers that when academics use a gender term, it is in the feminine, so it must be what is now regarded as politically correct, but I see it as just as incorrect as putting everything in the masculine. Is it now a world where the masculine is dirty?Report
This seems like a “reverse sexism” complaint in the style of “reverse racism.” I don’t believe in reverse racism, and for the same reasons I don’t believe in reverse sexism. Rather, I do believe in affirmatively acting in small, but meaningful ways to reverse the systematic and institutionalized sexism that has infected even our very standards of style and grammar. Sometimes this can be done by conspicuously and affirmatively choosing to use the feminine as the “every-person” in the way that the masculine was for millennia used for the “everyman” without second-thought. Personally, I think that opting only to use the neutral “he or she” at all times expresses the (false) idea that we are fully “gender-woke”–that gender parity has been reached, and no further self-reflective work needs to be done to root out implicit biases or linguistic double-standards. But this isn’t true: there is a lot more work to do, and actively prioritizing the systemically disadvantaged–even in expressive/linguistic ways–is a way both to highlight and help this.Report
I think Jane was using ‘sexism’ (and, if you’re right, by implication ‘racism’) in accordance with ordinary English usage, as codified in, say, the OED (and basically every other dictionary). So I don’t think your response works. It involves nonstandard use of the terms ‘sexism’ and ‘racism’, and an equivocation. Some people (including you) disagree with that, I realize…but it seems to me that folks on your side of that disagreement are at least obligated to *acknowledge* the disagreement, and that their use deviates from the norm. Otherwise you end up missing the point of objections like Jane’s, and/or baffling people who don’t know about the nonstandard meaning–and that means: the vast majority of English-speakers. I think that it would be better to come up with different terms for the different phenomena–one very popular term is ‘structural racism’, which seems to me to get across the desired meaning fairly efficiently.
Incidentally, I think that if you don’t go down that terminological rabbit hole, there might be some other, less controversial, responses to Jane’s objection that might work.
But also: nice work on that document. I’m skeptical about the usefulness of something so fine-grained…but obviously I could be wrong. And anyway, I can admire it even in my skepticism.Report
Thanks for the breath of fresh air, Jane and William. It’s good to know I’m not the only one who finds these kinds of facile responses nauseating. *Barf* Maybe they’re well-intentioned, but it’s getting out of hand and really makes no sense at this point. I don’t know if these people ever listen to themselves. Still, nice work on the rubric, Micah.Report
A lot of the comments for and against rubrics seem to invoke testable assumptions. So I find myself wondering:
What do the data say about using rubrics with college students?
In particular is there research that answers these questions?
1. How do learning outcomes vary between assignments with rubrics and other kinds of assignments?
2. How does critical thinking vary between assignments with rubrics and other kinds of assignments?
3. How many rubric items and degrees of quality are too many for the average student?
N. [Another question about rubrics that research can answer]?Report
This study found no significant differences in writing quality or self-efficacy between long rubrics, short rubrics, and open-ended feedback. But the sample was only 60 students, which means there were about 20 students in each condition. That far below the recommended minimum of 50/condition in good journals. So I don’t think we can infer much from this.
If you’re aware of better studies, please do share (either here or privately).Report
And this study found significant performance differences between students who wrote a paper with access to a rubric and students who wrote the same paper without the rubric. However, (1) it’s not clear how both sets of papers were graded (e.g., were they graded using the rubric?) and more importantly (2) one person did all of the grading which (a) makes it impossible to validate/calibrate the grader’s grading and (b) leaves open the possibility of the grader’s confirmation bias and/or expectancy bias influencing the data according to their beliefs about rubrics. (The author admits 2b in the discussion section).
I’m still on the lookout for studies/experiments with better methods.Report
Here’s my rubric.
This paper is pretty good. A
This paper is sort of OK. B
This paper is pretty bad and seems to be written by an illiterate. But there’s a flicker of understanding. C
This paper is terrible. But there was one sentence that, when read charitably, says something that is not preposterous. D
This paper is utter crap. FReport
Illiterate? Wow. Check your privilege and be a better teacher.Report
Brea, are you privileged enough to be teaching at a school that doesn’t have illiterate students, like the rest of us do?
Or do you just not care that the world is going to hell in a handbasket, with illiterate, unprepared, unmotivated students getting into university somehow and then getting a free pass for doing and achieving nothing in the form of a bunch of inflated grades, thus making the value of the accreditations we provide meaningless enough to put the higher education system in jeopardy?Report
Why bother with rubrics, a device beloved by ‘educators,” when your reading assignments (let’s hope) provide models of what good work looks like? Assignments, and I admit to being ancient, often had descriptions on how the prof wanted submitted papers organized. After that, profs curved the work. As in baseball, not everybody’s ERA is in the top 10%. Even if a prof had a class of bootlickers who managed to achieve total compliance with the rubric du jour, he may still wish to exercise taste to select the truly excellent papers from those that contented themselves manage to ape the rubric.Report
Honestly, this seems way too precise, given that college students may be hitting multiple rubric points in the same criterion. And yet, this kind of rigor indicates a commitment to excellence on the part of students that it commendable. Micah, if you every want to commiserate over student writing, look me up. My PhD is from a less prestigious institution (and not in philosophy), but I’d love to talk about your experience working with GA college students.Report