Is Anonymous Grading Really Better?
A philosophy professor has written in with some questions about anonymous or “blind” grading, in which the identity of the student whose work is being assessed is not known to the grader. The majority opinion in philosophy appears to be that there are strong moral reasons in support of anonymous grading. Yet, there are questions about evidence for it, as well as about balancing its advantages and disadvantages:
1) I wonder what the state of research is on the potential benefits of anonymous grading. I’ve poked around, and what I’ve found (not much) doesn’t clearly support the practice, so I’m wondering what we really know. The general literature on implicit bias is suggestive for the grading context, and that seems to be what everyone goes on.
2) There are disadvantages of anonymous grading, such as when a strong idea in a paper may have come from the professor in office hours. How should you grade that paper if you don’t know? Or what about a strong line of argument that is not clearly expressed, but recoverable by the teacher who spoke with the student about the ideas before the paper was written? Etc. The fact that anonymous grading reduces the possibility of bias isn’t enough to say whether that benefit outweighs these and other costs. Even assuming (despite the crisis of replicability) some implicit bias creeps in without anonymous grading, do we know whether academics would thereby favor men, and whites? Or might they unconsciously favor women, and blacks for ideological reasons? Or might these wash? Who knows? And in any case, how strong is the effect? If it’s small are we sure the benefits outweigh the disadvantages of anonymous grading?
Readers?
This is just anecdote, but I’ve found pedagogical advantages to anonymous grading as well. I announce my practice to my students and many report feeling it is a fairer practice. That’s a nice benefit, but more importantly it helps me draw a very clear line between the students themselves and their work. This allows me to truthfully explain that the grade was assigned on the basis of the work, not on my opinion of the student as a person or an intellect. This is really useful in both dealing with grade complaints and in giving constructive feedback because it makes it easier for students to take their grades less personally, and makes it easier to say “I think you’re great and capable of good work, but that paper was terrible, and here’s why and how it could be improved.” Put that in the plus column for anonymous grading.
Here’s a reason to favor blind grading even without knowing the answers to the questions in the OP. If I grade blind, and my students know that, then they can know that personal, racial, and gender biases didn’t affect their grades. That in turn means they’ll be less likely to view bad grades as unfair, and more likely to take my comments and criticism seriously. In addition to fairness itself, perception of fairness matters, not just to keep students happy, but also for good pedagogical reasons.
RE 2: I don’t understand the alleged disadvantages of anonymous grading (or more exactly, I don’t understand the implied advantages of non-anonymous grading).
(a) If you’re the professor who came up with the strong idea, you’ll presumably recognise it whether or not the paper is anonymous — so anonymity is irrelevant in that case.
(b) If you aren’t (e.g. you’re a TA marking papers in someone else’s course), are you seriously going to go ask the student’s tutor for interpretative help every time there’s a strong idea which is either possibly partially attributable to the tutor, or that isn’t clearly expressed? That’s going to be a whole lot of extra work — in particular, I’ve marked *a lot* of essays that meet that latter description. Which leads me to…
(c) I can see how people might differ on this, or have different grading criteria, but I don’t think you *should* be asking for interpretative help in such cases. You should mark what’s in the essay, not what might be recoverable from the essay. In fact, I think the difference between a strong idea clearly expressed and a strong idea explained obscurely is a key difference between, say, an A and an A- (or a high 2:1 against a 1st, or whatever).
There is some research on the benefits of anonymous anonymous review in general (including but not limited to grading). My coauthor and I recently wrote a paper on best practices in the discipline (in Ergo: dx.doi.org/10.3998/ergo.12405314.0002.003) that included a passage on it. Based on what we found in the literature, it’s not clear that anonymous assessment is good in all circumstances, but it seems clear that it is generally beneficial in an academic setting (see in particular this study: http://dx.doi.org/10.1093/restud/rdq005). There are many studies on the positive effects of anonymous grading, at all stages of learning (see, e.g., this recent post on Feminist Philosophers: https://feministphilosophers.wordpress.com/2015/04/05/anonymous-marking-better-for-everyone/). Meta-studies on anonymous peer-review find no clear negative effect (biolmedonline.com/Articles/vol1_4_Rev3.pdf). In our paper, we give some references about bias found both against and for minority students, in addition to stereotype threat, which all are good arguments in favor of anonymous grading.
One problem with the implementation of this practice is that it can hinder personal and more fine-grained feedback. To avoid that, it’s best to grade the paper anonymously but give feedback (e.g., write comments on a written assignment) after the name is revealed. Another post on Feminist Philosophers raised another concern: anonymous peer review seems to allow for harsher comments from the reviewers (which is something editors should pay attention to). I don’t know if that can be a worry when grading undergrad papers, but that’s another reason to distinguish grading (anonymously) and giving feedback (with the name revealed). Do you think there are more “costs” to anonymous grading that a non-anonymous feedback wouldn’t address?
My current institution is the first one I’ve worked at that does anonymous grading. While I laud the sentiment behind it, I continue to find I have a deep reservation concerning it, because it prevents me from implementing a pedagogical technique I think extremely important — for all university disciplines, but most especially philosophy — that of reading drafts, commenting on them extensively in writing and in person, and having the students revise. One of the most important skills that I can teach an undergrad is how to write clearly and with good argumentative structure. The best way to teach this is to teach the process of writing and rewriting, revising in the light of new information. But there is a limit to how much info I can give them in written comments — and a limit to how much info they can extract from my written comments. It really requires a dialogue between the student and the teacher to discuss the particular issues in a paper, and if I have to grade anonymously, I can’t do that.
Fair point. I agree that that’s a valuable pedagogical technique. But I’ve never taught at an institution where teaching loads/class sizes were small enough that I could possibly comment extensively on drafts, or engage in extended dialogue with students about individual papers, and so I’ve done bits and pieces to try and reach the pedagogical goal in other ways, at least some of which are compatible with anonymous final grading, e.g.
* Allow students to submit a plan/outline (not a full draft!) for early comment. These are usually sufficiently sketchy that they don’t effectively de-anonymise final essays.
* Peer-review revision among students, especially at upper levels: they read each other’s drafts (in a structured manner), you don’t.
* Make assessment cumulative, e.g. first assignment is a lit review, second is an argument summary, third is a critique, fourth is a final essay, all on the same subject.
Nothing perfect, but some things to think about?
But surely, in a well-run examination system, you shouldn’t be involved in *grading* work submitted for assessment that you were involved in *supervising*. Then there can be as much appropriate dialogue between the student and teacher in discussing issues as you rightly want.
But surely, in a well-run examination system, you shouldn’t be involved in *grading* work submitted for assessment that you were involved in *supervising*.
This seems difficult in practice. Usually the classes I teach are the ones where I’m most familiar with the material, and it’d be difficult for one of my colleagues to grade the material; and it’d be especially difficult to organize a swap (I might be teaching something one of my colleagues could grade while she’s teaching something I can’t grade).
Late to this thread, but for myself, I use blind grading of papers but always mention two things.
1) I do it to avoid unconscious bias in either direction. Beyond larger cultural biases, I at least mention that without it I can easily come across one paper and think “This student speaks up in class all the time and has great ideas.” and another paper and think “This student never says anything and has skipped class several times, I wonder how much they managed to learn.” In each of those cases, I am grading that paper in a VERY different mindset. Despite ideas on larger cultural biases or not, they seem to readily understand that and appreciate it.
2) That I am always willing to look at and discuss drafts/outlines/whatever. Although I talk about how beneficial anonymous grading seems to be, I make it very clear that it is more important to help people who want it even if it violates anonymity. Anonymity is a value, but helping anyone who wants it is a greater value. I worry I might discourage people from talking to me about drafts, but from my non-scientific anecdotal experience from before deciding to use blind-grading and after, I seem to get roughly the same number of people asking me to look at drafts. Might be slightly lower, but not significantly. (Usually only a couple students per assignment out of a class of 25. So certainly a low enough number that random variation can be a larger factor.)
“what about a strong line of argument that is not clearly expressed, but recoverable by the teacher who spoke with the student about the ideas before the paper was written?”
regarding this, I think there are some obvious issues – the reader of the essay should not need to have been the professor in office hours who understands the line of argument via prior conversation. Shouldn’t students learn to communicate the idea to a reader with whom they have not previously talked about the paper too?
That said, I think there could be some benefits to grading with identity in mind, though I think it would be useful to read the essay as anonymous, and then be able to learn who the writer is. For instance, if a paper has a very casual tone, it might be acceptable for an older student who you know understands how to write an academic paper, while it might indicate a lack of understanding in a first-year student.
“what about a strong line of argument that is not clearly expressed, but recoverable by the teacher who spoke with the student about the ideas before the paper was written?”
I would say that this thought actually reinforces one of the benefits of anonymous grading. If I have spoken with a student and think quite highly of him, I may be prone to exercise a lot of charity when interpreting an unclear but potentially suggestive line of argument and give that student the benefit of the doubt (plus a higher grade). But if I read that same passage from a student I think less of, I’ll be more likely dismiss it as simply confused. That seems to me to be pretty clearly unfair.
This is one of the reasons that I’ve switched almost entirely to completion-based grading. There’s much less concern about bias of any sort when the grade merely reflects that the student has met the requirements for the assignment. The real work is done not in the graded but in the feedback. Of course I don’t think it would hurt for me to anonymise grading, but without the possibility of biased grading it doesn’t seem to provide any benefit.
I think it’s fair to establish that a student has to express an argument with an accurate understanding. This is honed in within a mutual relationship between oral and written tradition. Insofar as a student learns an argument, they have to be able to both be able to voice it and write it. Anoynoumous grading is a tool for grading because it allows the professor opportunity to remove the subjective part of student-teacher relationships, and evaluate the argument objectively. I think it’s a judgement call as to whether someone wants anonymity in the drafting process or final paper, depending on how they want to influence student development.
I second Sara Uckelman’s worry: anonymous grading makes it impossible to read drafts and work in progress. Our institutional requirements on Writing Credit courses will make this mandatory soon. But even if it is not, I don’t want to tell students who ask me to read their work in progress that I won’t.
I have a system for this. Students have a deadline to submit anonymized drafts. I then comment on the drafts and return the work to the students (hard copy is easiest, but with an assistant it is possible to reunite drafts with names and upload them to a course management system).
It’s also possible for students to send in their papers (identified by student ID only) from an anonymized e-mail account or leave them in a department mailbox.
To what end though? At this point we must balance the risk of implicit bias in favor of (or against) those who come to office hours and go above and beyond in order to seek your feedback on work in progress with the (in my opinion) more substantial benefit of sitting with a student and engaging them in a discussion about their work. This–a discussion about their work–is where the real pedagogical payoffs are, again in my opinion. I think this is something that mere written feedback can’t actually accomplish.
If we are so concerned that merely discussing a student’s work during office hours will be biasing enough to try and control for, why do have office hours in the first place? They seem like a situation ripe for bias for any student who wants extra feedback or discussion. Similarly for classrooms. Surely our student’s performance in classroom discussion is a biasing factor. Moving things to an online-only environment where student’s names are replaced by random numbers would seem to be the best possible teaching environment in terms of blocking out implicit biases. I doubt that many would think the risk of bias is so strong that it would be worth going that far however. So I guess I’m wondering why some feel like office hour interactions (especially reading drafts of work) run a high enough risk for bias in terms of assessment that it is worth incurring a cost to student learning and development in order to prevent it?
Am I the only one who finds it non-obvious that we should try to remove all subjectivity from grading and try to grade entirely and only the work rather than the student? I grant the obvious dangers to doing so, and great and terrible papers will still stand out, but it also seems to me somewhat obvious that a conscientious student who comes to office hours several times to ask questions about the paper, etc, but still ends up with a mediocre product deserves a higher grade than someone whose paper is of the same quality but never shows up or shows any indication of caring about the material…
I think Abob and Uckleman’s concerns are really the two more significant problems with anonymous grading. Not being able to read drafts is really ridiculous, especially if one of your pedagogical goals is to help students become better and more critical thinkers (instead of just evaluating their ability to do so). I realize that reading a draft is onerous, especially with 100+ students per term. However, this is an easy enough problem to solve. In my case I state as a clear policy that I will only read drafts of papers during my office hours and will not, as a categorical policy, read or comment on drafts over e-mail. I even tell them that this is because it would be impossible for me to do this for everyone. As a sidenote, this has, at least in my experience, made office hours a more valuable commodity for students.
Re: Abob’s concern. This is something about which there is disagreement in my department. We have met to discuss the goal and function of grading in the past and it’s clear that there are those who believe that the purpose of grading is to somehow ‘objectively’ mark the quality of a piece of work without any other ends in mind. There are others who believe that at least one of the functions of grading is pedagogical in the following sense: a student’s grade could be used to assess quality and also to push them to do better. To speak to Abob’s point: a lazy student whom you know could do far better work terms in a paper of the ‘same objective quality’ as another student whose work you have seen improve dramatically as a result of significant effort and visits to office hours. At least in my department, there are disagreements about whether the two students should receive the same grade. Blind grading would remove this pedagogical use of grading. Now, if you think this pedagogical use of grading to be immoral or bad then that’s not really a cost for you (I’m not even saying which side of this debate I’m on, for the record), but there are at least those who believe that grades are not a simple ‘objective’ marker for quality.
I don’t know about whether one view or another is obvious, but my personal feelings on the matter are that if I want to give a student a reward in terms of a higher grade for something like coming to office hours several times, being conscientious in other ways, etc. I should make room for this in my syllabus with something like a participation score (that can include office hour visitations, email conversations, etc.) rather than doing this by giving them a higher grade on their paper than they would’ve gotten had they not made as much of an effort to demonstrate how conscientious they are.
This is for a few reasons. First, I’m a little worried about whether someone should get a better grade in a class just in virtue of putting a lot of effort in, compared to someone whose work is of the same quality but who hasn’t put in as much effort. People have all sorts of priorities in their lives, and if someone is able and willing to organize their life such that they put a lot of effort into my class, I’m glad, but there are some people for whom the class is perhaps just a required class that they’d prefer to skate by in, or people with lots of other things going on in their lives that preclude them from putting in the effort they would otherwise prefer to put in, and so on. It’s not clear to me that I ought to assign grades such that these sorts of things end up rewarding or punishing a student.
Second, I’m not overly confident about my abilities to pick out the conscientious students. Everyone is conscientious (or fails to be conscientious) in their own ways. Some students come in to office hours all the time and seem to me to be making a real effort; others come in to offer hours all the time but seem to me to be there largely because they showing up gets them brownie points – they ask rather perfunctory questions and spend lots of time looking at me as if they hope I’ll start saying things unprompted. Am I right when I make these judgments? How often am I right? Are some people just shy? Other students refrain from coming to office hours not because they are not conscientious but because they don’t feel like they learn very well like that or because they are even shyer (this is how I was as an undergraduate, and I consider myself to have been a fairly conscientious student back then). How good am I at reading into a student’s email or their in-class comments or anything else when it comes to figuring out how conscientious they are? I might just be overly self-critical but I suspect that all sorts of unconscious biases and blind spots play roles in my assessments of conscientiousness.
Third, I pride myself on being (at least in theory) open and transparent about why a student got the grade they got on a paper, and I would feel very uncomfortable saying to a student who came in to talk to me about their paper “the reason you got this B- is because, among other things, you came in and talked to me a lot. Otherwise you would have gotten a C+,” or even worse, “the reason you got this B- is because, among other things, you did not come and talk to me as much as the other students who wrote similar papers did, and they all got Bs.” Those do not seem to me to be the kinds of justifications I’d like to give a student. I want to be able to tell them that their paper got the grade it got because of various features of the paper that I can happily talk over with them.
Fourth, I want people to know where their grade is coming from – if I want part of their grade to include conscientiousness, I want that to be clear on the syllabus and I want them to know about how much of their grade can depend on this and how much depends just on, for instance, the papers they write. If I grade papers based on conscientiousness rather than keeping conscientiousness and related things in something like a “participation” category, a lot of stuff is “up for grabs,” so to speak – a student who needs a C+ to graduate, for instance, will look at the syllabus and say “I know I’ll need to put some degree of effort into making Danny think I’m a good student if I want to get my C+, but I’m not really sure how much, so I don’t know if I should spend the extra hour studying for my chemistry exam, or if I can pick up those extra hours at work to make some money, or if I need to come to his office hours.” Whereas if papers are graded just on the basis of the papers that they are, a student can say “at most, 15% of my grade is going to be based on how conscientious Danny thinks I am, so I’m going to need to get at least X on the papers, which are 45% of my grade and which are just determined by what I write.”
Fifth, in terms of skills I want my students to be learning, “writing effectively” is more important to me than “being able to convince someone I am conscientious,” and it is MUCH more important to me than “being able to write something the virtues of which depend partially on the reader having a prior belief that I am conscientious.”
Sixth, I don’t really care how much my students care about philosophy, and I thus wouldn’t want to punish someone who does not show “any indication of caring about the material” just in virtue of this. I love philosophy and I love it that many of my students love it, but I appreciate the fact that there are lots of things in life to care about, and I see it as my job to make my class engaging, rewarding, illuminating, and educational for everyone, not just for people to whom philosophy (or rather the subset thereof that we’re looking at in my class) is particularly compelling. I wouldn’t want my science classes grading me on how much I give a crap about isotopes or whatever because I give no craps about isotopes. I’m also not sure why Iggy Isotope sitting next to me in class ought to get a higher grade than me just by virtue of having been wired such that Iggy loves Isotopes. Ditto for philosophy, especially given worries about the degree to which philosophy can be a pretty parochial subject that appeals to a narrow subset of people, etc.
Seventh, I don’t really want one more thing to worry about when I’m grading a paper. It’s enough to be called on to judge the merits of the words on the page – I’d rather not add into the hopper the requirement that I also judge the student’s soul or personality or whatever it is that’s conscientious enough to call for a better grade.
Eighth, I am stupendously bad with names, such that in effect I grade anonymously for 90-95% of my students already simply in virtue of not knowing their names (or even worse, I grade a paper with the mental picture of one student in my head only to subsequently discover that it was an entirely different student’s paper, because I am indeed that bad with names).
I don’t think all (or perhaps even any) of these reasons are uncontroversially good reasons to think about things the way I think about them, but I do think they’re reasons that one might reasonably endorse (I’ve endorsed them, after all!) so it is not surprising to me that people might have these reasons or reasons like them, and on this basis might desire to grade papers just on the virtues of the paper rather than the virtues of the student too.
I think it would be unfair to give two students different grades when the two students produced work of equal quality and did an equally good job of meeting the general parameters of the assignment (e.g., submitting it on time, meeting the length requirements). There’s an assumption in Abob’s sample case that the student who visits office hours and asks questions demonstrates greater effort and interest than the other student. But for a variety of reasons, that may not be true. Perhaps the other student works a job that conflicts with those office hour times. Perhaps that other student is heavily introverted and concerned about appearing foolish in front of the professor. Furthermore, the student who visits office hours may have motives that are less admirable — the student may be brown-nosing or may be motivated entirely by getting a particular letter grade for the term and care very little about attaining a deep mastery of the course content.
My general point is that this type of grading policy is likely to be unfair. Even assuming that it would be acceptable to alter students’ grades on the basis of their character (which is a controversial assumption), it is still wrong to do so when the assessments of their character (e.g., levels of motivation, interest, and effort) are likely to be inaccurate. One major advantage of anonymous grading is that it eliminates concerns about this type of favoritism: it prevents us from being manipulated by the halo-effect and other biases derived from our evaluations’ of a student’s character and lets us focus solely on the quality of the work we’re evaluating.
I think the strongest justification for (sometimes) using grading techniques that are not anonymous is mentioned by Sara Uckleman above, since the pedagogical advantages in that kind of case may well trump the bias-related disadvantages. I also assume that grading need not be anonymous on certain assessments where interpretation of students’ answers is not an issue, such as a true/false pop quiz.
I am generally inclined to do some anonymous grading, where it makes sense (e.g., not on super small, everyday assignments which are largely, as Will Behun calls them, “completion-based”, or on the long papers that I have seen so many times that it would be silly to suggest that I could read them anonymously).
One aspect that I don’t think anyone has addressed yet is that one of the skills I think we should be teaching is the ability to take serious, evaluate and incorporate feedback. Often, part of the grade I assign is based on how well a student takes up feedback (either from me or from other students). [Note: of course, taking feedback seriously and agreeing with it are not the same, so I like to ask students to write a response to feedback in the same way we respond to comments on our papers when we submit to journals, which articulate which feedback is and which is not going to influence the revision, and making an argument for why.]
A primary assumption of many arguments for anonymous grading is that a grade should reflect an accurate and/or objective assessment of “the work.” Yet this model of grading is at best uncomfortably aligned with the way recent scholarship on teaching and learning (as well as philosophy of education) frame the value of educational assessment. On these accounts, the purpose of education is to produce change in the student and the purpose of assessment is to determine whether that change has occurred. We assess the work because it is often what we take to be the best evidence of that change. Yet this evidence is only indirect and, as with all indirect evidence, it can sometimes lead us astray. I’m sure all of us have stories of students who have demonstrated both sophisticated knowledge of the material at hand as well as an ability to perform well on the type of assignment we’ve given, but who fail miserably for any number of reasons unrelated to their actual learning (personal crises, major assignments in other classes, bad time management, etc.). Those of us who grade “the work” (as I’ve done for many years) face no dilemma here. But what if we want our grades to be a reflection of the student (or, more accurately, student learning), rather than the work (which is simply a particular performance in a particular moment)? Anonymous grading seems to block a great deal of useful evidence in this case.
Of course, none of this removes the very real possibility of implicit bias. And introducing more subjectivity into the process is likely to make that possibility more likely. But it’s important to recognize that we’re giving something up when we focus on the work instead of the student. I suppose this is one argument for course portfolios, where you can assess an entire semester’s work anonymously. But even there, you’re still blocking access to all the evidence individual teachers collect in their daily interaction with their students.
When I read a piece of philosophy or literature, my knowledge of the author often helps me to more accurately interpret that work. Why shouldn’t the same be true when interpreting student work?
I might think when reading philosophy or literature, my goal is not to judge the author’s abilities as demonstrated by the thing I’m reading, but rather to get what I can out of the piece with whatever methods I can.
Meanwhile, when grading a student’s paper, I might think my goal is not to get what I can out of the piece with whatever methods I can (after all, it’s unlikely that an undergraduate’s paper is going to have any deep insights into philosophy, especially if the aim of the paper is partially or completely expository) but rather to judge the quality of the piece, perhaps by way of judging the student’s writing abilities more generally.
So for instance if a point is vaguely expressed but comprehensible because I have personal knowledge of the person who made the point, this might fine for a philosophy article, but not very helpful for an undergraduate paper, because I want to be training my students to write papers that can be comprehended by everyone, not just their instructors and others who know them very well.
For similar reasons, if a colleague asks me for feedback on their article, I might tell them that a certain paragraph is very unclear and needs much more explanation, even though I personally understood it perfectly well because I have heard my colleague make the argument a million times. I would tell them this because I think that the article will be a better article if that paragraph is clearer, even though additional clarity will not help me one iota when it comes to understanding my colleague. Thus I might also give a student’s paper a better grade for being more clearly written, even if I know the student well enough that had it been a mess, I still would’ve understood it, and I might grade a student down for writing a mess of a paper even though, because they talked it over with me first, I know that they have a grasp of the ideas that, in this case, they failed to express. In other words, they wrote a worse paper and they got a worse grade, even though this had no impact on my ability to understand them.
There are a number of things I’m after when I’m interpreting student work, including assessing both their understanding of the material and their ability to engage with it. Betsy Barre offers a compelling account of what those things are. Part of helping students learn to write papers that are comprehensible for everyone is recognizing – and helping them to recognize – what stage of the process they went wrong at, so we can offer tailored advice for how to improve. Did they fail to read the original sources closely? Did they misunderstand the substantive points because of a reasonable misinterpretation of one or more passages? Did they understand the points but fail to explain them clearly enough for the reader? Is this the same mistake the student made last time, or is it a mistake that results from trying to do something new – or from trying to follow the advice I gave on their last assignment?
Presumably you do not have similar pedagogical goals in mind when critiquing a colleague’s papers, so perhaps you should find ways to do those anonymously instead.
I think a fair compromise position might be: Grade anonymously unless you have in place pedagogical practices that benefit a majority of your students but make it impossible to maintain anonymity. For example, I require students in upper-level courses to turn in a summary of the idea they are critiquing, an outline of their paper, then a draft of different parts of the paper, then do a presentation on the main idea of the paper, and so forth. It’s impossible for me to grade anonymously, but my students benefit tremendously from getting feedback (sometimes just checks on a rubric) at several stages of the paper writing process. As a consequence, the final papers of most of my students are much better than when I used to just read a handful of drafts from those who were already motivated/confident enough to come into office hours. But, of course, I can’t do this for all of my classes. When I can’t, then I think should aim to grade anonymously.
http://www.slate.com/blogs/xx_factor/2015/02/10/teacher_bias_in_math_new_study_finds_teachers_grade_boys_more_generously.html
People may find this interesting.
Am I the only one disturbed by the suggestion that because different assessors might have different biases that these might “wash”? I mean, surely the fact that I grade Sally higher than Joseph because my biases favor attentive female students and my colleague grades Joseph higher than Sally because he has a bias that favors quirky male students, neither Sally nor Joseph ought to be comforted by this fact. That’s not how fairness works.
I grade anonymously and think basically everything the question-asker mentions as a disadvantage of anonymous grading is actually an advantage. We are grading papers, not students.
Also, since no one has mentioned it, it is possible to give comments on drafts while grading anonymously; you just can’t discuss the comments with the students before the final draft is graded. You just need to set up some way to receive and return anonymized drafts (e.g. have them submit them to your mailbox in a “to be reviewed” folder and return them to your mailbox in a “has been reviewed” folder.) If they have questions, suggest that they puzzle over them with a friend in the class! This could be good practice for students. Now that I think about it, it might also be possible to set up a way for them to anonymously discuss comments with you over some electronic medium… I’d have to give it some thought, though.
It’s not that I disagree with your anonymous comment suggestion, it’s just that I think comments that are discussed are almost always better understood and better internalized than whatever I can fit into the margins of a paper (or into a word comment box).
In my SLAC with small classes it is very hard for me not to know who wrote many/most papers for a given class (especially upper division but also many of the lower division.) Grading anonymously seems disingenuous as it leads students to think that I don’t know who wrote which paper while for the most part I do know. It also seems to make things uneven where those that I do know get the advantage of me being able to put their papers in the context of other work but those I don’t know don’t get that. Finally, students benefit from coming to office hours and discussing their papers before they write, not all do it, but many do… this in effect makes their papers not anonymous anymore but it is a valuable part of their education that I would not want to take away. I think there is value in anonymous grading and I may endorse it for large courses. In smaller courses, the advantages may be outweighed by the disadvantages.
I think the worries about downsides to anonymized grading in this thread often seem to rely on the assumption that the reasons for anonymized grading entail all the different sorts of assignments and exams in a course must be anonymized when marked. But it is not the case that the benefits of anonymized grading force the conclusion that assignments of very different sorts must all be anonymously graded. So worries about the downsides likewise should not lead one to conclude that one should never grade anonymously. Certainly some assignments are more for the purposes of development than assessment. Others, like in-class final exams, are for the purposes of assessment moreso than development. Wouldn’t we all agree that in the same class, some sorts of things should be anonymized before evaluated for a mark, and other sorts of things should not?
I have my students put their names on the backs of papers. This is not an absolutely perfect system because sometimes I’ve discussed paper topics with a student, or students email papers without a name attached but I might faintly recall who sent what. Luckily it doesn’t have to be a perfect system to be generally useful. I expect that I’m unintentionally biased in favor of students I’ve talked to more about philosophy or have had in classes previously. I see this as a reason TO grade anonymously rather than a reason weighing against it. I have a large enough number of students and a poor enough memory that occasional imperfections in the process are unlikely to detract from the overall benefit. Because of the larger numbers I cannot always review drafts in advance but I do this whenever possible. I still have them put names on the back of the final draft because then I don’t immediately think of the student who wrote the paper when I see it – at least not consciously – and I prefer to read that way. I’m similarly concerned about unintentional bias against students who choose strange fonts or non-standard margins, so I ask them to use standard fonts and margins. Again, an imperfect system but better than it was before. And now I must return to reading those papers!
Are there any studies indicating whether students respond better (or worse) to personalized comments? Though inclined towards anonymous grading to avoid possible biases, it seems like it attempts to turn every small class into a large anonymous lecture hall (only by not paying enough attention to students views can the grading be kept truly anonymous.) Are there any studies that show the benefits of anonymous grading in small classes in the Humanities (no multiple choice exams, lots of discussions of students views, students expect and regularly receive individual encouragement)? And whether those benefits outweigh the costs of personalized encouragement (a student asks you to write a letter but you don’t remember their paper, you don’t know what topic they wrote on and can’t refer back in later conversations… etc)? The study linked to above (in the article from Slate) seemed to show bias in Math but not in English and writing classes… do we know that Philosophy is more like Math than English and writing?
Wait, of course there is bias in English assessment. The reason the study discussed in the Slate article is so noteworthy is because everyone assumes Math is *so objective* that bias couldn’t affect grading, but actually studies show that it does. Whether Philosophy is more like Math or English doesn’t matter, because in both cases biases affect assessment of work. We are biased. It affects our assessments. If you care about grading students’ work and not your preconceptions of your students’ abilities which are affected by implicit biases, you should try to implement anonymous grading policies.
I started using anonymous grading in my first independently-taught course as a grad student, over 25 years ago. I had a student in that summer course who had originally taken the course with me as his recitation leader the term before. Another TA caught him cheating on an exam, so he failed the course. When he retook it from me I didn’t want to hold the previous cheating against him. I tell students that I grade anonymously where possible but that they shouldn’t let this prevent them from bringing drafts to my office hours. So few students do this that only rarely is anonymity compromised in this way. I consider one of the chief benefits that it sends the message that I take fairness seriously. I teach ethics, but I don’t think that my readings, lectures, etc. do much to make students morally better. Perhaps, though, modelling ethical behavior in the classroom will make a small difference in this regard.
I second much of what above posters had to say regarding possible workarounds in response to some potential downsides to anonymous grading and the idea that it does not have to be all or nothing. I grade anonymously where I can. So that means for most assignments for my 30-40 person 100-200 level courses and not at all for a 4 person 300 level course. In between are 20 person writing courses where revision is a significant aspect of the course. I still tend to have students anonymize the final draft, though I will certainly know to whom *some* of those later papers belong, though often I actually don’t remember whose paper is whose 2 weeks later (maybe I just have a bad memory?). Of course in any class, students who come to office hours to discuss ideas or have me look over an outline or draft will almost surely not be anonymous to me when it comes to grading. Also some students have really distinctive hand-writing or use distinctive paper when handing in in class assignments and such and I can’t help but know who they are. And some also just noncompliant about keeping their names off the work they turn in. So it’s not perfect anonymity, but then I am not really striving for perfection.
When I first tried anonymous grading a number of years ago, I was drawn to it primarily because of implicit bias-type concerns regarding race, gender, etc. But the experience of doing it convinced me that a possibly even more salient kind of bias that became very obvious when I would reveal the student after assigning a grade. So often my first thought would be “Well I must have graded too harshly because that student really seems to get things in class discussion–how could they only get a B on the paper? They probably really deserve an A-.” Except no, I didn’t grade particularly harshly on *that* student’s paper–I graded the same as I did for every student; it’s just my expectation was that this particular student would earn a better grade. Had I seen that student’s name on the top of the paper, I would indeed have had that expectation going in to reading the paper. How could that expectation then not already bias me toward believing the paper deserved a high grade without having read a word of the text? In fact, my urge to tack on a few points once the student’s name is revealed–because I just must have been wrong about the quality of the paper–is so strong I have to rethink it through and stop myself each time.
My urge to go in the opposite direction–“Wow, that student has never said a word in class and I assumed they were uninterested in the material, but here they just earned an A-. I must have been to soft on this one. I should go back and be harsher.”–is much weaker. But I still have the moment of surprise and realize that my expectation of the student turning in a mediocre paper would have been set from the get go and possibly led me to assign a lower grade had I known whose paper it was. Perhaps there are other counterbalancing reasons against anonymous grading that I haven’t considered well enough, but my own experience of grading anonymously has made clear to me how shockingly biased I am in a completely mundane (and not particularly “political”) way. I think given I now know this about myself, it would take significant counter-balancing evidence–that I am actively doing harm or creating even more unfairness in a different direction in employing anonymous grading wherever I can to go back to traditional grading.
Amanda, I have *exactly* the same experience almost every time I record grades. I am almost always surprised by a student’s performance, and I am so tempted to bump them up. I always resist. This is the main reason I continue to think that anonymous grading it is really very important.
I have had the same experiences that Amanda and Kathryn have had, and I second everything Amanda said.
Me too! I started anonymous grading due to implicit bias worries, found myself regularly surprised at which students got which grades, even to the point that I second-guessed myself, and continued blind grading due to these surprises. Of course, there are downsides to the practice of blind grading, and as several comments have noted, there are ways to minimize those downsides. But a huge upside is that blind grading helps us screen off, for grading purposes, our prior views (biases and *also* reasonably held views too) about how well each particular student will do. Blind grading helps (it isn’t perfect, but it helps) minimize the extent to which the data on which we grade are theory-laden by these views.
Given that several people have reported the same experience with blind grading, I wonder whether it is a common experience among those of us who have tried the practice.
I agree with all the above. If I don’t grade anonymously, I’m subject to lots of anchoring biases based on very superficial information about students. If a student says cogent things in class and I know this is her paper, I might begin with an assumption that it will be very good. The burden of proof is on the bad grade, so to speak. If a student says nothing or says a few things that are off point, the opposite could hold. I’ve been anonymizing grades for years and have never found a good reason to go back.
Kathryn, the article from Slate cited above says that the effect was not the same in other subject, like English and Hebrew (in the long block quote):
http://www.slate.com/blogs/xx_factor/2015/02/10/teacher_bias_in_math_new_study_finds_teachers_grade_boys_more_generously.html
do you know of other studies that show bias in Humanities courses?
One of the co-authors of the article from Slate, Victor Lavy, found a strong anti-male bias in the grading of humanities and science courses. The study used the same early 2000s Israeli data set.
http://www2.warwick.ac.uk/fac/soc/economics/staff/vlavy/lavy_j.public.e_10.2008_gender_steriotypes.pdf
My point was that there are biases that affect assessments in English, not that biases are the same in every discipline. We should ensure that our biases are unable to affect our grading regardless of what those biases are. So, who cares if Philosophy assessment is affected by the same biases as Math or as English; in all cases biases affect our assessments if we let them. This is not controversial.
If you want to fairly assess the work your students actually produce, you ought to grade anonymously.
Kathryn (or anyone else for that matter), do you know of any studies that show biases in the assessment of English (or Humanities) papers? the Slate article suggests that the particular study being discussed indicates that they didn’t find such biases. It’s hard to tell why you don’t think it’s at least a reasonable question.
Given the overwhelming evidence we have that how we grade is influenced by our perceptions of students, I don’t see why the burden of proof is on me to show that there are biases in all disciplines. We (as individuals) have biases. Some of them are probably fairly innocuous, some of them are quite problematic. We should work to make sure that neither kind influence our assessments of student work. I don’t see why people are pushing back against this.
There are weighing concerns people have, that I understand. I disagree that we should weigh non-essential pedagogical benefits against (what strike me as) considerations of justice. But that’s a later-stage disagreement. Here is the main claim: that we all have biases (though those biases might differ between individuals) give us all reason to grade anonymously.
” I then use the heterogeneity in teachers’ discriminatory
behavior to show that classes in which teachers present a high degree of discrimination in favor of
girls are also classes in which girls tend to progress significantly more than boys, during the school
year but also during the next four years. Teachers’ biases also increase the relative probability that
girls attend a general high school and chose science courses.”
http://cep.lse.ac.uk/pubs/download/dp1341.pdf
“We then show that in maths, the classes in which teachers show
greatest bias in favour of girls are also the classes in which girls
progress the most relative to boys. This finding is in line with other
research highlighting that grading practices affect pupils’ motivation
and progress. It is also a possible explanation for the reduction in
differential achievements in maths observed between girls and boys
during junior high school.”
http://www.ipp.eu/wp-content/uploads/2014/12/n14-notesIPP-december2014.pdf
Very interesting in that this shows in France, math teachers have a pro-female bias in their math grades. Directly contradicts the article referenced in Slate that used data from Israel. That suggests that gender bias is not uniform across cultures, but rather culture specific.
In the end, I’m in favor of anonymous grading, and I do grade anonymously where appropriate (e.g., for papers and final exams). But there are some things worth noting here:
– postsecondary institutions occupy a rather strange place between a more rigid, formal educational system (i.e., primary and secondary education) and the workforce. And you’re not going to be evaluated anonymously in the workforce. Because of this, it seems entirely appropriate for postsecondary instructors to assign things like group work and presentations, which by their very nature are evaluated non-anonymously.
– students already have a tendency to vastly over-estimate the importance of the grade they receive in class, relative to other factors (e.g., the life skills they pick up, the transferable cognitive skills they gain, the social networking they engage in, etc.). I’m hesitant to do anything that exacerbates this. This is why I grade anonymously, but don’t make a fuss about it with students. I want them to think less about grades, not more.
This second point is really important. Concern about fairness in grading often assumes that grades are really important. In general, I think that learning is more important, and grades are primarily useful as a means of incentivizing the attention and effort the leads to learning.
I only began anonymous grading two years ago, but I am a complete convert. I agree with what Kate Norlock (December 15, 2015 at 5:16 pm) and Amanda Roth (December 15, 2015 at 9:57 pm) have said.
I wrote a post on this that some might find of interest. The link is here. http://jacobarchambault.com/2015/07/27/on-blind-review/
The immediate topic of the post is blind review, though the considerations can be (and are) applied *ceteris paribus* to issues of grading. The basic concern is that the use of anonymity doesn’t likely support – and may in some cases undercut – the ends for which anonymity is usually employed as a means: increasing fairness and eliminating bias.
There is a second issue I neglected to mention: the problem that anonymous grading creates for detecting bought papers and/or plagiarism. In order to know whether a student actually wrote the work they claim to have, one must have some grasp of their relative strengths and weaknesses in writing. The less one is able to attach a body of work to the student in the first place, the less surely this can be done. Software like Turnitin can detect the obvious cases, and of course this problem arises more for some kinds of anonymous grading than others (e.g. depending on whether the paper is attached to a name immediately after being graded, or whether all grading throughout the term is simply assigned to, say, an ID number), but more subtle cases may require a discernment of how a paper accords with other ways the student demonstrates her/his abilities.
You can always think about that after revealing the student’s identities and before recording and communicating the grades. That’s what I do.
I don’t hesitate to “pierce the veil” of anonymity when I have concerns about plagiarism. I had a case in which a student turned in an essay on feminism that was clearly written by a woman (this was clear from the pronouns that were used, I hasten to add, not the content of the views). I was suspicious as soon as I started reading, but when I looked and saw it was written by a man, well, game over. Apparently he thought that I’d never realize his gender.
I make sure to give clear instructions about what is required for anonymizing papers for grading, and one criteria is that they do not include information from which I can identify their race, gender, etc.
For what it’s worth… the most useful feedback I tended to receive as an undergraduate was always something of the ilk “I get what you’re saying here (in the paper), but the way you described it in class was a lot clearer … or isn’t the same for reasons X and Y… etc.”
I would without doubt be a worse philosopher (and have been a worse undergraduate) if it weren’t for comments like these.
[That said, final essays tended to be marked by different people to those who’d seen the drafts]
Honestly, the notion that the grading of philosophy essays can be somehow “objective” or fair is bullshit even when the process is anonymized. As for implicit bias, I don’t think anonymity gets rid of that, especially in cases where writing style, vocabulary, examples, and so on will reveal much about a student’s race, social class, region of origin, political beliefs, religion, and gender. The best way to combat bias is to attack it head on by asking oneself if one is giving a student a better or worse grade on account of the student’s race, gender, religion, etc. At least in the small undergraduate classes I teach in the USA, anonymous grading would seriously hinder my attempt to read and helpfully evaluate my students in a manner sensitive to who they are as individuals. If I had my way, grading would be abolished and replaced with a written statement by the instructor reviewing each student’s work and progress in a course. I know, of course, that won’t happen and we will descend further into the faux-objectivity of blindly administered assessment.
Since I’m reading this while taking break from what I should be doing (grading those essays!), trying my best to be fair in the process, allow me to ask why you think that the notion of fair grading is a poor one in the first place? After doing anonymous grading for several years, I’ve often found any initial assumptions that bubbled up in my mind about the student writers were incorrect. I find myself making far fewer such assumptions now, perhaps as a result of the practice. I don’t mean this to sound callous, but I honestly do not want to grade in any “manner sensitive to who they are as individuals” because even when I know some students pretty well, I don’t know all of them well enough to make decent judgments on that basis about their work. I would much rather stick to factors like whether a student has a clear thesis statement, or considers likely objections to an argument, or cites the text accurately. I’ve told all of them how I would like this to be done, so everyone has the opportunity to satisfy the standards they know I employ. It’s not a perfect (or perfectly “objective”) scheme but it seems like a fairer measure of success than anything more personalized I might devise. If I am missing something, however, I am interested in hearing more. Grading is difficult and I’m always trying to learn better ways to do it. And now back to the pile of papers!
Dear Laura:
I am largely in agreement with you, here. The idea that we can’t be fair is, at best, overstated as you point out. Perhaps Avi Z is unable to be fair to his students, and experiences some frustration because of the importance placed on grades which he sees as unfair by their nature.
I think the claim that we cannot be objective is more reasonable, though perhaps overstated as well. To be sure, we all bring our biases to the table, but as professionals I would imagine we also are rigorous in our standards and try to create situations in which the effect of those biases is minimised. Perhaps I’m biased against a student that asks me questions that are answered in the syllabus; that does not necessarily mean that I will grade him or her unfairly. Since grades reflect completion in most of my classes, there’s little room for my bias to have any impact. Anonymised grading is just one more technique to temper the effect of bias. Does it solve every problem? Of course not. Can it help? in some cases, sure. Does it carry with it some risks? Naturally.
I think that we can and ought to be sensitive to who our students are; but this seems more like something that is reflected in feedback more than grades.
Thank you for taking the time to read this. Now back to the grade mines with you. 😉
New research suggests that the attractiveness of female students is correlated with their grades. Perhaps another reason to implement anonymous grading: http://www.slate.com/articles/life/inside_higher_ed/2016/01/more_attractive_female_students_enjoy_higher_grades_in_the_classroom.html