How Do You Want Your Students To Assess You?
Most colleges and universities have students fill out forms to evaluate their instructors’ performance at the end of the term. Semesters are just beginning around now, but it is not too early to start thinking about those evaluations. One study found that “students’ ratings 2 weeks into the semester did not differ from end-of-semester evaluations.”
There are lots of problems with typical student evaluations of professors (see here and here, for example, and this post by Michael LaBossiere at A Philosopher’s Blog), but they are probably here to stay. They can be changed, though, and supplemented. I have heard of departments and individual professors adding questions to the standard forms, or gathering up their own feedback separately.
With that in mind, I thought it might be worth discussing what a good version of a student evaluation of a philosophy course and instructor would look like. Which questions should be asked? What form should the answers take? When and how often during the term should the evaluation take place? What procedures should be in place for administering the evaluation? And, most importantly, what kind of chocolate should you give to your students just before they evaluate you?
(art: Roy Lichtenstein, Mirror #10)
There is one nice trick that might help: formulate a question so that it directs the students’ attention towards themselves and how they could have improved the learning experience. To make them reflect on their own role in the process could make them understand (a bit) that teaching/learning is not just about the teacher.
But, sigh, isn’t there also a study somewhere that shows how important the very first *seconds* in class are? And I am not sure what this tiny first impression really shows about me as a teacher…
I don’t mind the current system, but I would also like to see an “exit interview” at the time of graduation, asking students to look back and evaluate their courses in retrospect.
I tell the students, at the end of the term, that the administration is interested in the numbers, but I’m more interested in their substantive comments about the course and their experience in it. The online survey my institution uses gives students the opportunity to write comments.
I ask them in particular to tell me what works well about the course in helping them to learn and to meet the stated objective and to tell me what could be improved. The more concrete and specific their feedback, I tell them, the more useful it will be in my ongoing design process.
The comments are always a mix of the thoughtful and the . . . less than thoughtful, but I usually derive some insight from them.
One problem with this system is that the end of the semester is too late for those students themselves: whatever I learn benefits the next group of students, or maybe the group after that.
At the moment, I’m thinking of setting up an anonymous online “suggestion box” and invite students to offer observations, questions and concerns about the course design or my own teaching. I include in my syllabi a statement on what students may expect of me, and it’s worth knowing up front whether they think I’m living up to those expectations.
It seems only fair, since I’m always telling them – in grades and in comments on their writing – whether they live up to my expectations of them.
Students shouldn’t assess professors. They’re not consumers and instruction isn’t your standard product. I’ve come to think that if you administer a good assessment of the student learning outcomes for that course, where those outcomes are carefully specified by the instructor and department, that tells you more about the instructor’s course design and instructional effectiveness than surveys. You can send that up the chute to program and institutional assessment czars to boot. I’m not sure what else is relevant for the purposes of assessing instructors.
Bob Kirkman’s reflections reveal the dual use to which evaluations are put. They are used in promotion, tenure, and salary review as well as developmentally. If evaluations are here to stay (and I hope that they are not, given the clear evidence that they do not track educational quality and do track social biases) then they should stay only for the purposes of development. I find the written evaluations and even some of the numerical evaluations useful for thinking about what I can do to experiment in the classroom. I filter these comments through my long experience and through my own sense of what I can do to make sure that students learn something, but they do make me think about things I might not want to think about. But if they are going to serve this purpose, they should not be used in promotion, tenure, and salary review. In particular, they should not be used as the sole piece of information about teaching in a promotion, tenure, and salary review file. Like Justin, I am eager to hear ideas about what could replace them. I understand that we need evidence of good teaching for promotion, tenure, and salary review. What realistic practice could be adopted to gather and present this evidence?
I guess I’m less concerned about how students should assess faculty and more concerned about how administrators and departments use and abuse those assessments. I think student assessment of faculty should consist in a small number of open-ended questions that get at how much effort the student put into the course, how much the student learned, and how effective the faculty member was. But most importantly, the results of such assessment should be shared only with the faculty member and a mentor of the faculty member’s choice. Administrators shouldn’t see them, as they’ve shown every inclination to use them for purposes that are neither statistically/psychometrically valid nor proper in running a university. They can serve the dual purpose of faculty development and helping give students a voice. I think these comments are very much along the lines of what Becko says above. I also definitely agree with David’s suggestion for an exit interview, and would be interested in other ways of giving students a greater institutional voice without playing into administrative abuse of power over faculty.
When I found out our administration not only uses student evaluation scores as part of an employee review, but also shares these scores with the rest of the department, I was a bit perturbed, to say the least. While my scores have been perfectly fine, I neither want them shared with my peers, nor do I particularly want to know theirs.
On the other hand, we do employ graduating student exit evaluations (not interviews) that encourage them to be specific about the classes, professors and other staff that affected them, positively or negatively, while they attended. My concern with these is that human memory is very provably fallible, and after four or so years of many classes and instructors, conflation and/or misremembering could be a cause for inaccurate evaluations.
Let me (again) take the opportunity to advertise the Oxford (and, to some degree, UK) model where teaching and examining are sharply separated. My students aren’t incentivised to prefer tutors who are easy graders because the tutor’s grades don’t contribute to their final degree class: only anonymously-marked work does, submitted to a pair of unknown markers one of whom may be the tutor but probably isn’t. So the tutor who sets little work and grades it lightly isn’t working in the interests of the students’ overall class, and the students are generally smart enough to know that. And (this speaks to LaBossiere’s final point) in any case a student evaluation isn’t really needed to tell how tutors are doing, because their students’ marks in the final exams will give a pretty clear indication.
No doubt there are many advantages to the US model where one person sets the syllabus, teaches it, and examines it. (I’ve not taught in that system; I wouldn’t presume to judge it.) But it does have costs and it’s worth being aware that there are other ways to do things.
I couldn’t agree more strongly with Becko Cophenaver’s point at 5: In my experience (as a white, cis-gendered heterto male with a an impressive beard), student evaluations can be incredibly helpful as input for teaching development, but mostly useless as an independent measure of effectiveness.
I will say that the student evaluations I received at one university (available on my website, together with the name of the relevant institution) were particularly detailed and helpful in that regard. While it’s possible that there was something about the particular questions or particular students, I think one key ingredient was that the evaluations were administered by the department work study. I don’t know if he said anything in particular to encourage students to take the exercise seriously, or if just the fact of having another student administer it meant that they felt that their opinion as students really mattered.
My considered opinion has long been that student evaluations probably do more harm than good, but then I got this one today, and suddenly I am sort of glad that the practice exists.
“9.5/10; would take again. I just realized that rhymed And now I’m feeling sublime. They call me Bertrand Russell You know, I’m all about the hustle Sense data is my life, it’s also yours, this ain’t a puzzle. I get recognized as “Ayer”, and “A true fuckin’ player.” but those terms are interchangeable my bad, ’twas a tautological err They sometimes call me Quine. I got bars, I got lines. I’ll eradicate your whole existence, (What existence?) it’s not in our space and time. I’m known round here as Kripke, The king of New York City They tried to bind me to their laws, too bad their laws weren’t explained to me. Aristotle, Plato, Socrates, Machiavelli, Mephistopheles Metaphysics is our business. Our swagger’s in our Philosophies!”
I do have doubts about the wisdom of using student evaluations of teaching in annual review and in tenure and promotion decisions. I teach at a large research university, so there is some pressure for assessment to be “scalable” . . . which means reducible to a few numbers that may be reported and digested without fuss.
At the level of my department, the numbers are reported, and we do discuss them in P&T meetings, but we don’t take them very seriously: the only time they become an issue is when the numbers are unusually low and, even in those cases, there are usually special circumstances to account for it.
What we take more seriously is peer evaluation of teaching: every few years, I am required to invite a committee of three peers from within my school to look over my course materials, attend one or more class sessions, and submit a report. Under the system, I have the right to review a draft report, offer comments and suggest amendments.
This raises a question, though, and one suggested by some of the comments, above: Why are we so reluctant to let others see and assess how we teach? Either we are supremely confident in our own wisdom and skill in the classroom, or we are supremely insecure and afraid of being exposed as frauds.
Neither attitude, I suggest, is conducive to excellence in teaching.
Some of the most valuable things I have gained about teaching and learning have come from the peer evaluation processes, whether from visiting others’ classrooms or having them visit mine as part of the formal evaluation process, or from informal discussions of teaching and learning in which my own approaches and expectations come to light for others to review and discuss and critique.
And, it must be said, I have even learned things from student evaluations. It become clear to me, a year ago, that students found my handwriting indecipherable: nearly every student commented on it, in one way or another. I have since set about trying to improve my handwriting, with mixed success.
Bob Kirkman: I think that peer evaluation contains many of the same promises and perils of student evaluations. When used as part of a good-faith collaborative effort to develop as fellow teachers, I think they can both be invaluable. But if you have reason to be suspicious about how such evaluations are going to be used, you can be reluctant to be evaluated without being overconfident or insecure about your teaching abilities. We often have good reason to be suspicious of what use our evaluations are going to be put by administrators (and sometimes by colleagues and department heads). I agree that such suspicion isn’t conducive to teaching excellence, but such are the results of the practice of pairing ‘accountability’ with justifications for austerity in education.
“There are lots of problems with typical student evaluations of professors (see here and here, for example, and this post by Michael LaBossiere at A Philosopher’s Blog), but they are probably here to stay.”
Self-fulling prophecy? Can we please allow ourselves to consider all the unfortunate feedback loops that result not only from the “student evaluations of teaching,” but also from the attitude that, no we don’t like [whatever it may be], but we probably can’t do anything about it, so we have a good excuse not to even consider rocking the nice, comfortable, but miserably un-educational boat we’re all riding in–? Hey, where does social inertia come from, anyway?
Student evaluations seem universally reviled by faculty, but maybe they actually have served an important purpose (I don’t know, just offering a hypothesis):
I remember reading an interview with an academic psychologist who had spent several decades teaching, and he asserted that college professors today were overall much better teachers than they had been at the start of his career, where it was common just for the teacher to spend the entire class time just bent over the podium reading from the textbook in a droning monologue. Maybe the idea that professors might have to actually be engaging at some level has helped end that practice as typical.