Student Evaluations of Teaching

We’re confusing consumer satisfaction with product value.

That’s Philip B. Stark, a professor of statistics at Berkeley, discussing a mathematical critique of student evaluations of teachers he has written with a colleague, Richard Freishtat. There’s an article about the critique in The Chronicle of Higher Education. The study itself is here. Here’s a recap of major points from the study:

● We might wish we could measure teaching effectiveness reliably simply by
asking students whether teaching is effective, but it does not work.
● Controlled, randomized experiments—the gold standard for reliable inference
about cause and effect—have found that student ratings of teaching
effectiveness are negatively associated with direct measures of effectiveness.
Student teaching evaluations can be influenced by the gender, ethnicity, and
attractiveness of the instructor.
● Summary items such as “overall effectiveness” seem most susceptible to
extraneous factors.
● Student comments contain valuable information about students’ experiences [not necessarily teacher quality].
● Survey response rates matter. Low response rates need not signal bad teaching,
but they make it impossible to generalize reliably from the respondents to the
whole class.
● It is practical and valuable to have faculty observe each other’s classes at least
once between “milestone” reviews.
● It is practical and valuable to create and review teaching portfolios.
● Teaching is unlikely to improve without serious, regular attention.

Newest Most Voted
Inline Feedbacks
View all comments
David Wallace
David Wallace
6 years ago

It’s only problematic for student assessments of teaching to be affected by the attractiveness of the teacher if there’s independent evidence that teaching quality is unaffected by teacher attractiveness. Is there such evidence?

(If I had to guess, I’d predict that people pay more attention to attractive than unattractive people and so attractiveness is a good thing to have as a teacher; it’s easy to make up a just-so story the other way around, though.)Report

6 years ago

There’s also the important question of what counts as “effectiveness” in the humanities. Is our task as teachers the distribution of facts with the long-term goal of maximum retention? If not, then these studies aren’t tracking effectiveness as we ought to understand it. I also suspect that if we were to look at what good teaching in philosophy really is, it will be more strongly correlated with charisma (and other so-called ‘superficial’ qualities) than critics of teaching evaluations usually suppose.Report