Oral Exams in Undergrad Courses?


Between the developments in large language models (like GPT-3) and their possible use by students, and being in the thick of end-of-term grading of papers, the idea of making use of oral exams, as suggested in a recent New York Times column, seems tempting.

[Avigdor Arikha, “Self-Portrait with Open Mouth” (detail)]

It would be useful to hear from philosophy professors who have used oral exams in undergraduate courses. What are the advantages and disadvantages? What are some tips for doing it well? What are some problems to try to avoid? How structured are they? How uniform across students? Do you use a rubric, and if so, what’s on it? What kinds of philosophical questions do they work well with, and which not? What do the students think of them?

And for those who haven’t used oral exams in this setting: do you have other questions you’re hoping could be answered by those who have? Do you have particular concerns about doing so?

Tell us all about it. Thank you!

guest

25 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Nick Laskowski
1 month ago

I’ve been administering oral exams to advanced undergrads for ~five years now. It’s a practice I picked up while a post-doc in Germany, where oral exams were a standard part of the curriculum. I’m now something of an oral-exam evangelist.

Generally, I require students to study two complimentary texts from the course on which they expect to write their term papers. I invite them to begin the exam with a ~3-5 minute summary of the texts’ core parts. I then ask them mostly prepared follow up questions for ~15 minutes to further probe their comprehension and ability to critically or constructively explore a view or argument. I run these ~20 minute sessions back-to-back over the course of two days in classes of ~25 students, typically.

Nearly all of my students report never having been examined orally. Most of them are visibly nervous at the outset. But my students report overwhelming at the end of every term that they appreciated the experience. Nearly all of them tell me that orals led them to know a text more deeply than any other assignment type. And in preparing for their orals, students are then in a position to write their term papers breezily.

When I’ve mentioned this practice to other teachers, the most common worry I hear expressed is that orals unfairly privilege clever students that can think on their feet. I see no reason to think that the various ways in which other assessment types privilege other skillsets, like the ability to write, are any less unfair. Are orals perfect? No. But all assessment types come with trade offs. Oral exams are one among many useful tools for teachers to have in their kits.

I’d be happy to chat about my experience with oral exams in more detail over email if anyone is interested.Report

Postdoc
1 month ago

I love this idea and want to do it.

One question (for Nick or anyone): Would you audio record the conversation (with everyone’s knowledge and consent, obviously)? The reason I’d be inclined to is that, once I start assigning grades, I wouldn’t want to rely solely on my memory of the conversation with no option to review the audio. I might forget part of what they said, or how they said it, what they left out, or conflate my conversation with one student with a conversation I had with another, etc. It would also come in handy if a student challenges a grade. I might want to say “You said X, but X is not correct.” If they say “I did not say X! I said Y!” and I have no recording, I’m in not-so-great shape.

My only concern is that recording things might make students feel even more anxious. Or maybe there are privacy issues in this vicinity? Feels weird having a tape recorder sitting on my desk as we talk, but maybe that’s just my hang-up.Report

Chris
Chris
Reply to  Postdoc
1 month ago

This is a good question. When I was a graduate student, I was a Teaching Assistant for a course with oral exams. We didn’t record them. But one student was very upset by their grade, and we didn’t have the usual (written) things to point to to justify the grade. (Of course you have your own notes that you took, but those can be disputed.) It was a bad enough experience that the instructor I TAed for never used oral exams again. But recording them might have solved this problem.Report

Nick Laskowski
Reply to  Chris
1 month ago

Disputes can be an issue, though I haven’t experienced them myself.

I have recorded for this reason on top of taking notes in the past. Not sure it makes them any more nervous.

However, since covid, I’ve gotten into the habit of running orals over Zoom. That comes with the added benefit of auto-generated transcriptions to use in grade disputes. I’ve also noticed that students are more at ease taking orals over Zoom than in my office. Two birds, one Zoom?Report

Kenny Easwaran
Reply to  Nick Laskowski
1 month ago

Zoom is also particularly useful for short meetings – I schedule 15 minutes to talk with students about their papers after they’ve been graded, and over Zoom, a 15 minute meeting is fine, but getting someone to come all the way to my office (and probably have to sit outside for a few minutes) for a 15 minute meeting would be really annoying.Report

John Keller
1 month ago

I’ve used oral exams a fair amount, with good results. I usually have four a semester, so they’re not covering too much material. I then break the material into three or four “chunks” (if you use a textbook, chunks will likely correspond to chapters): each student is asked, at random, about one of those chunks. I distribute a study guide ahead of time with a bunch of “concepts”* on it. Students can choose to either have me ask them questions, in which case I ask them about the “concepts” on the study guide related to their chunk, or they can “give their own spiel”, in which case they can talk about whatever they want, as long as it is related to the chunk I’ve asked them about, and as long as they cover the most important “concepts” related to that chunk. I print out a bunch of “evaluation forms” that have the “concepts” related to each chunk on them so that I can record what people talk about and how well they talk about it. This works reliably enough, imo. Oral exams are scheduled for ten minutes each: I have them sign up on a google doc, and I make sure to schedule breaks for myself so that I don’t get burnt out, and so if I get behind I can catch up. While the slots are ten minutes, the examination portion is really only 8-9 minutes so that I can give students some brief feedback.

I’ve experimented with letting students use notes on one side of a 3×5 card during the exam (they bring multiple cards so they have one corresponding to the chunk they’re asked about). That seems to make students much less nervous, but some students will just read from the card. I penalize them for that, somewhat significantly: they’re supposed to be demonstrating their ability to explain things, not their ability to read. (Students are told there’ll be a penalty for excessive reading ahead of time.) Still, I’m not sure allowing even these minimal notes is a good idea. Plus you have to police the notecards a bit. An alternative, which I think I prefer, is to hand them a piece of paper with their chunk’s “concepts” on it. I’ve also had them use nothing at all. That works, but a much higher percentage have me ask them questions. Since part of the value (see below) of the experience is tied to “giving your own spiel”, I’ve tended in the direction of allowing a crutch of one form or another in recent years.

In general, I think this approach has worked well. I “sell” the idea to the students by saying that being able to explain things verbally is a skill that they’re likely to need in the future, whereas taking written exams is something that they’ll almost never do once they finish college. I explicitly talk about the value of “steering” a conversation (towards things they know better) if they give their own spiel, and how the ability to steer a conversation can be useful in job interviews, presentations, conversations with in-laws, etc. I should note that I think all this is true, it’s not just a sales pitch! I’ve had success with this method in a variety of different courses.

Advantages: I prefer the experience to grading, it’s very rewarding when students do well, some students really get into it and shine, it’s nice to interact with students one on one, if someone just gets “mixed up” there is the opportunity to clarify things with a follow-up question or whatever.

Disadvantages: It’s awkward when students do *very* poorly, although that’s been rare for me. Policing notecards is annoying. Sometimes students don’t show, or don’t sign up. Very rarely I’ve had trouble rescheduling such exams, but in those very rare cases it was pretty infuriating. (I should note that I have a bunch of exam slots during “class time” on the day when an in-class exam would be, so that I know there are slots that fit everyone’s schedule.)

*I’m putting ‘concepts’ in scare quotes because many of the items aren’t really concepts, but arguments, objections, etc.Report

Monte Johnson
1 month ago

I’ve been giving oral exams for the last three years. In my larger lower-division (freshman and sophomores) course, I meet with each student over Zoom for 5 minutes at a midterm and then a final examination. I carefully schedule each student and distribute the schedules and zoom information well in advance. I record the sessions. I prepare a standardized set of questions for each day of the examination, but change the questions each day. I type the exams (and answers) out and print out a copy and put the student’s name at the top of the page. During the exam, I make various marks on this page (plusses, minuses, check marks, Xs, etc.). I also put a provisional grade (like A, B, C, D, F) at the bottom of the page. After all the exams are done, I grade them in alphabetical order and report these grades to all the students at the same time. (I do not show them the marked up pages with my notes.) The questions are sometimes True/False, sometimes “fill in the blank”, sometimes descriptive (e.g. explain Aristotle’s definition of the soul), sometimes comparative (e.g. compare Callicles’ view of desire with Epicurus’), sometimes evaluative (e.g. do you think Epicurus or Epictetus has a better view about justice, and why). Depending on the student’s answers, I may ask clarificatory or factual questions or, on the other hand, more open-ended invitations to discussion.

What surprises me most about this method of grading is how much easier it is to figure out whether the student has actually done the reading, whether the student has given any thought to the reading, and whether the student has any intelligent thoughts about the reading. I found that with written exams, students could use mere linguistic competence to write something down that seems to deserve at least partial credit, even if it is not clear that they really have read, thought about, or had any intelligent thoughts about the reading. The pretense of having read something that you haven’t actually read seems to disappear when you are speaking directly to someone you can interrupt and ask for clarification, examples, or textual references. The problem mentioned above about “clever students who can think on their feet” seems to me a much bigger problem when you are dealing with a frozen block of their writing than when you are able to cross-examine them on the spot about what they are saying.

By the way, five minutes of continuous interaction with a single student turns out to afford more opportunity for questioning than it might seem at first glance. On average I can ask 3-7 questions, and in some cases add follow-up questions. Sometimes thing get bogged down and less than this is possible. But I have not found that there is a large advantage to, for example, doubling the time of the exam. (I started out with 10-minute exams, but eventually reduced them to 5-minute exams).

As for student nervousness, I find that you can reduce that by preparing them in advance with examples of the kinds of questions you are going to ask, and in fact make it a practice all along to discuss just these types of questions in the class during lecture. Also, ask them if they would be less nervous writing a three-hour exam or speaking about the material in a five-minute exam. I have had only positive reactions to the oral exam in student evaluations, and I have never had a grade dispute (although I have had one request for an explanation of why a certain grade was received).

In my upper-division (junior and senior) course which is restricted to philosophy majors, there is a research project which involves writing a proposal and several drafts. At the end there is a “final research conference” at which each student presents the results of their research before the entire class for 5 minutes, followed by 5 minutes of questions or discussion. Here again weak student research becomes obvious (sometimes painfully obvious), but overall students seem motivated to work harder knowing that they must speak about their research in front of other students and myself, and must be able to answer questions about it coming from both their peers and me.

In general, I feel a lot better about employing oral examinations than I did about written examinations. As the author of the NYT suggests, there is something traditional and wholesome about teaching philosophy with oral exams. The kind of philosophy that Socrates practiced seems to have consisted of getting together with a group of youngsters and examining their thoughts about the virtues. I begin to wonder why we ever got away so far from this and got stuck with the idea that we should evaluate their ability to write Montaignesque essays.Report

Tom Sparrow
Tom Sparrow
Reply to  Monte Johnson
1 month ago

Monte, you’re describing what I would like to do in my courses. Could you tell me how many students are in one of your larger lower-division courses?Report

Monte Johnson
Reply to  Tom Sparrow
1 month ago

Hi Tom. I ended up with 60 in my lower-division course this time. I could send you the schedule and exam samples if you’d like to see how I did it. Just send me a backchannel message.Report

Jeanne
1 month ago

I love the idea of oral exams (though I’ve never held them), and I’m so happy to read through these extremely helpful, thoughtful posts!

Would anyone mind sharing examples of questions that you posed during these exams? I’m trying to get a sense of what sort of questions would work well in this format (e.g., questions aimed at allowing students to demonstrate mastery of some particular argument in some paper they’ve read, or questions that try to get a student to develop an original argument, etc.). Would also love to hear more about grading these.Report

Monte Johnson
Reply to  Jeanne
1 month ago

Here are some examples from the final exam I’ve just been giving. Students are allowed to skip a question. And I sometimes have follow up questions or even discussion if that seems more productive in this particular case.

(1) Compare Callicles’ view of justice with Epicurus’ view of justice.

(2) Does Epicurus think that courage is something good and that others should be encouraged to pursue?

(3) How could Epicurus console someone suffering from chronic pain?

(4) Compare Epicurus’ and Epictetus’ views about god.

(4) Why does Epictetus say that one should not be angry with wrongdoers?

(5) Complete this sentence: According to Epictetus, it is not events that disturb people but __________________. (If successful: Can you give an example?)

(6) Epictetus thinks we should only focus on what is in our own power and our freedom. Why then does he consider being enslaved not a bad thing?

(7) Compare the literary structure of Plato’s Gorgias with Epictetus’ Discourses.Report

Gary Bartlett
Gary Bartlett
1 month ago

I’m definitely giving serious thought to using oral exams.

But I’d like to ask those who have used them how they (the oral exams) interact with essay assignments. Do you assign different topics for the essays versus the orals? Are they linked in some way, or are they just two independent assignments? How much weight do you give to each?

I’m assuming, of course, that those who use orals are still also requiring some essay assignments. I guess I might stand to be corrected on this. That would be interesting too. Has anyone abandoned essays entirely? Does anyone think they might do so? I’m worried that that would not be a good thing; there are surely important skills to be learned in writing essays that can’t be learned by taking oral exams, just as the reverse is also true.Report

Peter Finocchiaro
Peter Finocchiaro
Reply to  Gary Bartlett
1 month ago

In the courses where I administer some sort of oral assessment, I abandon essays entirely and minimize other writing-based exercises. It is a good thing. There are important skills to be learned in writing essays. But they are to be learned in other courses (some of which I teach) where those skills are more easily scaffolded. Trying to cram too many skills into a single course is a good way to fail to develop any of them.Report

Monte Johnson
Reply to  Peter Finocchiaro
1 month ago

I agree with this. In my larger lower-division class there is some writing on online discussion boards, but no essay writing, just two oral exams. In my smaller upper-division course, there is a research assignment involving 3 drafts of writing, plus an oral presentation at the end based on that writing (held during the final exam and in some cases during the last week of instruction as well). So both courses involve some written and some oral components, but the emphasis is on the oral performance in the lower-division class, and written performance in the upper-division one.Report

Carissa Phillips-Garrett
1 month ago

I regularly conduct oral exams in my upper-division classes, and I have (in the past) done it with lower-division students. Mine are longer (20-25 minutes) and I give them sample questions, which usually ask them to explain and evaluate critically some argument or compare how two of the people we read would respond to some particular question, etc. However, to combat nervousness, I let them pick where we begin the exam: e.g., they can pick one of the sample questions, select an author/topic, and/or bring their own argument to the exam. I tell them they will be evaluated based on their demonstration of their engagement with the material, so once they explain and evaluate, I follow up with challenges and see how they respond. I tell students that I will be led by their engagement: while I will ensure we cover a selection of topics/questions from across the semester, if they have a lot of good things to say, I won’t intervene as much. During the exam, I take notes and mark the rubric and then take another 5-10 minutes afterward to finish writing comments and determine the grades. (I give preliminary grades initially, and then compare after the first 4 or 5 to ensure consistency. I do not record the exams.)

The pluses: students often show me their engagement in doing philosophy beyond what I expect. They are (usually) fun, and students get pushed to do philosophy (not just learn about it.)

The drawbacks: when they go poorly, it is terrible for everyone. Since no exam is the same, there is always a worry that some exams are easier or harder than others. It is draining with a large class (I have had semesters where I was conducting about 20 of these, and that was tough, so now I do them in classes of 15 students or fewer, typically).Report

Logistician
1 month ago

I am interested in doing oral exams. However, I teach a 3-3 load with no TA assistance. My larger sections border on 50 students. Have people successfully conducted oral exams with this sort of schedule? I imagine the logistics become daunting.Report

Monte Johnson
Reply to  Logistician
1 month ago

I have just done it with a class of 60 students. You have to give over not only the final exam period (here = 3 hours) but also one week of instruction (here = 3 hours). Each student can then be given a 5-minute oral exam within the official scheduled meeting time of the class. If one does a midterm, it may require giving over 2 weeks of instruction if everything is to be kept within the official scheduled meeting time. If you can work out for students to come outside official meeting time– for example posted office hours, then you may be able to avoid that. Although this requires more time actually doing the examining, it greatly reduces the time required for grading, and for answering student complaints about the amount of partial credit given on written exams.Report

Jason Brennan
Jason Brennan
1 month ago

I used to do this when I worked at Brown, but only in small seminars. I told students they could sub an oral exam in lieu of a paper. We would agree ahead of time about what the topics would be about (whether it would be on some readings, or whether they would have to defend an original thesis, etc.) I told them we would start with easy questions and toward the end I would give them some curveballs.

IMO, it demonstrated mastery of the material and ability to think deeply far more than many of the papers did. However, it was very time consuming. (I did 50-minute oral exams.) I’m not sure if it would have worked well had the exams been shorter.Report

Kenny Easwaran
1 month ago

The biggest question I have (which was only mentioned in passing in the NYTimes piece) is about the amount of time it takes. I don’t exactly know how long it takes me to read and grade written papers, but my guess would be that oral exams would take longer.

It’s possible it wouldn’t though, since I’ve discovered that scheduling 15 minute meetings with students to give them feedback on their papers actually gets me done with grading faster than giving written feedback (though a big part of this is that the scheduled meetings forces me to read and think about the papers by a certain date, instead of allowing me to procrastinate).Report

F F
F F
1 month ago

I come from a country where, traditionally, all university exams are oral. I’ve always had oral exams as a student and then I’ve been giving oral exams as a faculty member. I loved them as a student. I hate them as a faculty member. It takes days. I am rested and bright in the morning, I am tired in the late afternoon. it’s difficult to keep the same level of difficulty, it’s hard to know one’s biases, it’s hard not to overcompensate. Thus, I switched to written exams only.Report

Zita Toth
1 month ago

I also used to give oral exams when in the US, and while some students were much more nervous about them than their written counterparts, they got used to it and I usually received positive feedback. I also had the sense that because of the nervousness, they prepared more.

I would usually give out a question list in advance, with about 20 questions, and then the students would get asked 2 random questions each (an icosahedron die can come in handy for such a purpose — they liked that because it seemed more randomized than asking from the top of my head). The questions were connected to the main topics of the class, and the grading was done pretty much as it would have been on an essay question, except in my head: there were things that the students were supposed to “hit on” (like the correct premises in an argument, or certain claims when explaining a theory, etc.). If they didn’t, I would ask them follow ups.

I found it was much more rewarding than grading essay questions or grading a test, and what I liked about it the most, it usually evolved into a real sort of discussion. In the case of weaker students it also seemed like the last opportunity to try to explain to them something, and sometimes it seemed that (perhaps due to the 1-on-1 nature of the situation) it was effective.
I would schedule 10 or occasionally 15 minutes per student, depending on class size (but 10 was usually enough).Report

anton
1 month ago

I was a professor, but in the humanities, had colleagues in philosophy, and took oral exams in philo undergrad subjects which were part of the core curriculum. Here’s what I experienced as a student:

Before final exams, you’re given ten statements referring to readings from class. You select a date and time for your defense.

During the oral exam, you pick out a piece of paper from a bowl containing a number referring to one of the statements. You’re given five minutes to explain the statement. The professor may raise follow-up questions in the event that you get stuck during your explanation or did not explain something clearly.

In order to study for the exam, for each statement you need to construct an ouline consisting of the main argument of the philosopher given the assigned text and his supporting points. Thus, it’s similar to a essay-type exam, except that you don’t select from statements; rather, one is selected randomly, which means you need to study all assigned texts carefully.Report

Louis Zapst
1 month ago

I have been giving oral exams in some form for years now. It is clearly a more accurate method of assessment than written exams or take-home papers. Originally, I did them one on one, in my office, and I recorded them. That was a wonderful way to engage selected students in further conversation and to suggest they consider majoring in philosophy. I recruited a number of majors that way. Then, when it seemed no longer appropriate (due to the increasing awareness of harassment as a problem and the risk that some students might feel uncomfortable being questioned in the close confines of their male prof’s private office), I reluctantly gave up the practice. (As an aside, I would now recommend doing these kind of oral exams in a public space – a library foyer, for example – with other people nearby.) I finally settled on having students come to the final exam period in the classroom and present a five minute “communication” to me and the other students, explaining the thesis and arguments of their term paper they had handed in the day before and answering my (and other students’) questions about their work. I like this approach a lot, as it makes good use of the final examination period, keeps students accountable for understanding what they’ve handed in, gives students a sense of intellectual community, and means I don’t have to take extra time to give oral exams individually.Report

Daniel Weltman
Reply to  Louis Zapst
1 month ago

Do you grade the students on their communications? And do you give them much feedback on the communications?Report

Francesco Orsi
27 days ago

Having studied in Italy, almost all of my philosophy exams were orals. At the time I thought them barbaric and prone to go wrong for the wrong reasons. But the standard format was (and still is, I’m afraid) you were required to read hundreds of pages, with little idea what the lecturer would ask you about. Often it was even stuff barely touched upon in lectures. Reading this thread, I see there are different and more sensible ways to go about it. A pre-given set of potential questions with impromptu followups seems like a good idea. Preparing students for the orals during seminars also makes sense. One thing nobody mentioned so far: having a TA or anyway PhD student sit with you during these sessions might help against biases, loss of concentration, and moody behavior towards students.Report