Explanation Option on Multiple Choice Exams
posted by Lawrence Cunningham
Law professors may struggle to determine optimal exam format, especially between essays and multiple-choice questions or a combination. Student appetite varies. But many students prefer essay exams. They may express concern that multiple choice questions limit ability to identify and explain ambiguities.
Teachers may find a multiple-choice format optimal for many reasons, including psychometric evaluation, the nature of substantive material to test (e.g., statutory versus common law), or simple time budgeting to grade exams (e.g., a professor teaching both Contracts and Corporations in a single term to large enrollments simply cannot grade 200+ written exams within deadline).
One way to offer multiple-choice exams while meeting that student concern is to give students a limited option to address perceived ambiguities. This can be done for a limited number of questions on a separate attachment to the multiple choice exam. I’ve done this for years, using an approach passed on to me by Bernie Black years ago when he was at Columbia and I taught a course there.
The mechanism is easiest to explain by excerpting below the related instruction that appears on the general instructions page to my exam; it is followed by the form of explanations page I attach to the exam booklet. I always circulate the instruction and sample form of explanations page in the weeks before the end of the term and explain the method in the beginning of the term when summarizing the course evaluation method.
From Instruction Page
6. While there is no obligation to do so, you may, if you think a question is ambiguous, explain why on the “explanations page” at the end of this exam. Then make whatever assumptions you need to resolve the ambiguity and answer the question. If there is a real ambiguity, your assumptions are reasonable, and your analysis is correct given your assumptions, you will receive credit even if your answer is not the one sought. But don’t look too hard for alternative explanations of what a question is asking for, or you may perceive ambiguity that isn’t really there. You can also lose credit if, though you selected the correct answer, your explanation convinces me that you did so for the wrong reasons. You can exercise this option to provide explanations for up to [_____] questions. [NB: I usually make this about 10-15% of the questions.]
From Explanations Form Pages
OPTIONAL: Explanation of Ambiguous Questions (Maximum of _______)
For each Question No. you believe is ambiguous, please write down:
(1) The answer you select (“Student Answer”)
(2) The answer you think I intended or may have intended (“Sought Answer”)
(3) The ambiguity and the assumptions you made in your selection
[Do not exceed space limits indicated by the lines below and please write neatly]
Question No. _______
Student Answer _______
Sought Answer _______
Explanation:
November 16, 2008 at 6:27 pm
Posted in: Education
Print This Post
Responses (10)
snitty - November 16, 2008 at 7:08 pm
Do you find that a large number of students pick the same questions to explain? Or did they when you first started?
In other words, do you find this helps you write better multiple choice questions?
Howard Wasserman - November 16, 2008 at 7:12 pm
I have used multiple-choice-with-explanation by making each question worth 3 points–1 point for getting the right answer and 0-2 points for the explanation. The trick is that a wrong answer with an explanation that shows some comprehension gets some credit; a right answer with an explanation that is totally off gets zero points (a bad explanation means the student guessed).
But I like what you do here and think there may be a way to utilize both.
Lawrence Cunningham - November 16, 2008 at 7:57 pm
Snitty,
Yes, yes and yes: The device can be useful to highlight problems with given questions, in a way that is often more helpful than statistical review of results alone. Of course, the best multiple choice questions are those that have been used many times and are improved with usage. But given changes in law, coverage, current events and emphasis from year to year, new questions or revisions of old questions seem inevitable. So this feedback continues to have potential value even for questions that are well-designed. And, yes, as one’s inventory of questions grows and experience with administration expands, fewer instances occur requiring such refinements.
suijuris - November 16, 2008 at 8:04 pm
Ah, was that why Corporations is smaller this spring than last year? Was sorry to get bumped from your section. Hope to rejoin next year.
I like the idea of the explanation section as you’ve described it. It seems like it would lower stress for students while not completely ruining the benefits to the professor of multiple choice (statistical analysis, etc).
Lawrence Cunningham - November 16, 2008 at 8:07 pm
Howard,
Your method is very interesting too. My impression is that yours is more of a hybrid between multiple-choice and essays that offers considerable value but both requires students to make the contributions and professors to evaluate them. The approach I describe is optional and narrowly limited.
Lawrence Cunningham - November 16, 2008 at 8:16 pm
suijuris,
Good inference.
TRE - November 17, 2008 at 1:13 am
One of my profs does the same thing but will consider voiding the question for everyone based on a response. They still maintain the threat of punishment. I’m curious why you would have the punishment aspect. Aren’t you better off guessing? Also is it fair to keep the question if it is ambiguous?
A.W. - November 17, 2008 at 9:03 am
i would say that multiple choice should have very little use in the law. the fact is my professor’s ability to weed out all the ambiguities are limited. Some “smartass” student can always point out most questions are not as clear cut as the teacher hoped. but that smartassiness (is that a word?) is a good trait for a lawyer to have.
Bluntly, in law school, a multiple choice exam is as much a test of one’s ability to take a test as that of the student’s knowledge. And we should be beyond that.
Now, as far as all of tha overwhelming you, then really the school needs to give you high-quality help. and if they don’t, then you have to pick a least worst option, which might be multiple choice.
recentgrad - November 17, 2008 at 2:52 pm
“the best multiple choice questions are those that have been used many times and are improved with usage”
Isn’t there some risk that savvy students will find copies of old exams, and thus have the questions in advance?
Lawrence Cunningham - November 17, 2008 at 5:40 pm
recentgrad,
MC exams must be restricted, meaning all booklets must be returned to get credit for the course.
Leave a Reply