Comments on: Replicability, Exam Grading, and Fairness http://www.concurringopinions.com/archives/2007/02/replicability_e_1.html The Law, the Universe, and Everything Sun, 22 Nov 2009 17:51:16 -0700 http://wordpress.org/?v=2.8.3 hourly 1 By: Belle Lettre http://www.concurringopinions.com/archives/2007/02/replicability_e_1.html/comment-page-1#comment-55335 Belle Lettre Tue, 13 Feb 2007 02:10:53 +0000 http://www.solove.org/archives/2007/02/replicability-exam-grading-and-fairness.html#comment-55335 Speaking as both a graduate law student and an aspiring prof who has graded hundreds of exams as a TA, I say cynically that a bona fide attempt at fairness is requisite--but it's the apearance of fairness that for some reason matters. Jeff's grading system is standard, most professors in other disciplines (I speak of political science, sociology, English lit) request that you do so. And Dave, your grading system "appears" fair; that is you have created a set of metrics by which each student will be judged. Question is, do metrics ensure objectivity? The complaint about the law, social sciences and humanities from science-trained students is that we're so subjective in our grading. Do we get around subjectivity by creating "objective" indicia? Or is everything too subjective--the framing of the question, the ranking of of the issues to spot--to get around? I kind of like the subjectivity of the law, there are some obvious right things to spot but it's in the "massaging of the facts" and the application of the law to them as my Civ Pro prof said--that we find the answers. My co-blogger Jeff Harrison at Money-Law happens to think multiple choice questions are easy outs for faculty, and as a student I can say I hate them. They just bug me. I'm a bit disturbed by this retro kick to objectivise the law and try to create a metric for everything--aren't we supposed to be old school legal realists? (Leiter has declared ELS to not be the New Legal Realism, it's law + stats). But I do understand the end goal of fairness and measurability, I'm just wary of the methods. It's all about methods in ELS. But for more pointed grading strategies, in every class I've graded (six so far) we made a key (like you, Dave) and we took a few models answers of A, B, C, D range papers. If 2-3 are of consistent quality per range, they're good models as you go along for the rest. And I've found that when I type up comments students seem to think that lends validity and credibilty to my assessment. Speaking as both a graduate law student and an aspiring prof who has graded hundreds of exams as a TA, I say cynically that a bona fide attempt at fairness is requisite–but it’s the apearance of fairness that for some reason matters.

Jeff’s grading system is standard, most professors in other disciplines (I speak of political science, sociology, English lit) request that you do so. And Dave, your grading system “appears” fair; that is you have created a set of metrics by which each student will be judged. Question is, do metrics ensure objectivity? The complaint about the law, social sciences and humanities from science-trained students is that we’re so subjective in our grading. Do we get around subjectivity by creating “objective” indicia? Or is everything too subjective–the framing of the question, the ranking of of the issues to spot–to get around?

I kind of like the subjectivity of the law, there are some obvious right things to spot but it’s in the “massaging of the facts” and the application of the law to them as my Civ Pro prof said–that we find the answers. My co-blogger Jeff Harrison at Money-Law happens to think multiple choice questions are easy outs for faculty, and as a student I can say I hate them. They just bug me.

I’m a bit disturbed by this retro kick to objectivise the law and try to create a metric for everything–aren’t we supposed to be old school legal realists? (Leiter has declared ELS to not be the New Legal Realism, it’s law + stats). But I do understand the end goal of fairness and measurability, I’m just wary of the methods. It’s all about methods in ELS.

But for more pointed grading strategies, in every class I’ve graded (six so far) we made a key (like you, Dave) and we took a few models answers of A, B, C, D range papers. If 2-3 are of consistent quality per range, they’re good models as you go along for the rest. And I’ve found that when I type up comments students seem to think that lends validity and credibilty to my assessment.

]]> By: KipEsquire http://www.concurringopinions.com/archives/2007/02/replicability_e_1.html/comment-page-1#comment-55334 KipEsquire Mon, 12 Feb 2007 23:30:09 +0000 http://www.solove.org/archives/2007/02/replicability-exam-grading-and-fairness.html#comment-55334 In the spirit of "interdisciplinary studies," you might find <a href="http://longorshortcapital.com/adjusted-gpa-on-a-pro-forma-basis.htm" rel="nofollow">this post</a> on "grades and Wall Street" interesting. ;-) In the spirit of “interdisciplinary studies,” you might find this post on “grades and Wall Street” interesting. ;-)

]]> By: Jeff Lipshaw http://www.concurringopinions.com/archives/2007/02/replicability_e_1.html/comment-page-1#comment-55333 Jeff Lipshaw Mon, 12 Feb 2007 22:02:43 +0000 http://www.solove.org/archives/2007/02/replicability-exam-grading-and-fairness.html#comment-55333 Your method sounds a lot like mine. I don't think it's a good idea to look at the test as a whole. To the point about relative scores versus objective scores, I do one question at a time without looking at the student's scores on preceding questions, and try, if possible, to do all of a question at one sitting. And I reverse the stack of exams on each question. It gives the student a fair shake on each question without preconceived notions. And I find that it still shakes out into a curve - by and large, the people who get As have done well consistently across all the questions. Your method sounds a lot like mine. I don’t think it’s a good idea to look at the test as a whole. To the point about relative scores versus objective scores, I do one question at a time without looking at the student’s scores on preceding questions, and try, if possible, to do all of a question at one sitting. And I reverse the stack of exams on each question. It gives the student a fair shake on each question without preconceived notions. And I find that it still shakes out into a curve – by and large, the people who get As have done well consistently across all the questions.

]]>