the Law, the Universe, and Everything 

Search

Concurring Opinions is a
general-interest legal blog
operated by Concurring
Opinions LLC, a Pennsylvania
Limited Liability Corporation.

Yale University Press

ad-logo5.jpg

Our Podcast

Subscribe to Law Talk

Law-Rev-Forum-2.jpg

law-rev-contents2.jpg

Law-Prof-Blog-Census.jpg

Categories

Administrative Announcements
Administrative Law
Admiralty
Advertising
Agricultural Law
Anonymity
Antitrust
Architecture
Articles and Books
Bankruptcy
Behavioral Law and Economics
Bioethics
Blogging
Book Reviews
Capital Punishment
Civil Procedure
Civil Rights
Conferences
Constitutional Law
Consumer Protection Law
Contract Law & Beyond
Corporate Law
Criminal Law
Criminal Procedure
Culture
Current Events
Cyberlaw
DRM
Economic Analysis of Law
Education
Empirical Analysis of Law
Employment Law
Environmental Law
Family Law
Feminism and Gender
First Amendment
Food
Google & Search Engines
Health Law
History of Law
Humor
Immigration
Insurance Law
Intellectual Property
International & Comparative Law
Interviews
Jurisprudence
Law and Humanities
Law and Inequality
Law and Psychology
Law Practice
Law Professor Blogger Census
Law Rev (Boston College)
Law Rev (Boston University)
Law Rev (California)
Law Rev (Chicago)
Law Rev (Columbia)
Law Rev (Cornell)
Law Rev (Duke)
Law Rev (Emory)
Law Rev (Fordham)
Law Rev (Georgetown)
Law Rev (GW)
Law Rev (Harvard)
Law Rev (Illinois)
Law Rev (Indiana)
Law Rev (Michigan)
Law Rev (Minnesota)
Law Rev (Northwestern)
Law Rev (Notre Dame)
Law Rev (NYU)
Law Rev (Penn)
Law Rev (S Cal)
Law Rev (Stanford)
Law Rev (Texas)
Law Rev (UCLA)
Law Rev (Vanderbilt)
Law Rev (Virginia)
Law Rev (Wash U)
Law Rev (Yale)
Law Rev Contents
Law Rev Forum
Law School
Law School (Hiring & Laterals)
Law School (Law Reviews)
Law School (Rankings)
Law School (Scholarship)
Law School (Teaching)
Law Student Discussions
Law Talk
Legal Ethics
Legal Theory
Media Law
Movies & Television
Philosophy of Social Science
Politics
Privacy
Privacy (Consumer Privacy)
Privacy (Electronic Surveillance)
Privacy (Gossip & Shaming)
Privacy (ID Theft)
Privacy (Law Enforcement)
Privacy (Medical)
Privacy (National Security)
Property Law
Race
Religion
Reparations
Science Fiction
Securities
Social Network Websites
Sociology of Law
Supreme Court
Tax
Teaching
Technology
Tort Law
Web 2.0
Weird
Wiki
Wills, Trusts, and Estates

Recent Comments

Monica on Examining Law School Exams

Anjesh on Examining Law School Exams

Bruce on Examining Law School Exams

Amy on Examining Law School Exams

James Grimmelmann on Examining Law School Exams

Howard Wasserman on Examining Law School Exams

1L on Examining Law School Exams

Daniel J. Solove on Examining Law School Exams

MR on Examining Law School Exams

Daniel J. Solove on Examining Law School Exams

Archives

May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
August 2005
July 2005
June 2005

 

« Spending Time | Main | Empirical Studies at ALEA »

May 08, 2006

Examining Law School Exams

posted by Daniel J. Solove

pen6a.jpgThere are a lot of really good discussions going on in the blogosphere about law school exams recently.

Ann Althouse asks whether exams are a rewarding educational experience in and of themselves for students. Jonathan Adler offers his thoughts here. Rick Garnett chimes in at PrawfsBlawg.

In most law school courses, the grade is based on one final exam given at the end of the semester. Eugene Volokh offers a defense of this practice:

Naturally, some students might value the greater accuracy of the multi-exam approach, and the lower risk to them that this approach provides (since they don't need to worry as much about tanking a class because they screw up one exam). But for many students the downside of yet more studying and yet more worrying exceeds the upside of less risk and better measurement accuracy. And on top of that, grades are on balance mostly a zero-sum game — more accuracy of testing will just rearrange the As, Bs, and Cs, so as many students will lose as will win. Increased time needed to study for exams is a net loss for students as a whole. Therefore, you'd expect that more accurate measurement devices will please only a few students, while greater need to study will displease many more students.

So professors don't want to institute more exams, even if having more exams provides more accurate measurement. More students, I suspect, don't want to institute more exams, even if having more exams provides more accurate measurement. Who would predictably benefit from more accurate measurement, and lose nothing from it? Employers, and perhaps indirectly the legal system generally.

My take on exams is that the big problem with them is the format, not the one-exam-decides-it-all system. Exams are given under immense time pressure, thus favoring students who are able to analyze and write cogently in a very quick timeframe. This is a good skill, although it is significantly undervalued in practice, where law firms reward associates based on billable hours. Ironically, law firms therefore should select students who don't do well on law school exams, since it will take these students longer to complete assignments, and hence generate more billable hours. In the land of the billable hour, the inefficient lawyer is king.

Perhaps the most appropriate thing for law school exams to measure is a student's ability to produce a thoughtful legal memo under less onerous time constraints. For some classes, I experimented with take home exams where students had 24 hours to complete them. If 24 hours is enough time for Keifer Sutherland to save the world, it's enough time for a student to write a short memo. I figured that this would give students more opportunity to think and analyze rather than just write in a frenzy. However, there are some problems with take home exams. First, take home exams increase the possibility of cheating. Second, many students find them to be especially onerous. Although my take home exams were designed to be completed in about four to six hours of work time (spread throughout the day so students had some time away from the exam), students would often spend the full 24 hours toiling on the exam. Many weary-eyed students told me that they preferred a three-hours-and-you're-done exam rather than pulling an all-nighter on my 24 hour take home exam. Finally, I was surprised to find that the quality of the take home exams wasn't that much better than the in class exams. I can't explain why this was the case; it cut against everything I had theorized.

Are there better ways to measure student performance in a class? In thinking about this question, Volokh's point about efficiency is worth thinking seriously about. More accurate metrics of performance often have costs in time and stress, and sometimes the less perfect but more efficient metric is best. Part of the analysis depends on the answer to Ann Althouse's question. If exams are in and of themselves a rewarding educational experience (rather than just a metric of academic performance), then this is a factor that must be weighed in the balance. I remain uneasy about traditional in class exams, but I have not yet hit on an alternative that is significantly better. Take home exams might be a marginal improvement, and in the future I might experiment more with take home exams.

Posted by Daniel J. Solove at May 8, 2006 07:53 PM

Trackback Pings

TrackBack URL for this entry:
http://www.concurringopinions.com/movabletype/mt-tb.cgi/815.

Comments

Did you put a length limit on the 24-hour exams? As a student, I found 24-hour exams with strict (both in magnitude and in enforcement) word or page limits to be the most humane examination system, followed by scheduled closed-book exams, then by scheduled open-book exams. Take-home exams without length limits (or limits that weren't very constraining) were dead last. Length limits diverted more time into thinking and less into writing, put an upper bound on the number of topics worth exploring, and forced hard (but thought-provoking) choices about which of several points were worth making.

Posted by: James Grimmelmann at May 8, 2006 09:50 PM


Yes, I put a rather strict length limit on the exam -- I think it was something like 15 pages, double spaced, courier font. I've often found that longer exams aren't generally better, just wordier. My length restrictions were difficult, and they probably required students to do a bit of editing and trimming, which in my mind is a very good skill to learn.

I also established a penalty system for late exams. My policy: "If an exam is turned in late (beyond the 24 hours allotted), I will deduct a third of a grade. Additionally, I will continue deducting a third of a grade every 2 hours until the exam is turned in." Surprisingly, even under the penalty system, about 10% of the exams came in late. One even came in over 6 hours late!

Posted by: Daniel J. Solove at May 8, 2006 10:07 PM


I agree with most of this, but I must take issue with the notion that firms value billable hours to the exclusion of quick analytical thinking.

First, such thinking makes for better litigators, which means you can send associates to certain un-fun tasks with more confidence. Second, there is value in handling many matters at once, which can be done only with efficiency. Third, a lot of work is last minute, and if the work isn't getting done, billable hours are useless. Fourth, the modern trend is that clients want efficiency and are actually looking at bills these days.

I personally preferred the three-hour exam for the most part. Quick and over quickly. I tended to fill the time alotted working, so 24 hours = 24 hours of work.

Posted by: MR at May 8, 2006 10:46 PM


MR -- Everything you say I agree with -- analytical thinking and getting work done efficiently are some of the most valuable skills a lawyer can have. But with the structure of many law firms these days, billables seem to be such a large part of what is expected of associates.

There are some people who can work very intensely and well in short bursts of time. They are very valuable for litigation, but after a very productive burst of work, their energy is spent. Few people can work like this for 60+ hours per week. Their billables will look terrible, and they'll probably start getting pressure from the firm to up their billables.

Of course, firms may say that they value great efficiency, but what matters are the incentives law firms create. And at least from what I've seen and read, the incentives are skewed toward the billable hour.

Posted by: Daniel J. Solove at May 8, 2006 11:27 PM


I think what Mr. Volokh's comments suggest is that students are being done a favor by having one exam. As a 1L(so my experience is limited) I find this to be untrue. Doesn't this approach really just make it easier for the professors when it comes to grading. Having a more accurate measure should trump any supposed increased pressure on the students. Law School is all pressure, all the time. Whether you are studying for an exam or preparing for class the pressure still exists. What should be considered is producing the best lawyers and doing the best job to accurately determine who is the most knowledgeable.

Giving the wrong grade to the wrong person because a student had a bad day or the subjective nature of a pressure packed 4.5 hour exam to determine 14 weeks of work when a better alternative exists seems to be what professors should be striving for and explaining why they MUST test multiple times despite the additional pressures because it is the most accurate way to prove who is the correct student that is exceeding in the class.

Everyone knows how much weight grades can have in a person getting a job and determining the lives and futures of the students. How could relieving stress be a proper suggestion, when a person's future could be at stake(I understand I am being dramatic),by people who have been through the system, usually at the most prestigious schools in the country?

To suggest that it is a zero sum game and you will only be rearranging the As, Bs and Cs is like suggesting that we shouldn't have the 4th Amendment because it really only protects criminals.

I find this all a little convenient and as a student it seems that the real ease of burden is on Professors, many who are teaching from the same book and notes that they have been using for years.

I guess you can tell I just finished my spring exams today.

Posted by: 1L at May 9, 2006 12:28 AM


I am a big believer in what you do--24 hours, at-home, page-limits. Not sure what to do about cheating, of course. But I like the product that results and I like taking away the time crunch. I end up spending a lot of time during the semester repeating that they should not/cannot spend 24 hours on it, that it is not that daunting so don't freak out, etc.--sometimes students even believe me.

I also have been experimenting in smallish upper-level classes with a model proposed by one of my colleagues. We might call it a paper/exam combo--a semester-long project, written as a paper, but answering one of several specific questions that require students to engage in the deep structure and application of the doctrine.

Posted by: Howard Wasserman at May 9, 2006 07:22 AM


I misread "I will continue deducting a third of a grade every 2 hours until the exam is turned in," as saying, "I will continue deducting a third of the total grade every 2 hours until the exam is turned in," and wondered to myself, "Did you give the 6-hour-late exam a negative grade?"

Posted by: James Grimmelmann at May 9, 2006 08:39 AM


I can see the necessity of page limits on take-homes, but for those not naturally concise, they actually created more work - I would literally spend half the allotted time editing.

What about 8-hour take-homes - i.e., basically a normal-length exam with twice as much time to write?

Posted by: Amy at May 9, 2006 09:12 AM


Dan, I'm with MR -- I did not perceive any pressure when I was at a law firm to work more slowly. Quite the opposite. I think the reason for this is related to the reason, which puzzled me at first, why cab drivers in NYC who get paid by the mile drive so fast. The reason is that, the quicker they get you to your destination, the sooner they can pick up another fare. The number of fares per hour, not the distance they travel, is the primary limit on their earnings.

Same with law firms. The number of repeat clients per year, not the hours you bill to them, is typically the limiting factor on law firm profits. Clients usually want an answer (or a motion, or what have you) as fast and as cheaply as possible. An associate who does a task in 1 hour as opposed to 2 makes the client happy on both counts. Happy clients give you more business. Also that associate is available sooner to make other clients happy. And if the junior associates are good and efficient, you can delegate more work down to them, which increases the cap on difficult work that would otherwise be imposed by the limits on partner availability. Inefficient associates, on the other hand, mean you have to hire more of them to do the same number of tasks in the same amount of time, and that work is likely to be of lower quality. Hiring more associates is a huge expense. This is why associate salaries keep going up; because of benefits and overhead, it's cheaper to pay $20,000 more per year and ask for insane hours from each associate (often way more than 60 per week -- which I assure you is possible), even if attrition rates are high, than hire more associates at lower pay and have them work fewer hours.

Posted by: Bruce at May 9, 2006 10:23 AM


Hi,can i have a model of the test papers for november 2006 exams

Posted by: Anjesh at September 28, 2006 01:27 AM


I'm a student, aspiring prof, and three-hour law exams are threatening to ruin my career.

The purpose of a law exam should not be to assess who will the quickest litigator for a big firm. Many law students, amazingly enough, do not aspire to be litigators at big firms. At my law school, one does not need good grades to be a litigator at a big firm, but you need good grades to do things like teaching and impact litigation. Neither of those call on the "skills" 3-4-hour law exams do.

I am in favor of the 24-hr page-limited exam. It actually tests a skill necessary for lawyering: concision.

Posted by: Monica at February 19, 2007 03:02 AM


Post a comment




Remember Me?

(you may use HTML tags for style)

Authors

Daniel J. Solove

Website
Understanding Privacy

Kaimipono Wenger

Website
SSRN Page

Dave Hoffman

Website
SSRN Page

Nate Oman

Website
SSRN Page

Frank Pasquale

Website
SSRN Page

Deven Desai

Website
SSRN Page


Guests

William Birdthistle
Elaine Chiu
David Fontana
James Grimmelmann
Dan Kahan
Sam Kamin
Anita S. Krishnakumar
William McGeveran
Michael O'Shea






ad-logo3.jpg

blawg100_winner2.jpg

Previous Guests

Michael Abramowicz
Michelle Adams
Robert Ahdieh
Michelle Anderson
Laura Appleman
Francesca Bignami
Jeremy Blumenthal
Bruce Boyden
Donald Braman
Al Brophy
Bill Burke-White
Scott Burris
Anupam Chander
Miriam Cherry
Jack Chin
Jennifer Collins
Allison Danner
Brannon Denning
Deven Desai
Mike Dimino
Christine Haight Farley
Kim Ferzan
Dan Filler
Amanda Frost
Timothy Glynn
Rachel Godsil
Eric Goldman
Craig Green
Jeffrey Harrison
Erica Hashimoto
Laura Heymann
Christine Hurt
Heidi Kitrosser
Adam Kolber
Russell Korobkin
Anita S. Krishnakumar
Greg Lastowka
Joseph Liu
Solangel Maldonado
Jason Mazzone
William McGeveran
Salil Mehra
Carrie Menkel-Meadow
Scott Moss
Eric Muller
Jaya Ramji-Nogales
Elizabeth Nowicki
Paul Ohm
Michael O'Shea
Rafael Pardo
Marcy Peek
Eduardo Peñalver
Neil RIchards
Lori Ringhand
Alice Ristroph
Paul Secunda
Peter Smith
Charles Sullivan
Rick Swedloff
Steph Tai
Robert Tsai
Steve Vladeck
Sarah Waldeck
Melissa Waters
Alfred Yen
David Zaring
Timothy Zick
Jonathan Zittrain

Blogroll

Above the Law
ACS Blog
Althouse
Balkinization
Becker-Posner Blog
Beltway Blogroll
BlackProf
BoingBoing
Chicago Law Faculty Blog
Conglomerate
Convictions
CrimLaw
Crime & Federalism
CrimProf Blog
Crooked Timber
Discourse.net
Dorf on Law
Election Law
Emergent Chaos
Feminist Law Profs
43(B)log
Freakonomics Blog
Freedom to Tinker
Google Blogoscoped
How Appealing
Ideoblog
Info/Law
Instapundit.com
JD2B.com
Juris Novus
Jurisdynamics
Law and Letters
Legal Profession Blog
Legal Theory Blog
Legal Times Blog
Leiter Reports
Brian Leiter's Law School Reports
Lessig Blog
Madisonian
Mirror of Justice
National Security Advisors
Opinio Juris
Point of Law
Political Theory Daily Review
PrawfsBlawg
ProfessorBainbridge.com
Property Prof
Red Tape Chronicles
The Right Coast
Schneier on Security
SCOTUSBlog
Security Dilemmas
Sentencing Law and Policy
Simple Justice
Sivacracy.net
The Situationist
Susan Crawford
TalkLeft
Talking Points Memo
TaxProf Blog
Tech & Marketing Law
Truth on the Market
Volokh Conspiracy
WorkPlace Prof Blog
WSJ Law Blog
Wonkette
The Yin Blog

Pajamas Media BlogRoll Member