« Gamble on Gambling | Main | Replicability, Exam Grading, and Fairness »
February 12, 2007
Exam Review Culture
I had a really fantastic weekend in St. Louis, where I attended the Conducting Empirical Legal Scholarship: The Advanced Course workshop at Wash U. Or, as I heard folks describe it, "Stats Camp: This Time We're Not Screwing Around."
In any event, it got me in the mood to measure things. This morning, as I conducted an exam review for one of my students, I considered whether there was a relationship between the curve, exams reviews, and the production of scholarship.
The basic story would go like this. At some schools - including the one where I teach - there is a strong culture of encouraging students to come to professors' offices after receiving grades to review the exam and find ways to improve their performance. To my mind, this is a very good thing - not just for students, who can be taught to do better on an economically consequential activity - but for professors, who can figure out exactly how badly written exams confuse test-takers. Somewhere between half and two-thirds of my fall semester class came in to meet with me over the last two weeks (at a half-hour a meeting). But, looking back at where I went to law school, I can't remember ever going to talk to a professor about my exams, nor any of my friends doing so either. Casual inquiry among conference participants suggests that a culture of encouraging colleagues to undertake individualized exam review is more common at schools outside of the traditional top tier.
Why? It surely isn't because students at top-tier schools lack incentives to get to know professors. And, I doubt it is because professors at elite institutions don't care about teaching. Nor, in the end, is it because exam review isn’t helpful, or because grades don’t matter at schools without a culture of review.
Basically, I think that an exam review culture is a function of a mandatory curve. When professors give out Cs, Ds, and Fs, students receive a strong signal that their performance is subpar. (By contrast, receiving a B at Harvard, the equivalent to a C at Temple (which mandates a 2.85 mean) in terms of relevant class rank, is no signal at all.) As a result, they demand review. A faculty culture encouraging review is simply a reaction to consumer demand.
So there is a tradeoff, as always. A curve helps increase bar passage by signaling students about their class position. It results in more faculty time spent teaching. More time teaching is traded off against (a) leisure; and (b) other faculty work, most significantly, time spent writing. Thus, one way we might imagine grade inflation is as a subsidy for scholarship. Since scholarship is already well-subsidized by high class size and low teaching loads, this seems to be a weird allocation of school resources. (By contrast, a harder bar exam starts to look like a a tax on scholarship, albiet indirectly.)
More half-baked ideas from the AELSC to come.
Posted by Dave Hoffman at February 12, 2007 11:02 AM
Trackback Pings
TrackBack URL for this entry:
http://www.concurringopinions.com/movabletype/mt-tb.cgi/1728.
Comments
I didn't get it. How can a mandatory curve explain the variation in outcomes if there is no variation in the existence of a mandatory curve?
Posted by: Kate Litvak at February 12, 2007 02:17 PM
I'm confused by your comment, Kate (probably my fault in the original post.) Maybe this helps?
The difference is that there is no explicit mandatory curve at HLS and like schools i know about, and to the extent that there is one, the mandatory mean is set significantly higher than at other schools. As a result, students on left (bottom) side of the curve think "I may be on the bottom of the class, but I'm still getting a B, which isn't a bad grade. I must be doing ok." But contrast, where the curve's bottom is a C, the relevant student says "Oh @#$@#. I must need help."
Does that help, or have I muddied the waters more?
Posted by: Dave Hoffman at February 12, 2007 02:26 PM
I went to a top tier law school, and I remember in my first year being encouraged to do exam reviews with the professors--at least in my small section, with one particularly eager junior prof encouraging his large section to do the same. Thereafter, no prof ever brought it up, I guess the idea is that by your second year in an elite law school you should have gotten "the hang of things." Perhaps there's more hand holding first year?
I'm now in a top 10 law school in which I haven't heard any of this mentioned, despite the fact that 1Ls are allowed first-year electives. It seems so random without an established institutional pedagogical culture--perhaps some profs do suggest it, but it hardly seems de riguer here--I don't even know where the Academic Support Office is. Perhaps it's incumbent on profs who view their jobs as teaching in addition to writing (not a zero sum game!) to suggest it to their students. There's a strong disincentive in elite, competitive schools to seek help from professors or academic support programs (and for minorities, that "stigma" problem). So perhaps if professors put it on the table, students would take it up.
It's a catch 22; the responsibility of professors to teach outside the classroom vs. the responsibility of the students to take a more active role in their education. Still, given the rigid institutional culture at law schools, professors would probably be in a better position to do this.
Posted by: Belle Lettre at February 12, 2007 03:06 PM
Surely one difference is that at top schools there is much less of a down-side to being "average". Average students at Harvard several other top schools still get very high paying jobs most of the time, while the average student at Temple, right or wrong, does not. Students know this or can figure it out so many at top schools quickly don't feel the need to do more than 'average' since they will still get the big pay day either way. In that sense it might be perfectly rational for them to not spend time doing exam review.
Posted by: Matt at February 12, 2007 04:30 PM
Matt's suggestion is plausible, but I think doesn't exclude the possibility of the curve being a part of the story. The control would be to look at a top school with a low mean, and to see whether students will feel so sanguine about their job prospects. Even if employers are educated to look at rank, not grades, I bet that even top students in the bottom half of the class would realize that they are being out competed by their peers and seek help.
Belle's different experiences reinforce my sense that this is a cultural issue. The question is: why are some places institutionally bad at teaching, and others good (with noticeable exceptions by great or terrible professors.)
Posted by: Dave Hoffman at February 12, 2007 04:48 PM
Hello!
I am student in Croatia(Europe). I have one question. Do you have subject called legal culture?
Posted by: Ivan at June 29, 2007 07:36 PM
