Home | About | RSS Feed | Contact and Publicity Guidelines | Comment Policy the Law, the Universe, and Everything 


advertise-here4


Slip Opinions


Most under-appreciated thing about Warren Buffett: he built Berkshire to last well beyond him.  (LAC, at BRK annual meeting via Motley Fool, here.)

University governance as a new topic of public discussion.

An unusual profile of Mary Anne Franks (kw)

Aggressive copyright litigation run amok. (fp)

USA Today's Matt Krantz quoting me on Warren Buffett joining Twitter.  (LAC)

Private prisons? Why, sure! What could possibly go wrong? (kw)

TNR profiles Susan Crawford (kw)

Berkshire Hathaway is bigger than Warren Buffett.  Manual of Ideas (LAC).

Guns don't shoot people, kitchen appliances shoot people (kw)

Via Glom, Sat Eve Post review of The Essays of Warren Buffett.


Our Podcast

Subscribe to Law Talk


  • Posts by Author

  • Categories

  • Archives


  • Recent Comments


    • Shag from Brookline on National Referenda

    • PrometheeFeu on Tumblr, Porn, and Internet Intermediaries

    • Kyle on Contract Evolution

    • Bruce Boyden on Tumblr, Porn, and Internet Intermediaries

    • Orin Kerr on The Varying Use of Legal Scholarship by the U.S. Supreme Court across Issues

    • Guy Spier on Symposium Redux: Essays and Lessons

    • John Mihaljevic on Is Berkshire Hathaway Really a Psychology Experiment?

    • Sy Lorne on The Many Audiences of Buffett's Letters

    • Lawrence Cunningham on The Skeptical Principal

    • Lawrence Cunningham on Berkshire's Dividend Policy: Part II

    • Lawrence Cunningham on The Many Audiences of Buffett's Letters

    • Lawrence Cunningham on Deals without Bankers: Salomon and Benjamin Moore

    • Brett Bellmore on National Referenda

    • Gerard Magliocca on National Referenda

    • mls on National Referenda
  •  

    Site Meter

    About the Blog

    Concurring Opinions is a multiple authored, general interest legal blog.

    (Image: Wikicommons)

Law School Ranking: Measurement vs. Characterization

posted by Frank Pasquale

I have long been concerned about negative externalities from ranking systems. Perhaps people and institutions are always prone to try to distinguish themselves. If so, Brian Leiter’s expert consultation for the MacLean’s rankings of Canadian law schools may be a good thing, since, as he states, it resulted in “a ranking system that can not be gamed, that does not depend on self-reported data, and is not an indecipherable stew of a dozen different ingredients.”

However, Benjamin Alarie at Toronto has critiqued Leiter’s efforts. Here are a few issues:

One of the central problems with how Faculty quality is measured is that it doesn’t assess influence in publications aside from 33 Canadian law journals. As an initial matter, I think it is fair to say that academics seek to publish in places with the most active audiences for particular types of research—for example, the best journals to publish law and economics research in are likely to be American peer reviewed journals such as the Journal of Legal Studies, or the American Law and Economics Review, or even professional economics journals.

[I]t is unlikely that frequency of citation is a perfect proxy for quality; for example, overly provocative papers are sometimes cited for being so provocative.

It is unclear what the threshold used was for including the firms as among the “elite firms” used by Maclean’s.

[A national reach] measure . . . misses “international reach” of the law schools that regularly place students in the excluded top New York and Boston firms, in international NGOs, and in various other attractive positions.

[By the way, I put the links into those block quotes above.]

I think these are all valid points, but the problem is even larger. The consumers of these rankings are, by and large, students looking for a good education and firms looking for well-trained lawyers. Why so much focus on whether the law schools are feeders for “top” firms? Perhaps the best law teaching is that which manages to train people for diverse careers in law.

Finally, a more philosophical point.


I fear that a misguided quest for objectivity can lead us to present many judgments that ought to be characterizations as, instead, measurements or rankings. Leiter deserves great credit for not trying to mash an incommensurable array of statistics into a single figure. But I still think true accuracy in law school assessment might be better found in words, not numbers–in a long-form assessment of the school’s service to students, to the academy, and to its community. Perhaps such an evaluation would take a few hundred pages to cover all of Canada’s law schools. But if someone about to embark on a career in law is not willing to try to make sense of such a document, perhaps they should not be a lawyer in the first place.

Obviously such long-form documents would not achieve the concision of a ranking system. As someone who’s decried information overload, I see rankings’ appeal. But I also recall a favorite saying of Hilary Putnam on approaches to epistemology: “any theory that fits in a nutshell belongs there.” Perhaps the same can be said of rankings. There are some virtues to “knowing” less.


 September 13, 2007 at 9:04 pm   Posted in: Law School (Rankings)   Print This Post Print This Post

Responses (3)

  1. Michael Lee - September 13, 2007 at 11:25 pm

    Just as the law cannot command a useless act, so should we not undertake or acknowledge a useless act, i.e., to pay attention to wholly subjective rankings.

  2. Matt - September 14, 2007 at 9:43 am

    The sorts of criticisms noted in the quoted section can be taken in two ways- one reasonable and the other stupid. The reasonable way is as either noting things that might be improved on in the future or else as noting things that should lead smart consumers of the rankings to ask more questions or otherwise serve as caveats on using the rankings. I’m sure that Leiter has no objections to such remarks. The stupid way to mean them would be to believe that there could be a perfect ranking system, one that combines all desirable elements and has no undesirable ones. It’s hard to tell here which way the remarks are meant so I’ll assume it’s the good way. Many critics of rankings, however, seem pretty clearly to mean the stupid thing. (Leiter, for example, has given some good reasons for focusing on Canadian journals. They might not be fully persuasive in the end, but it wasn’t an arbitrary choice, and including American journals would have some clear draw-backs, too.)

    Finally, the idea that rankings of schools in general is bad seems silly to me. Unless one thinks there is no significant difference between the schools (an unlikely proposition) then rankings can be useful to students. They are not perfect but of course anyone who uses them in a stupid way probably ought not go to law school, even more so than someone who would not read a 300page narrative account.

  3. frank cross - November 29, 2007 at 2:44 pm

    I find lawyers’ reaction to numbers and rankings very odd. These evaluations are made all the time, which is the better law school, who’s the best scholar, etc. The rankings simply provide information on this question. They aren’t perfect but your subjective opinion on the issue sure as heck isn’t perfect either.

    There’s a strange fear of numbers here. There are various reasons for the fear of numbers, I think. One common, and distressing, reason is that they are less manipulable to reach the result the person wants, than is subjectivity. But the fear I see mostly is the thought that the numbers are somehow the “answer.” That they magically end the discussion. That’s wrong. They simply inform the discussion. May they be misused? Sure, but that is true of everything and hence is a criticism of everything (and hence, of nothing).

Leave a Reply

Spam protection by WP Captcha-Free


  • « Previous post
  • Next post »

Authors

Daniel J. Solove
Kaimipono Wenger
Dave Hoffman
Frank Pasquale
Deven Desai
Danielle Citron
Lawrence Cunningham
Sarah Waldeck
Jaya Ramji-Nogales
Solangel Maldonado
Gerard Magliocca

Guests

Kelli A. Alces
Taunya Lovell Banks
Ryan Calo
Claire Hill
Jay Kesten
William McGeveran
Meredith Render
Aaron Saiger
David L. Schwartz
Olivier Sylvain
Charles K. Whitehead
Aaron Zelinsky


















Previous Guests

Michael Abramowicz
Michelle Adams
Robert Ahdieh
Marvin Ammori
Michelle Anderson
Laura Appleman
Derek Bambauer
Taunya Lovell Banks
Ann Bartow
Steven Bellovin
Adam Benforado
Gaia Bernstein
Francesca Bignami
Josh Blackman
Joseph Blocher
Jeremy Blumenthal
Kathleen Boozang
Bruce Boyden
Donald Braman
Khiara Bridges
Al Brophy
Neil H. Buchanan
Bill Burke-White
Scott Burris
Paul Butler
Ryan Calo
Naomi Cahn
Anupam Chander
Miriam Cherry
Jack Chin
Glenn Cohen
Gabriella Coleman
Jennifer Collins
Caroline Mala Corbin
Thomas Crocker
andré douglas pond cummings
Allison Danner
Laura DeNardis
Brannon Denning
Deven Desai
Mike Dimino
Mark Edwards
Maxine Eichner
Jessica Erickson
David Fagundes
Lisa Fairfax
Joshua Fairfield
Christine Haight Farley
Kim Ferzan
Dan Filler
Mary Anne Franks
Susan Freiwald
Michael Froomkin
Amanda Frost
Brian Frye
Timothy Glynn
Rachel Godsil
Eric Goldman
Kyle Graham
David Gray
Craig Green
Tristin Green
Jonathan Hafetz
Vivian E. Hamilton
Meredith Harbach
Michelle Harner
Angela Harris
Jeffrey Harrison
Hosea Harvey
Erica Hashimoto
Jennifer Hendricks
Carissa Hessick
Laura Heymann
Robert Hillman
Gilbert A. Holmes
Nicole Huberfeld
Christine Hurt
Darian Ibrahim
Sherrilyn Ifill
John Ip
Shavar Jeffries
Kevin Johnson
Kristin Johnson
Jeff Jonas
Courtney Joslin
Dan Kahan
Jeffrey Kahn
Brian Kalt
Sam Kamin
Michael Kang
Chimène Keitner
Alicia Kelly
Orin Kerr
Nancy Kim
Heidi Kitrosser
Adam Kolber
Russell Korobkin
Alex Kreit
Anita S. Krishnakumar
Susan Kuo
Greg Lastowka
Sarah Lawsky
Youngjae Lee
Margaret Lewis
Erik Lillquist
Jeff Lipshaw
Jonathan Lipson
Jacqueline Lipton
Matthew Lister
Joseph Liu
Michael Madison
Tayyab Mahmud
Kevin Noble Maillard
Solangel Maldonado
Jason Mazzone
Linda McClain
William McGeveran
Salil Mehra
Carrie Menkel-Meadow
Max Minzner
Viva Moffat
Scott Moss
Eric Muller
Janai Nelson
Jaya Ramji-Nogales
Helen Norton
Elizabeth Nowicki
Paul Ohm
Angela Onwuachi-Willing
David Opderback
David Orentlicher
Michael O'Shea
Kristen Osenga
Mary-Rose Papandrea
Rafael Pardo
Marcy Peek
Eduardo Peñalver
Robert Percival
Michael J. Pitts
Marc Poirier
David Post
Amanda Pustilnik
Shruti Rana
Geoffrey Rapp
William Reynolds
Neil Richards
Lori Ringhand
Alice Ristroph
Marc Roark
Brishen Rogers
Sasha Romanosky
Tuan Samahon
Susan Scafidi
David Schleicher
David Schraub
Paul Secunda
Lea Shaver
Jonathan Siegel
Jessica Silbey
Peter Smith
Judd Sneirson
Adam Steinman
Charles Sullivan
Rick Swedloff
Peter Swire
Olivier Sylvain
Steph Tai
Andrew Taslitz
Robert Tsai
Jenia Turner
Joseph Turow
Steve Vladeck
Ari Waldman
Spencer Weber Waller
Howard Wasserman
Melissa Waters
Elizabeth A. Wilson
Frank Wu
Alfred Yen
Corey Yung
David Zaring
Timothy Zick
Michael Zimmer
Jonathan Zittrain

Ownership

Concurring Opinions is a
general-interest legal blog
operated by Concurring
Opinions LLC, a Pennsylvania
Limited Liability Corporation.

Blogroll

Above the Law
Access to Justice
ACS Blog
Althouse
Balkinization
Becker-Posner Blog
BlackProf
BoingBoing
Chicago Law Faculty Blog
Conglomerate
CrimLaw
Crime & Federalism
CrimProf Blog
Crooked Timber
Derechoalderecho
Discourse.net
Dorf on Law
Election Law
Emergent Chaos
The Faculty Lounge
Feminist Law Profs
43(B)log
Freakonomics Blog
Freedom to Tinker
Google Blogoscoped
How Appealing
Ideoblog
Info/Law
Instapundit.com
Juris Novus
Jurisdynamics
Just Books
Law and Humanities Blog
Law and Letters
Law Librarian Blog
Legal Profession Blog
Legal Theory Blog
Legal Times Blog
Leiter Reports
Brian Leiter's Law School Reports
Lessig Blog
Madisonian Theory
Media Law Blog
Mirror of Justice
The Moderate Voice
National Security Advisors
Opinio Juris
Point of Law
PrawfsBlawg
Privacy and Security Training
ProfessorBainbridge.com
Property Prof Blog
Red Tape Chronicles
The Right Coast
Schneier on Security
SCOTUSBlog
Security Dilemmas
Sentencing Law and Policy
Simple Justice
Sivacracy.net
The Situationist
Susan Crawford
TalkLeft
Talking Points Memo
TaxProf Blog
TeachPrivacy Blog
Tech & Marketing Law
Truth on the Market
Volokh Conspiracy
WorkPlace Prof Blog
WSJ Law Blog
Wonkette
The Yin Blog


© Concurring Opinions

Powered by WordPress