March 25, 2008

posted by Daniel J. Solove

Create a ranking for law schools that wish to participate in the national arena. Then have separate regional rankings for schools that aren't striving to be as competitive in the national arena, but have a lot to offer prospective students that are planning on in a particular state or region.

oops... that should read "planning on working or practicing in a particular state or region."

There was an outstanding conference at Indiana on the rankings:
http://www.law.indiana.edu/front/special/2005_rankings_nextgen/

My sense is that Jeff Stake's article did a great job explaining how the rankings can be manipulated by extraneous accounting and other factors...so perhaps the best way to improve them is to do the opposite of the practices that Stake criticizes.

Use an aggregation formula that doesn't force the lowest score to be zero. It's been shown ( I can't remember the author's name, but the paper came out last year ) that this forced-to-zero-range creates all sorts of statistical artifacts / oversensitivity to data noise.

Use an aggregation formula that doesn't force the lowest score to be zero. It's been shown ( I can't remember the author's name, but the paper came out last year ) that this forced-to-zero-range creates all sorts of statistical artifacts / oversensitivity to data noise.

Bar passage rates and job placement rates.

End of story.

I'd also appreciate it if they rewarded schools that refrain from flooding the market with law grads who drive down the price of legal services, but that seems a tad overreaching.

Seth R: Bar passage rates vary wildly jdx to jdx, and job placement rates are very normative. This may work better outside the t14 where 100% employment is not taken for granted, but what job placement stats would you use to distinguish between Chi and Yale? All are employed at graduation. Chi grads may have a higher mean salary, but is that because of higher talent level or because of self-selection at yale into public interest and faculty positions. For disclosure's sake, I am a student at Stanford Law, who chose it more for the weather and the faculty members within my very narrow subspecialty than for any particular career or prestige concerns...

My point was that, from the student's perspective, after those two concerns - passing the bar and employment, everything else is really of secondary or minor importance.

Too many schools attempt to game the rankings by accepting a large body of transfer students as 2Ls (e.g., Georgetown); by having sizeable part-time student bodies that may not "count" in the same way as full-time student bodies (e.g., Fordham); by attracting a stronger student base than normal because of a geographic location to a higher-ranked school, but inevitably losing many of the best students to that transfer school (e.g., BU and BC to Harvard); or calculating or spending exorbitant amounts of money on mundane services (e.g., Illinois's Lexis "tab").

One statistic I might use, on account of these problems, is the LSAT and GPA medians of the 2L class as a separate calculation. While it might "punish" schools who accept students with low LSAT/GPA numbers but who perform admirably their first year and deserve a transfer, the effect would be negligible if the transfers were, in fact, a negligible proportion of the 2L class. Additionally, it would provide a more accurate portrait of who sticks around for a second year.

Ranking based on the number of books in the law libraries is a way to encourgage the sale of more law books which in turn makes law profs happy and more supportive of the rankings.

My company, Vault, released our law school rankings a few days ago. We focused on a single criterion: the "employability" of the schools' grads as seen from the firms' perspective. Results here: http://www.vault.com/lawschool/top25/

Our take is obviously subjective and narrow in its emphasis on private firms. There's no perfect survey or rankings scheme, so the more the merrier. Some reaction to our findings: http://www.abovethelaw.com/2008/03/because_the_world_needs_more_l.php#more and http://randazza.wordpress.com/2008/03/25/rankings-are-garbage-but/

Actually, the way you inflate volume counts in the library is not by buying more books, but by keeping huge runs of print reporters (in three or four copies), retaining superseded volumes and never weeding the collection. The thing is nobody but a few rich schools can afford the space to do that any more. Fortunately, volume count is around 0.75% of the total US News rankings.

I think you should rank law schools by their proximity to quality major league ball parks...alright!, Maryland is #1

I think the average level of debt at graduation should be matched up with the average pay level of graduates.

As a graduate of a law school that last I looked was ranked 5th among the three law schools within its state, I have an axe to grind with US News and their rankings. I think the rankings have value but their value has to be put into context by considering a couple of things.

1. It is pointless and stupid to attempt to rank law schools precisely 1 through 100 or whatever. Debating whether Harvard or Yale is really better than Chicago or Stanford is the equivalent of publishing a drunken debate at the Algonquin Club. The schools should be rated by tiers without ranking each school specifically within the tiers.

2. It is pointless and stupid to compare schools that serve different functions. Some schools like the aforementioned Harvard and Yale are national schools who aim to fill a niche of producing new wage slaves for the top firms, under qualified political appointees, judicial clerks, and new law professors. Other schools are there to provide actual practicing lawyers to their local communities. The two types of schools should not be compared or evaluated by the same criteria. For example, among the regional schools, bar passage rate is an extremely important factor. Since everyone at the national schools gets a job and many don't bother to take the bar because they go into alternative careers, bar passage rates are less important. Since salaries vary from region to region, starting salaries are not a very good measure of regional law schools. Since the graduates of the national schools compete for largely the same jobs at a few elite law firms, they are a very good measure for the national schools. The same goes with national reputation of the professors. It doesn't take a genius or a national reputation to teach basic civil procedure but it does take a good teacher. The two don't always match up in the same person. No one goes to place like Harvard because the of the teaching quality. They go there to say that they went there and be able to say they had Larry Tribe for Civil Procedure. It really doesn't matter whether or not Larry Tribe can actually teach. In the same way, to a studen at a regional law school, it really doesn't matter if his civil pro teacher has published many law review articles, it matters whether the professor can teach because the student at a regional law school is going to be expected to pass the bar and be able to know something and won't just have a job handed to him. Therefore, national reputation of the faculty should be an important factor in judging national law schools but not that important when judging regional ones. For regional law schools, overall student satisfaction with their education should be more important than reputation.

The rankings really ought to be put into several tiers. There should be a two tier rating system for "national law schools" in which the top ten recieve top tier status and the other twenty or thirty are divided into second and third tier status. Then the other law schools ought to be divided into regions, judged differently, and placed into three tiers, top, middle and lower among each region.

This system would provide real valuable information to prospective students. Not every student wants to go to a national law school or work for a big firm. Some students want to live in a particular area and practice law. This system would tell them for example, "if you want to get a job at a good firm in Portland or city X, you don't have to go to Harvard, you can go to this school which is the most respected regional school in the area with the highest bar passage rates." That would make a lot more sense than ranking schools that serve different functions 1 through 100.

As a graduate of a law school that last I looked was ranked 5th among the three law schools within its state, I have an axe to grind with US News and their rankings. I think the rankings have value but their value has to be put into context by considering a couple of things.

1. It is pointless and stupid to attempt to rank law schools precisely 1 through 100 or whatever. Debating whether Harvard or Yale is really better than Chicago or Stanford is the equivalent of publishing a drunken debate at the Algonquin Club. The schools should be rated by tiers without ranking each school specifically within the tiers.

2. It is pointless and stupid to compare schools that serve different functions. Some schools like the aforementioned Harvard and Yale are national schools who aim to fill a niche of producing new wage slaves for the top firms, under qualified political appointees, judicial clerks, and new law professors. Other schools are there to provide actual practicing lawyers to their local communities. The two types of schools should not be compared or evaluated by the same criteria. For example, among the regional schools, bar passage rate is an extremely important factor. Since everyone at the national schools gets a job and many don't bother to take the bar because they go into alternative careers, bar passage rates are less important. Since salaries vary from region to region, starting salaries are not a very good measure of regional law schools. Since the graduates of the national schools compete for largely the same jobs at a few elite law firms, they are a very good measure for the national schools. The same goes with national reputation of the professors. It doesn't take a genius or a national reputation to teach basic civil procedure but it does take a good teacher. The two don't always match up in the same person. No one goes to place like Harvard because the of the teaching quality. They go there to say that they went there and be able to say they had Larry Tribe for Civil Procedure. It really doesn't matter whether or not Larry Tribe can actually teach. In the same way, to a studen at a regional law school, it really doesn't matter if his civil pro teacher has published many law review articles, it matters whether the professor can teach because the student at a regional law school is going to be expected to pass the bar and be able to know something and won't just have a job handed to him. Therefore, national reputation of the faculty should be an important factor in judging national law schools but not that important when judging regional ones. For regional law schools, overall student satisfaction with their education should be more important than reputation.

The rankings really ought to be put into several tiers. There should be a two tier rating system for "national law schools" in which the top ten recieve top tier status and the other twenty or thirty are divided into second and third tier status. Then the other law schools ought to be divided into regions, judged differently, and placed into three tiers, top, middle and lower among each region.

This system would provide real valuable information to prospective students. Not every student wants to go to a national law school or work for a big firm. Some students want to live in a particular area and practice law. This system would tell them for example, "if you want to get a job at a good firm in Portland or city X, you don't have to go to Harvard, you can go to this school which is the most respected regional school in the area with the highest bar passage rates." That would make a lot more sense than ranking schools that serve different functions 1 through 100.

The result attempted here is futile. Law School rankings are ultimately subjective. Dramatic appeal ultimately outweighs any consideration of statistical accuracy.

As someone describes such processes, this US News Holy Writ appears to be a splendid example of sham science and spurious specificity. Ranking consists of r= a+b+c…+z. The problem with some of the components, say “p” for peer review, is that it is arbitrary, subjective, guess work. Survey-responding profs & administrators from Harvard, Yale, Columbia always have a warm, fuzzy feeling toward their Ivy League brethren & non Ivy league responders have the same warm, fuzzy feeling toward Ivy League schools. Also a school like Stanford wound up with two recent Supremes (Rehnquist & O’Connor) & were raised from the hoi barbaroi to the Sweet 16, & even sometimes to the Top Ten.

Interesting that one of the Blog commenters notes that he/she is

“a graduate of a law school that last I looked was ranked 5th among the three law schools within its state”.

5th outta three? Presumably a typo & I don't want to start a war over a typo.

Also, one of the Blog commenters takes a shot at Fordham Law School thusly

“Too many schools attempt to game the rankings …by having sizeable part-time student bodies that may not "count" in the same way as full-time student bodies (e.g., Fordham)….”

Not sure know how this works. Does it mean having a Four-year Night school?

Should've been that Stanford "was raised".

“a graduate of a law school that last I looked was ranked 5th among the three law schools within its state”.

That wasn't a typo is was a joke. The football coach John McKay while he was coaching the woeful expansion Tampa Bay Bucaneers that the experts had picked them to finish "sixth in a five team division". I was merely commenting on the low esteem US News holds my law school.

Solution: don't rank 'em. Take a cue from Hillary Clinton, who can find a variety of post-hoc arguments why she is winning the democratic primary, even when she is losing by the agreed upon measures. Let every school say they are the best and tell everyone why they believe that is true. The marketplace can then sort out whose arguments are the best, rather than some purported objective third party

By contrast -- look at baseball. Everyone wants to play for the winning teams because they want to win (and get paid more). Then, the next year, the better teams have even better players.

merevaudevillian said:

"by attracting a stronger student base than normal because of a geographic location to a higher-ranked school, but inevitably losing many of the best students to that transfer school (e.g., BU and BC to Harvard)"

I'm sure this affects some schools, but your example is faulty. HLS's transfer class is tiny, 2-5 students a year, and exists merely to get the class to the historical number of 556 should it be short due to dropouts or whatever (or at least this was true as recently as 2002 when I graduated). You basically had to be top of your class AND have a good reason for wanting to come there (to study with a specific prof or pursue a specific specialty, or be closer to family or spouse). I knew most of the transfers, those that I remember came from Michigan, Georgetown, Tulane, Boalt, Penn, and Yale (yep, and one of ours transfered to Yale). I don't remember any from BC or BU.

The reputation survey should be focused on hiring attorneys at firms of different sizes and in different areas of the country.

Outside of the tiny, tiny minority of people who want to go into academia, who cares what Dean X or Professor Y thinks about your law school?

What prospective students should care about is what Hiring Partner Z thinks.

The Vault has rankings based on surveys from partners who make hiring decisions. The survey ascertains "which schools prepare their students the best for law school." This is much more useful when doing your tangible or intrinsic net present value analysis of law school, because it gives you an idea of the implied economic value of your degree.

The real problem here is that U.S. News purports to be--and is the closest thing to--a comprehensive ranking system. In truth, the issue of law school quality is so multifaceted that its difficult if not impossible to do this fairly.

A more atomized set of surveys looking both quantitatively and qualitatively at different law schools would be more meaningful. Of course, there are many such ranking systems out there, but because they don't purport to be the all-encompassing gold standard in law school rankings (which I might be unfairly conjecturing that US News does deliberately), they don't share the ubiquity of the US News rankings.