Concurring Opinions » Law School (Rankings)

Saving the Environment

Gerard Magliocca — Sat, 12 Sep 2009 20:40:10 +0000

Law professors and deans often wonder whether their work can have a real impact on the world’s problems. As I was going through my mail on Friday, I came up with a great way for all of us in legal academia to help our environment and demonstrate our commitment to green policies.

Quit sending me brochures about your law school. I don’t vote in the U.S. News & World Report survey. Nor do I know anyone who does. I’m willing to stipulate that your school is the best in the nation. And I’m sure your faculty is comprised of the bravest, warmest, and most wonderful human beings in the world. All of the stuff that you send me does not Pass Go, does not collect $200, and goes straight in the trash. I know that the law school rankings create incentives for this sort of behavior, but please — enough already.

A Logic Puzzle

Gerard Magliocca — Fri, 24 Apr 2009 15:20:21 +0000

Several conclusions can be drawn from the following comparison. Which one do you take away?

1. In Grutter v. Bollinger, the Supreme Court described the admissions goals of the University of Michigan Law School (and law schools more generally) this way: “Our conclusion that the Law School has a compelling interest in a diverse student body is informed by our view that that attaining a diverse student body is at the heart of the Law School’s educational mission . . . [T]he Law School’s admissions policy promotes ‘cross-racial understanding,’ helps to break down racial stereotypes, and ‘enables [students] to better understand persons of different races.’ These benefits are ‘important and laudable,’ because ‘classroom discussion is livelier, more spirited, and simply more enlightening and interesting’ when the students have ‘the greatest possible variety of backgrounds.’ . . . Numerous studies show that student body diversity promotes learning outcomes, and ‘better prepares students for an increasingly diverse workforce and society, and better prepares them as professionals.’

2. . Consideration of diversity in the U.S. News and World Report Rankings: None

A Foucauldian View of Law School Rankings

Frank Pasquale — Fri, 13 Mar 2009 15:43:07 +0000

Sociologists Michael Sauder and Wendy Nelson Espeland (NE&S) recently published an insightful article on the disciplinary function of law school rankings. They apply both Foucauldian and organizational theory to “unpack the power and influence of rankings as a peculiar type of environmental pressure.” They conclude:

[that r]ankings simultaneously seduce and coerce, and . . . [the fact that] this complex interplay of co-optation and resistance is conducted in the bland language of numbers makes it all the more compelling. At schools with improving rankings, even critics may find it hard to avoid a flush of pride, along with relief and anxiety about next year. The allure of rankings may be subtle, but it shapes resistance while securing the engagement of critics and supporters alike.

NE&S document several responses to the culture of rankings. I found their description of a dialectical “gaming/surveillance” dynamic particularly interesting, given some recent research I’ve been doing on trade secret protection for ranking algorithms:

“Gaming” is one example of how resistance extends discipline by restructuring relations both among law schools and between law schools and the rankings. We define gaming as cynical efforts to manipulate the rankings data without addressing the underlying condition that is the target of measurement. [For example,] some schools encourage underqualified applicants to apply to boost their selectivity statistics, “skim” top students from other local schools to keep entering first-year cohorts small[, etc].

Such gaming strategies prompted USN to change its methodology and reporting, develop more explicit rules about how to measure rankings criteria, and monitor information more closely. The result, predictably, is a more precise and stringent discipline and more ingenious forms of gaming.

NE&S’s work also suggests a reason why there are so many dean searches presently. A law school dean is under great pressure to improve her or his school’s ranking, but “administrators’ ability to manage them is limited. Work that demands responsibility without control is especially stressful.”

NE&S also deserve commendation for their exhaustive empirical work:

Along with open-ended interviews of law school personnel (described below), we conducted 92 brief interviews with prospective law students, visited seven law schools, observed and participated in professional meetings and conferences, analyzed 15 years of admissions and yield statistics (Sauder and Lancaster 2006), monitored online bulletin boards for prospective law students weekly for an entire admissions cycle, and analyzed the content of Web sites, newspaper stories, and organizational documents (including strategic plans, marketing plans, promotional brochures, and internal memoranda). To identify distinctive effects on law schools, we interviewed 35 business and dental school administrators (Sauder and Espeland 2006) and reanalyzed evidence from two other research projects. . .

While law school rankings may seem insignificant in the grand scheme of things, I think they are well woth studying for their implications in many other realms of life. I used scholarship on the topic to critique the practices of search engines in my piece Rankings, Reductionism, and Responsibility. McKenzie Wark’s fascinating book Gamer Theory (published both by Harv. U.P. and online here) extrapolates the condition of video gaming to the world at large:

The whole of life appears as a vast accumulation of commodities and spectacles, of things wrapped in images and images sold as things. But how are these images and things organized, and what role do they call for anyone and everyone to adopt towards them? . . .

Everything has value only when ranked against another; everyone has value only when ranked against another. . . The real world appears as a video arcadia divided into many and varied games. Work is a rat race. Politics is a horse race. The economy is a casino. . . . Games are no longer a pastime, outside or alongside of life. They are now the very form of life, and death, and time, itself. . . . You are a gamer whether you like it or not, now that we all live in a gamespace that is everywhere and nowhere.

As network power accumulates behind certain ranking systems, platforms, languages, and methodologies, individuals are both “forced and free” to accept them. The collective freedom manifest in coordination and political action could perhaps enable us to develop a rankings system that better accommodated the diversity of law school aims and missions. But it’s a safer bet that we’ll succumb to drift, and the disciplinary powers now shaping law schools will eventually reach down to shape the careers of lawyers themselves. Doctors appear much better able to influence the rankings systems developing in their field than lawyers have been.

How is Tom Barr Like Shane Battier: Or, Measuring Individuals’ Roles in Group Success

Dave Hoffman — Mon, 02 Mar 2009 19:32:28 +0000

Michael Lewis recently published a Times Magazine story on NBA player Shane Battier. The article is largely an anecdotally driven portrait of Battier, a player who supposedly makes his teammates better and opposing players worse, while engrossing few individual gains. But the Houston Rockets, who employ Battier, recognize his value, because they’ve finally cracked the nut of regressing success in group sports. According to Lewis, the Rockets use a sophisticated plus-minus measure:

One well-known statistic the Rockets’ front office pays attention to is plus-minus, which simply measures what happens to the score when any given player is on the court. In its crude form, plus-minus is hardly perfect: a player who finds himself on the same team with the world’s four best basketball players, and who plays only when they do, will have a plus-minus that looks pretty good, even if it says little about his play. Morey says that he and his staff can adjust for these potential distortions — though he is coy about how they do it — and render plus-minus a useful measure of a player’s effect on a basketball game. A good player might be a plus 3 — that is, his team averages 3 points more per game than its opponent when he is on the floor. In his best season, the superstar point guard Steve Nash was a plus 14.5. At the time of the Lakers game, Battier was a plus 10, which put him in the company of Dwight Howard and Kevin Garnett, both perennial All-Stars. For his career he’s a plus 6. “Plus 6 is enormous,” Morey says. “It’s the difference between 41 wins and 60 wins.”

The problem with the article is that it offers no perspective at all on how the Rockets tweak the statistic to make it useful and a competitive advantage. In that sense, the piece could be thought of as Moneyball III: This Time With No Data and No Human Interest. (Moneyball Had Data; Blind Side had a compelling story; this piece is unripe on both fronts.)

Nevertheless, in some quarters Lewis’s work has again caught the attention of legal innovators. Jim Chen, who has already opined that Deans should use a version of plus-minus to evaluate faculty performance, suggests that Battier is a promising case study: “the single factor that makes a great team player is the mirror image of the single factor that turns even the most productive scholar into a toxic Arschloch: selfishness.” To which an astute commentator responded: “If anything, a stats-driven evaluation process will almost certainly lead to the Battiers of academia being under-rewarded, rather than the reverse. Wouldn’t it be enough to reward those who just seem to distinguish themselves by their selflessness? . . . Note that, even within the NBA — in which it is much easier to do a plus/minus assessment — Battier gets undervalued by most teams, and if he weren’t still riding a six year contract would probably get paid a lot less even by the Rockets.”

I’ve thought some about the problem of motivating good faculty performance. On the one hand, notwithstanding self-interested arguments to the contrary, merit-based pay systems are a good thing, as they punish real shirking. But as between faculty members who aren’t obviously checked out, merit evaluation is a bit tough. Chen suggests that we ought to reward selflessness, which sounds to me like a subsidy for the creation of public goods. But that suggests that we could identify selflessness in faculty conduct. Maybe the person who takes on administrative duties or summer teaching loads is selfless and needs a reward to compensate for giving up lateral opportunities. But maybe that faculty member hates writing or thinks s/he’s bad at it. It’s difficult to look behind actions to motives.

The problem with the Moneylaw approach to faculty rewards is that it has failed to fully define what universities are designed to maximize. That’s not an easy question to answer, obviously, and I don’t think there’s just one approach. For a few law schools, like Florida-Coastal, that answer is obvious: to make money. For others, law school’s function as a profit center within a larger university umbrella. (This is the cynical, but likely accurate, motive for the few recent start-up law schools within Universities.) But for most law schools, the ultimate criterion of faculty success is just unclear. Giving students a return on their investment is much of it, but it’s not the whole story, since tuition doesn’t pay nearly all the bills. We’ve responsibilities to alumni donors, to the State, to the Bar, etc. Shane Battier just needs to help his team win games. We don’t know what winning is. We don’t know what game we’re playing. And who’s our team again?

All this suggests that the quantitative measurement of individual contributions to group success is a field of study that (a) is underdeveloped; and (b) may never yield dividends for faculty performance. The problem is that we’re left with even more imperfect measures, like citations and SSRN downloads. What we’d really want is a way to understand individuals’ roles in creating a firm-wide culture of competence. As Gulati and Choi found when studying the 7th Circuit, a particular hardworking judge may inspire others, even life-tenured others, to better performance. I was reminded in reading that paper, and in this more recent plus-minus debate, of a famous Cravath partner named Tom Barr.

Barr, a former marine, led CSM’s defense of IBM in the antitrust case of the last century. In so doing, he set the culture of the Firm for at least a generation (and probably more.) He coined the term “beachmaster” for the young attorney who would manage day-to-day operations at trial. (Based on the analogous marine role). He also created (I think) the self-image of the modern CSM lawyer: hungry for more work, intense to the point of obsession, honest with the Court, and very well paid. If the business model is dying, maybe it’s because Barr isn’t around at most firms to create the kind of culture that justifies and coordinates the madness. I’m sure Barr was an tremendous biller, but if he wasn’t, it wouldn’t have mattered, since he created a larger culture that made money. And he wasn’t selfless: he was just good, and he inspired others to greatness.

(Image Source: The Green Bag, HLS Faculty in 1968).

Announcing the Moss Law School Rankings: Harvard #1, Yale #2!

Scott Moss — Sat, 13 Dec 2008 05:38:19 +0000

Congratulations to Harvard on ranking #1 in the newly minted Moss Law School Rankings! Below are the raw statistics, which I explain below:

#1: Harvard (7 points)

#2: Yale (4 points)

#3: Tulane (3 points)

#4: NYU (2 points)

#5: Georgetown (2 points)

#5: Cincinnati (2 points)

#5: Rutgers (2 points)

#5: Pepperdine (2 points)

#5: Louisiana State (2 points)

#10: Fordham (1 point)

#10: Washington & Lee (1 point)

My ranking is unorthodox, I admit, but all the great statistical innovations yield unintuitive outcomes, no? Let me explain my methodology.

Law schools accrue points by having alumni who were high public officials convicted, or simply forced to leave office, following criminal or otherwise serious unlawful misconduct they allegedly committed while in office in the 1990s or 2000s. [Footnote: I carefully say “alleged” so nobody on this list should sue Dan Solove, Concurring Opinions LLC, or (especially) me.] A law school gets four points for a President, two points for a Governor or Senator, and one point for a member of Congress or non-Gubernatorial high statewide official. Here is the list I compiled:

• Harvard: Gov. Elliot Spitzer (NY) (prostitution; possibly abuse of state police resources); Rep. William Jefferson (LA) (bribes); Sen. Ted Stevens (AK) (bribes); Sen. Brock Adams (WA) (sexual harassment)

• Tulane: Sen. David Vitter (LA) (prostitution); Rep. Robert Livingston (LA) (prostitution)

• Yale: Pres. Bill Clinton (perjury, and a number of other things that may or may not have been illegal)

• NYU: Sen. Bob Packwood (OR) (sexual harassment)

• Georgetown: Gov. Don Siegelman (AL) (bribery)

• Washington & Lee: Chief Judge Sol Wachtler (NY) (criminal threats related to extramarital affair)

• Fordham: Rep. Vito Fossella (NY) (DWI while visiting child from extramarital affair )

• U.Cincinnati: Gov. Robert Taft (OH) (illegal campaign contributions)

• Rutgers: Sen. Bob Toricelli (NJ) (illegal campaign contributions)

• Pepperdine: Gov. Rod Blagojevich (IL) (bribes; misuse of government funds to try to punish political opponents; etc.)

• Louisiana State: Gov. Edwin Edwards (bribes)

Kudos for such strong showings not only to Harvard but also to the Louisiana schools (alleged crimes by three notable public officials who attended law schools in the state), which reminds me that I’m surprised no New Jersey schools ranked higher. Condolences to Stanford, Columbia, and U.Chicago for lacking any presence on this list; I’m sure there are many other fine ranking systems that reflect some other strengths of your schools.

This is just a comparative ranking among law schools, not evidence that law schools produce public corruption. Here is a short list of non-lawyers who otherwise would qualify: Gov. John Rowland (CT); Rep. Randy “Duke” Cunningham; Rep. Rick Renzi; Rep. Dan Rostenkowski; Rep. Mark Foley; Rep. James Traficant; Sen. Larry Craig; Gov. Fife Symington (AZ).

A final note: by gerrymandering the criteria (a) to include only 1990-present illegality and (2) to cover only statewide or federal officeholders, I spared by former employer, Marquette University Law School, a high spot on the list based on Sen. Joe McCarthy (whose 1950s antics do not qualify) and recently convicted Providence Mayor Buddy Cianci (who never has held federal office despite sufficient popularity to get elected after a conviction for assaulting a man with a lit cigarette, an ashtray and a fireplace log). Were I to stress these two alums, the school slogan — “We are Marquette!” — might take on a somewhat different meaning than intended.

Do Mailings Lead to Better Rankings?

Susan Kuo — Tue, 21 Oct 2008 01:15:58 +0000

A haiku to celebrate the season:

Fall is in the air.

Law school mailings everywhere.

Rankings on the rise?

It’s fall, the season for tailgating, bonfires, and trick-or-treating. It’s also the time of the year when the U.S. News and World Report Magazine begins collecting data for the purpose of ranking U.S. law schools. When I check my work mailbox, I can almost always count on receiving postcards, pamphlets, magazines, and other mailings from law schools extolling the virtues of their programs and recent faculty hires. One of my colleagues – my most recently tenured colleague – even received law school swag.

Do these mailings actually positively impact the rankings of the schools that send them? Last year, the ranking methodology for law schools included a quality assessment based on two scores – a peer assessment score and an assessment score by lawyers and judges. Do these mailings lead to higher peer assessments and assessments by lawyers and judges?

A Voter Aptitude Test for U.S. News Law Rank Voters?

Geoffrey Rapp — Fri, 17 Oct 2008 20:00:52 +0000

Professors are ablog about the U.S. News ballots recently arrived in law school mailrooms around the country. At Moneylaw, rankings guru Tom Bell (Chapman Law) relates interesting news of possible voting irregularities in the academic reputation balloting — with individuals other than deans, associate deans, recruiting chairs and most-recently-tenured profs receiving ballots even though they aren’t supposed to be U.S. News voters. At Prawfs, Jason Solomon (Georgia) reminds voters they are supposed to be assessing schools’ quality, rather than reputation.

Solomon explains that the ballots task voters to “Identify the law schools you are familiar with, and then rate the academic quality of their J.D. program at each of these schools.”

One of the remarkable aspects of this charge is that voters are asked to identify the schools they are “familiar” with, and rank only those schools. One wonders how many ballots contain “un-ranked” schools. That is, do voters really decline to rank programs they don’t know much about? Sure, you know the schools you went to, that you fight with over recruits, and you might have a single conference or blogging pal on the faculties of any number of schools. You might vote high on schools you like or went to, low on those which sport unpleasant types, and of course “game” the system by giving your own school a “5″ and its rivals a “1.” But my guess is, for the vast majority of voters for the vast majority of the nation’s two-hundred law schools, any knowledge they have is at best a fleeting snapshot of a far more robust J.D. program.

U.S. News could easily purge the ignorance from this system by subjecting voters to a “familiarity” test. In order to have votes for particular schools count, for instance, voters could be required to name at least two (or three) active faculty members at the school, or provide some other mildly accurate indicator of quality or lack thereof (”sub-80% bar passage”; “Roger Williams top 50 outside of the top 50 ranked faculty”; etc.). If voters can’t come up with a name for a single faculty member or relevant indicator of quality, or be bothered to look one up on google, then they could select “Ignorant” with respect to a particular school.

To be sure, the fact that no one knows about a particular school might tell us something relevant. It might tell us that the school’s faculty could be more prominent, oriented out of the building, etc. Or it might tell us nothing — that the voter simply has never encountered any meaningful point of reference for a school that doesn’t get to the NCAA final four or the BCS bowls and is located 2500 miles away. U.S. News could do us a favor by reporting not just the mean ranking a school receives, but also the pecentage of voters who selected “Ignorant” for each of the “ranked” schools. If 80% of voters are “ignorant” about the quality of the University of West Dakota law school, that might be more noteworthy than that the school — like so many others — ended up with a mean peer assessment somewhere in the 2.0 – 2.5 range.

Then again, I’m still waiting for someone to put a law rankings ballot up for sale on ebay.

Should the US News Ranking Include Part-Time and Evening Law Students?

Daniel Solove — Tue, 01 Jul 2008 17:49:28 +0000

Via Brian Leiter, I learned that Rob Morse, the ranking czar of the US News law school rankings is considering including the LSAT and GPA stats for part-time and evening JD students in its calculations for law school rankings. Morse writes:

The first idea is that U.S. News should count both full-time and part-time entering student admission data for median LSAT scores and median undergraduate grade-point averages in calculating the school’s ranking. U.S. News’s current law school ranking methodology counts only full-time entering student data. Many people have told us that some law schools operate part-time J.D. programs for the purpose of enrolling students who have far lower LSAT and undergrad GPAs than the students admitted to the full-time program in order to boost their admission data reported to U.S. News and the ABA. In other words, many contend that these aren’t truly separate part-time programs but merely a vehicle to raise a law school’s LSAT and undergrad GPA for its U.S. News ranking. We have used only full-time program data because we believed that the part-time law programs were truly separate from the full-time ones. That no longer appears to be the case at many law schools. So, it can be argued that it is better analytically to compare the LSAT and undergrad GPAs of the entire entering class at all schools rather than just the full-time program data.

While much in the US News rankings should be changed, this change would wreak more havoc on legal education than it will solve. It is true that schools game the system with part-time and evening students, but any change should be focused on the gaming, not on including LSAT and GPAs of part-timers and evening students in with the law school’s regular LSAT/GPA stats. Leiter writes:

For many, probably most, part-time programs serve older, working students, who might not have time for fancy LSAT prep courses, but who bring levels of dedication, seriousness, and pertinent experience that enrich legal education and the legal profession. What a loss it will be if, out of fear of US News, schools start cutting back their part-time programs or rejecting these students whose numerical credentials might impede their crusade for a “higher ranking.”

I wholeheartedly agree. The result will be a dramatic curtailment of evening and part-time programs unless schools want to take a hit in their ranking. This will penalize schools with such programs, and I bet these programs will shrink rather dramatically if US News makes this change.

First, since many students in these programs are older and have been working many years following graduation from their colleges, their GPAs don’t matter as much. Such programs are a way to accommodate students who may not have excelled in their undergraduate studies but who have blossomed in the years afterward.

Second, these programs are also an small escape valve from the tyranny of the LSAT, which is often the end-all and be-all of law school admissions. While the LSAT is important and is correlated to successful performance at law school, it is also true that many students who didn’t do well on the LSAT also have success in law school and in their legal careers. Furthermore, statistically, several minority groups generally have lower LSAT scores than whites. The LSAT shouldn’t dominate so heavily in law school admissions, but it does (due in large part to the US News rankings). At least the evening and part-time programs could escape from this problem, but if US News makes the proposed change, there will be no escape.

Third, there are many factors that might indicate whether a student will be successful in law school. Work experience can be a very important factor, and it doesn’t count at all in the US News system, which measures student quality at law schools with LSAT and GPA scores. Evening and part-time programs are a way to admit students who have something different to offer, who excel based on other factors such as work experience, that US News doesn’t consider.

Perhaps there’s a way for US News to go after the abusers rather than merge the stats for evening/part-time and day. Otherwise, if US News makes the proposed change, it will threaten evening and part-time programs which are very important. These programs are not like the day programs. They are designed for students with a different profile. Changing them to make for a one-size-fits-all system will really hurt these programs — they will either start to look indistinguishable from day programs or they will shrink dramatically or cease to exist. Maybe some schools will stand tall and not change their programs, but they might pay a big penalty in their ranking to do so.

I hope that the US News doesn’t make this change. Law school education is ironically being shaped considerably by a magazine, and I hope that this magazine makes the responsible choice. There are plenty of other problems with the rankings that need to be fixed and that will not destroy an important dimension of legal education in the process.

Disclosure: I teach at a law school with an evening division.

This Month’s SSRN Rankings

Dave Hoffman — Mon, 12 May 2008 06:41:55 +0000

Following up on postings in February 2008 and May 2007, here’s this month’s SSRN download ranking, measured by total new downloads. (The numbers in parentheses are the rankings from February. Total new downloads for these fifty institutions: 914,252)

1 George Washington University – Law School (1)

2 Harvard University – Harvard Law School (2)

3 Columbia University – Columbia Law School (3)

4 University of Chicago – Law School (4)

5 Yale University – Law School (6)

6 University of Texas at Austin – School of Law (5)

7 University of California, Los Angeles – School of Law (7)

8 Georgetown University – Law Center (9)

9 Stanford Law School (8)

10 New York University – School of Law (11)

11 University of Illinois College of Law (10)

12 University of Pennsylvania Law School (12)

13 University of California, Berkeley – School of Law (13)

14 Vanderbilt University – School of Law (14)

15 University of Minnesota – Twin Cities – School of Law (16)

16 George Mason University – School of Law (18)

17 Duke University – School of Law (17)

18 University of Tennessee, Knoxville – College of Law (15)

19 University of San Diego – School of Law (19)

20 University of Michigan at Ann Arbor – Law School (20)

21 University of Southern California – Law School (21)

22 Northwestern University – School of Law (22)

23 Temple University – James E. Beasley School of Law (28)

24 Florida State University – College of Law (25)

25 Boston University – School of Law (27)

26 Fordham University – School of Law (24)

27 Yeshiva University – Benjamin N. Cardozo School of Law (26)

28 American University – Washington College of Law (31)

29 Loyola Law School – Los Angeles (23)

30 University of Virginia – School of Law (29)

31 Cornell University – School of Law (34)

32 Ohio State University – Michael E. Moritz College of Law (30)

33 Suffolk University Law School (32)

34 Emory University – School of Law (36)

35 University of Louisville – Louis D. Brandeis School of Law (37)

36 Brooklyn Law School (35)

37 Indiana University School of Law-Bloomington (33)

38 Chapman University – School of Law (38)

39 St. John’s University – School of Law (43)

40 University of Florida – Fredric G. Levin College of Law (47)

41 Case Western Reserve University – School of Law (41)

42 Notre Dame Law School (40)

43 Boston College – Law School (39)

44 Rutgers, The State University of New Jersey – School of Law-Camden (44)

45 University of Houston Law Center (Off-list)

46 Wayne State University Law School (Off-list)

47 Loyola University of Chicago – School of Law (Off-list)

48 University of Arizona – James E. Rogers College of Law (46)

49 Northern Kentucky University – Salmon P. Chase College of Law (Off-list)

50 Seton Hall University – School of Law (48)

The Contradictory Goals of Law School Rankings

Daniel Solove — Tue, 01 Apr 2008 19:06:41 +0000

As usual, a ton of blogospheric attention has been devoted to the US News law school rankings. Over at PrawfsBlawg, Geoffrey Rapp has found a way to get the numerical rankings of law schools in the Third and Fourth Tiers. At TaxProf, Paul Caron ranks the law schools by reputation score. At Brian Leiter’s Law School Reports, Brian Leiter offers suggestions for improving the rankings. At Law Librarian Blog, Joe Hodnicki tracks law school rankings from 1996-present. I, too, have posted about the US News Rankings.

If we step back from this year’s frenzy, I believe that there’s an important fact about law school rankings that accounts for much of the displeasure about them. Law school ranking systems have contradictory goals. Here’s why. Law schools, like many institutions, are not incredibly dynamic and changing in the short term. They often change slowly, not dramatically. The result: We shouldn’t see much movement year to year in the rankings. Most schools should stay about where they are. A few schools might move over time, but any one year’s movement is not significant in the grand scheme of things. So to be accurate, rankings shouldn’t change all that much.

But rankings systems have a contradictory goal: They need to reflect some kind of change, or else looking at the rankings each year would be like watching glaciers move. There must be some drama in the rankings year by year. We eagerly await our rankings each year, and we don’t want rankings at five or ten year intervals. And we don’t want stable rankings — we want changes to cheer and kvetch about.

There is another value in rankings reflecting some degree of change each year beyond our enjoyment of babbling on about them. Law schools work very hard on hiring new and lateral professors, promoting their reputations, improving their schools, increasing their admissions selectivity, and so on. We want our work to be reflected in a tangible manner. We want results for a year’s worth of hard work in improving the school. We don’t want to wait a decade or longer to see results. Unfortunately, the US News rankings often don’t reflect this work very well. But they do show that something is happening. We can then complain about the disconnect between what we’re doing and our ranking: “We did all this, and our ranking hasn’t moved. Damn that US News for their flawed system!” Or, we can justify rises in our rankings: “We’ve moved up several spots in the rankings. This is, of course, due to all the wonderful improvements we’ve been making to our school.” Either way, at least we have something to talk about.

The reality is that probably very little we do has much effect vis-a-vis our ranking with other schools over a period of time. We might improve our faculty by hiring some great laterals, but over the course of time, our competitor schools will also likely have done the same. True, one school might outpace another, but big shifts are the exception not the norm.

So the rankings need to reflect a state of affairs that is largely static, with a few gradual changes over the course of a long time. They must do so in a way that keeps people interested and excited. The rankings must display glacial change in a dramatic way. To use another metaphor, the rankings must make a turtle race seem exciting.

A few years ago, Dan Filler and I created a chart of the US News rankings for the top 25 law schools from 1997 to 2006. The interesting thing about the chart is how little movement most schools demonstrated over the course of time. Let’s look at Cornell Law School. In 1997, they were 12, then their ranking went like this over the next decade: 12, 10, 10, 12, 13, 10, 12, 11, 13, 12. When they drifted from 10 to 13 over the course of a few years, there were probably cries of outrage for dropping out of the top 10. When they suddenly jumped from 13 to 10, they probably celebrated with great cheers. Headline: “Cornell dramatically rises to the top 10!” In reality, Cornell is trapped in an orbit around 11.5 (that’s their average ranking over the past decade). And they barely go much higher or much lower than that. From year to year, it appears that there is something going on — Cornell appears to be moving. But it’s just a clever illusion, created by US News to achieve the two contradictory goals of rankings.

Paul Caron provides links to law schools responding to this year’s rankings. David Lat collects emails from law schools responding to rankings fluctuations.

At the end of the day, I believe in the following points:

1. For the average law school, US News ranking doesn’t change that dramatically. Only a few law schools make any major advances or drops in rankings.

2. In reality, schools don’t change that rapidly. Some schools that appear to have moved significantly in their US News ranking may have moved due to changes in methodology more than actual changes in the institution.

3. The legal world goes into a frenzy each year when the rankings come out, but changes in the rankings from one year to the next can’t possibly have any meaning. What matters is changes that occur over the course of a long period of time.

4. US News knows how to sell issues. Its rankings must change each year, or else nobody would care to buy the issue each year. It knows the two contradictory goals for rankings systems. It’s solution is a rankings system that shuffles things around a little bit each year, enough to give us the drama we crave. Although most schools go up and down each year, over the course of time, they basically stay in the same place.

Is is true that some schools move significantly over a period of time. So there are exceptions, but there aren’t very many.

To be more meaningful, rankings should probably be done in five-year intervals rather than one-year intervals. Information over the course of five years should be factored into the rankings, not just information for any one given year. Would such a ranking system be successful? Probably not. US News wants to sell issues each year, not every five years. Moreover, we don’t want to wait every five years for a new ranking. We want something exciting to talk about each spring.

And so the game will continue on. . . .

US News 2009

Dave Hoffman — Wed, 26 Mar 2008 06:50:37 +0000

They again seem to have leaked early at lawschooldiscussion.org: read ‘em and weep. Swayed by some of the arguments Brian Leiter makes here, I’m not going to reproduce the list. (And besides, it seems like the folks who excavated the information deserve the hits, not that the equities much matter or that others will feel the same way.). After satiating your curiosity, come back here and talk about ways to make the system better.

Improving the US News Rankings: A Wish List

Daniel Solove — Wed, 26 Mar 2008 03:11:46 +0000

A new article in the ABA Journal profiles Bob Morse, the US News & World Report “rankings czar.” I recently corresponded with Bob when he wrote to me about my parody of the rankings. He took my humor in good spirit. According to the ABA Journal article:

Since it began the rankings in 1987, the magazine is often attacked as wielding too much power; its methodology is denounced as easily manipulated and too subjective to carry such inordinate weight.

No one understands this more than Robert Morse, the man who created the law school rankings for U.S. News. As the magazine’s data research director, Morse says he, too, feels a high level of anxiety each year when the law school rankings are revealed. . . .

He also feels the heat from those who resent their enduring influence. For a ratings czar, he is a very reluctant despot. Far from being impervious to complaint, he maintains a blog where he explains his rankings and encourages constructive criticism. He’s been known to show up unannounced at gatherings likely to denounce him.

The article goes on to note that Bob Morse is open to suggestions for improving the rankings:

Morse says he understands and agrees that the rankings are not perfect, and he would like nothing more than to discuss with law school deans ways to improve them.

“Deans are welcome to call me or come by my office in Washington,” Morse says. “I want to work with them to improve the rankings.”

For better or worse, the US News rankings are here to stay. They are tremendously influential, and despite our constant complaints, I doubt that the influence of the rankings will diminish. So we can continue to gripe and grumble, with probably little effect. Or, we might be to work with the magazine to improve the rankings. Bob says he’s amenable to suggestions for improvement:

Bob Morse has his own blog that invites comments and criticisms. He’s shown up uninvited to university symposiums dedicated to fighting the U.S. News rankings he created. He wants to hear what the critics have to say.

So since Bob is listening, I pose the question: How ought the rankings to be improved?

The current US News methodology is here.

Here are a few things I’d recommend:

1. The reputation surveys are not sent out broadly enough. They go out only to deans and to newly-tenured professors. A broader cross-section of law school faculties should be used in the poll.

2. A more granular reputation scoring system should be used. The current 1-5 score isn’t granular enough. For starters, how do top schools like Yale and Harvard have average scores less than 5. Who’s giving them a score of 4? Seems fishy to me. Suppose a dean thinks Yale is the best and that Chicago is excellent — not quite as good as Yale, but very close. Yale therefore gets a 5. Does that mean Chicago gets a 4? That’s a big drop. Giving Chicago a 5 says it is equal, which may not be the dean’s view. There’s a problem here — the scale isn’t granular enough.

3. The number of library volumes shouldn’t be a part of the scoring system. This strikes me as a silly factor in ranking law schools.

These are just a few ideas. What are yours? The purpose of this thread is not to gripe about the rankings, but to propose fixes and improvements, so please focus your comments on suggestions for reforming the US News rankings.

The Official Leaked US News Law School Rankings, Plus Ranking Secrets Revealed!

Daniel Solove — Thu, 06 Mar 2008 07:54:03 +0000

I’ve got the scoop of the year! An anonymous source from US News & World Report leaked this memo to me. It is a memo written by the magazine’s “law school ranking executive” describing how the magazine arrived at this year’s official rankings.

See below for a sneak peak at this year’s rankings as well as some amazing secrets about how US News ranks law schools.

UPDATE: I was contacted by Bob Morse at US News & World Report. He reports that although he realizes it was a joke, some people have been emailing US News, thinking that this is really a leaked memo. News flash to the very gullible: The memo is fake. After all, could the real ranking process possibly be as rational as the memo?

A Market in Rankings?

Nate Oman — Mon, 03 Mar 2008 07:59:20 +0000

Complaining about law school rankings is a cottage industry in the legal academy. (Or rather more than a cottage industry, I suppose.) Everyone — or nearly everyone — dislikes the current system, and while I am less skeptical than most — it doesn’t seem unreasonable to me that students planning on shelling out $70,000+ in tuition might want some comparative measure of quality — I agree that the current system leaves something to be desired. It seems to me that we could set up a market based solution.

A student recently suggested to me that inTrade ought to set up a prediction market in U.S. News Rankings. That way students could hedge against the risk that the value of their degree may drop if their school shifts in the rankings. It is not a bad idea, but the problem is that such a market — while allowing a bit of U.S. News risk arbitrage and hedging — would ultimately be about simply predicting the mysteries of the U.S. News system. Suppose, however, that we set up contracts for something other than U.S. News status. For example, one might purchase a contract predicting that West Dakota Law School’s graduates would have an average starting salary of $100,000 or more. This would provide information of the kind that most students care about. Alternatively, one might create a contract that pays out if East Carolina Law School’s faculty places 10 articles in top-ten law reviews this year or some other measure of scholarly accomplishment. Then we could compare the share prices for Harvard and Yale. Of course, we would still just be getting a market in prediction of a particular outcome, rather than actual quality, but it might not be a bad proxy and it might capture more of the dispersed knowledge about law school quality. Of course, in order for the system to work you would need a relatively thick market in the contracts offered and even if there were only three or four contracts per law school, the number of contracts available in the market would be huge. On the other hand, law profs and law students are nothing if not obsessed with status, and I suspect that there would be a sizable contingent eager to cash in on their obsession.

What do you say? What contracts do you think inTrade should offer?

The Green Bag Asks: Your Law School (Really) Got Game?

Paul Secunda — Tue, 26 Feb 2008 17:13:35 +0000

This year, my last, at Ole Miss Law School, I was asked to Chair an ad hoc faculty committee on law school rankings. Like many law schools, ours has been flustered by the seemingly arbitrary way that our school has fluctuated in the U.S. News & World Report yearly rankings. And like others, we wanted not to care about such capricious things, but alas, others (including prospective students, current students, and alumni to name a few) did care. So as an institution we (myself and four faculty committee members) set out to study the factors one by one and try to determine where we could change policies, add money, etc., to constructively move factors that we had some control over.

What struck me during last semester as the committee met on a bi-weekly basis was that some schools that were perpetually labeled elite (by being in the First Tier) really did not have that many prolific or productive scholars. On the other hand, the opposite was also true: many a Third and Fourth Tier (though certainly not all) were bustling with faculty activity and innovation. So what was going on? Why wasn’t any current ranking system capturing these characteristics of the law school market?

Though I have not figured out the answer to this question, Inside Higher Ed reports today that The Green Bag Journal plans to put law school’s extravagant claims about having the best and greatest faculties in the universe to the test:

On their Web sites and in the other marketing materials that law schools distribute to raise their profiles — sometimes derided as “law porn” — virtually every law school boasts of having a faculty made up of stellar scholars, brilliant teachers and selfless public servants. “We continue to add depth to our already diverse and multifaceted faculty — excellent teachers whose high-quality research impacts leading academic and public policy issues,” reads the Web site of Northwestern University’s law school . . . .

But how are applicants — for admission and/or jobs — to know whether the schools are living up to their promises on faculty quality, that all-important indicator of the institutions’ overall quality? asks the Green Bag, which describes itself as “an entertaining journal of law.” . . . .

The Green Bag plans to step into that breach, the journal announces in an editorial in its forthcoming issue. Starting this spring, it will begin work on the “Deadwood Report,” which it envisions being an annual assessment of “whether faculty members do the work that the law schools say they do.” The journal acknowledges that the ranking will provide “rough and admittedly partial” measures of law school faculty quality, but posits that by being transparent (it will disclose the sources of its data and how it derives its numbers and rankings from those data), and by bringing more information into public view, “it will help law school applicants make better decisions about where to study or work…. We are trying to do some good here.” (The editors have an ulterior motive, too: compelling law schools to make public better information about their operations — more on that later.)

What exactly will the Deadwood Report measure? Law schools, the editors write, “generally hold themselves out as institutions led by faculties whose members are committed to teaching, scholarship, and service.” They argue that the best teachers tend to be active scholars and vice versa, “and all the best lawyers of every stripe engage in service for the public good…. Evidence of the law schools’ commitment to this view is reflected in the practically universal requirement of high achievement in all three areas for tenure. And so we should be able to say with some confidence that a good law school will have a faculty consisting of hard-working teacher-scholar-humanitarians,” the Green Bag editorial says.

Count me as very intrigued by this idea which animates the Deadwood project. I hope it sheds light on those schools who are coasting on old reputations made, on schools stuck in institutional inertia, and on those that are breaking new ground and deserve a second look.

And maybe even Green Bag will start a law school dean bobblehead series based on its Deadwood Report.

Ridiculously Unscientific Ranking: SSRN February Edition

Dave Hoffman — Mon, 04 Feb 2008 23:57:23 +0000

The February 2008 SSRN law school rankings based on downloads are out. I thought I’d be irresponsible and compare the data to the list compiled in May, 2007, when I last republished rankings based on new downloads.* The number in the parenthesis represents the change, the “big” number is the total new downloads. Snarky comments in brackets are mine.

1. George Washington University – Law School 80571 (+10) [They've got nothing to hide!]

2. Harvard University – Harvard Law School 53984 (-1)

3. Columbia University – Columbia Law School 37401 (-2)

4. University of Chicago – Law School 35080 (same)

5. University of Texas at Austin 30469 (-2) [A prediction market, anticipating Leiter's departure?]

6. Yale University – Law School 30317 (+1)

7. University of California, Los Angeles – School of Law 29491 (-2)

8. Stanford Law School 28123 (-2)

9. Georgetown University – Law Center 26502 (same)

10. University of Illinois – College of Law 25501 (+2)

11. New York University – School of Law 23416 (+2)

12. University of Pennsylvania Law School 23165 (+5)

13. University of California, Berkeley – School of Law (Boalt Hall) 22649 (+2)

14. Vanderbilt University – School of Law 19329 (same)

15. University of Tennessee, Knoxville – College of Law 18094 (+15) [Instapundit deploys!]

16. University of Minnesota – Twin Cities – School of Law 17226 (same)

17. Duke University – School of Law 14477 (+2)

18. George Mason University – School of Law 14206 (+4)

19. University of San Diego – School of Law 14096 (+2)

20. University of Michigan at Ann Arbor – Law School 13048 (+3)

21. University of Southern California – Law School 12993 (-1)

22. Northwestern University – School of Law 12811 (-4)

23. Loyola Law School – Los Angeles 12222 (+3)

24. Fordham University – School of Law 12132 (+8)

25. Florida State University – College of Law 11518 (-1)

26. Yeshiva University – Benjamin N. Cardozo School of Law 11159 (+1)

27. Boston University – School of Law 11050 (+2)

28. Temple University 10569 (+9) [Validates entire SSRN ranking project]

29. University of Virginia – School of Law 10212 (-1)

30. Ohio State University – Michael E. Moritz College of Law 10012 (-20) [@#$@]

31. American University – Washington College of Law 9902 (+17)

32. Suffolk University Law School 8711 (Offlist)

33. Indiana University School of Law-Bloomington 8521 (-1)

34. Cornell University – School of Law 8369 (-3)

35. Brooklyn Law School 8228 (Offlist)

36. Emory University – School of Law 8217 (-28)

37. University of Louisville – Louis D. Brandeis School of Law 7116 (Offlist)

38. Chapman University – School of Law 7092 (Offlist)

39. Boston College – Law School 6703 (-14)

40. Notre Dame Law School 6613 (-1)

41. Case Western Reserve University – School of Law 6579 (-6)

42. University of Colorado Law School 6182 (-4)

43. St. John’s University – School of Law 6030 (Offlist)

44. Rutgers, The State University of New Jersey – School of Law-Camden 5927 (-2)

45. Washington University, St. Louis – School of Law (-5)

46. University of Arizona – James E. Rogers College of Law 5630 (-1)

47. University of Florida – Fredric G. Levin College of Law 5595 (Offlist)

48. Seton Hall University – School of Law 5491 (+2)

49. University of Iowa – College of Law 5466 (Offlist)

50. New York Law School 5447 (Offlist)

*You can slice these data many ways, including per capita, total downloads, total papers. In my view, all methods are similarly (un)scientific.

Interdisciplinarity, Leiter and the Bluebook

Dave Hoffman — Mon, 03 Dec 2007 20:12:04 +0000

Gordon Smith has a nice summary post of the debate between Brian Leiter, Mary Dudziak and others on whether Brian’s faculty citation rankings accurately measure “impact in legal scholarship.”

The basic framework of the debate is

Objection: “But you didn’t measure X…”

Leiter: “True. Let a hundred flowers bloom, and do your own data collection!”

(Which strikes me as pretty persuasive.) I wanted to add a different ingredient into the pot. I think Leiter’s rankings mismeasure impact in interdisciplinary scholarship for a reason unrelated to his methodology or its merits. Simply put: the Bluebook itself undervalues interdisciplinary collaborations and thus scholarship.

I’m not nearly the first to observe that the Bluebook’s citation rules have an ideological component. See, e.g., Christine Hurt’s great piece on that very topic. But consider the interaction between Bluebook Rule 15.1, 16 and Leiter’s study. R.16 states that the citation of author names in signed law review articles should follow Rule 15.1. R. 15.1 states that when there are two or more authors, you have a choice:

Either use the first author’s name followed by “ET AL.” or list all of the authors’ names. Where saving space is desired, and in short form citations, the first method is suggested . . Include all authors’ names when doing so is particular relevant.

This seems to me to express a pretty strong non-listing preference. The “problem” is that much good interdisciplinary work results from collaborations among more than two authors – it is the nature of the beast. Take, for example, my colleague Jaya Ramji-Nogales’ forthcoming triple-authored article Refugee Roulette: Disparities in Asylum Adjudication, which was front-paged by the Times back in June. Two of the article’s authors are in danger of being ET AL.’ed in many law review footnotes, and consequently ignored in subsequent Leiter citation counts (unless the citing article’s author chooses to mention them by name in the text). This seems like a trivial objection, but it will take on increasing weight over the next ten years as empirical legal studies really comes online in the major law reviews. (Obviously, I’m writing in part because I’ve two articles in the pipeline where I’m a part of three-author teams, and the “et al.” problem is somewhat salient.)

Bluebook editors: I know you are lurking here! Can you fix this silly problem in the 19th edition?

Law Schools Ranked Lower Than Their Parent Universities

Daniel Solove — Mon, 19 Nov 2007 20:01:27 +0000

Last week I mentioned that Professor Paul Caron had compiled a chart of law schools ranked higher than their parent universities. Now, Paul has compiled a chart of law schools ranked lower than their parent universities.

More on Law School Rankings vs. Parent University Rankings

Daniel Solove — Wed, 14 Nov 2007 18:05:44 +0000

Earlier this week, I blogged about Paul Caron’s chart of law schools that were ranked more highly than their parent universities. Some commenters pointed out that because not all universities have law schools, there is a greater chance that universities might be ranked lower (because there are more of them).

Paul now has a new chart. He writes: “[S]ince U.S News ranks more national universities (262) than law schools (184), a more meaningful measure would look at the disparity by percentiles rather than in absolute rank.”

Law School Rankings vs. Parent University Rankings

Daniel Solove — Tue, 13 Nov 2007 18:42:37 +0000

Over at TaxProf, Professor Paul Caron has a chart of law schools that outrank their parent universities in the US News rankings. Some law schools far outrank their universities. I often wonder what effect the standing of the main university has on a law school. Paul’s chart demonstrates that law schools can thrive in the rankings even when their parents are not highly-ranked. Does this mean that law schools can establish a reputation that is by and large unaffected by the ranking of their parents? Or perhaps they would be ranked even higher but for their parents. Is their ranking differential due to greater reputation scores? Or do other US News factors account for the differential?