Law Review Citations and Law School Rankings
posted by Alfred Brophy
There’s no shortage of writing on law reviews or law school rankings, to say the least. So why not combine the two?
Questions about law review ranking abound. How does one compare offers from journals at relatively equal schools? Is it better to publish with a journal that is more frequently cited or with one at a higher ranked law school? Is it better to publish with a main law journal at a top 40ish law school or the secondary at a top 10 law school? Questions about law school rankings abound as well, particularly for schools outside of the top 30 or so. (Or so it seems to me.)
I’m partial to citation studies as a way of judging quality. I know that citations have lots of problems as a way of ranking journals (or individual authors). However, I like the objectivity citation studies provide. And so I’m partial to the Washington and Lee Law Library’s website, which provides comprehensive data on citations to hundreds of law journals by other journals and by courts. I’ve found it useful in trying to draw some comparisons between journals. Other people often draw comparisons between journals by looking to the US News ranking of the journal’s school.
But this leads to some further questions: what’s the relationship between citations to law journals and the reputation of the school that publishes it? I have a vested interest in this question, because I’m the faculty advisor to the Alabama Law Review. (I like to argue to my dean that we need lots of money to host symposia, which he gives us.) And as someone who wants to encourage good scholarship, I hope that good scholarship and good journals are rewarded–that a good journal will reflect well on the reputation of the school that publishes it.
So that led me to analyze the US News data for 2006 (which actually appeared in the spring of 2005) and the 2004 W&L Law Library citation data (which measures citations to works published from 1997 to 2004). There’s a high correlation (.86) between citations and peer assessment scores for the US News top 50 schools.
I also looked at Professor Brian Leiter’s reputation survey, which I think represents a significant improvement methodologically over the US News data for the schools that he surveys. Some interesting stuff here–there’s a high correlation between Leiter’s reputation scores and the US News peer assessment (.91) and journal citations (.83).
But then things begin to get a little more surprising. For schools in the US News 52-102 range, the correlation is not nearly so high (.57). The correlation becomes even weaker when we consider journals at schools in the third and fourth tier (.41). The correlations are significantly weaker at each level between peer assessment and citations by courts: .66 for US News top 50 schools; .12 for US News 52-102 schools; .25 for US News tier 3 and 4 schools.
My paper contains detailed tables reporting the data and speculates some on their meaning. I suspect that people at schools whose journals over-perform (like Fordham, Cardozo, University of Miami, University of Kansas, DePaul, Albany, Indiana–Indianapolis, University of Colorado, and Houston) are going to be quite pleased with the results. Faculty at other schools may have other explanations–like the frequency of citation doesn’t mean much. And on that they may be correct. A lot of really, really fine work is rarely cited. I know that to be the case in legal history, the area of scholarship I know best, and suspect it’s true for some other important areas of the legal scholarship. Part of the problem with citations to legal history is that relatively little scholarship is being written in that area, so there are comparatively few opportunities to cite work.
There are some serious limitations with citation studies, of course. But the data are worth considering. One implication that I suggest is that for third and fourth tier schools, the citation data may be a way of bringing some precision to the peer assessments of school quality. Perhaps US News should look to citation data to gauge something about the intellectual orientation of a school.
Because my time as a guest at concurringopinions is about to expire, I rushed a little to get a draft of the paper out before I turn into a pumpkin. There’s some more I plan to do with this (including using the recently posted 2005 citation data and looking more at variances between amount of citations at each tier of school). A special thanks to Brian Leiter, who’s doing a lot to bring some more rationality to the rankings world.
Related posts by some other folks:
Kaimi Wenger’s The Uneasy Case for the US News Law School Rankings
Dave Hoffman’s Ann Coulter on Law School Rankings
Betsy McKenzie’s Law Students as Consumers of Rankings
Brian Leiter’s April 2004 More Thoughts on the US News Law School Rankings
Brian Leiter’s March 2005 Updating the 2003-04 Law School Rankings
Brian Leiter’s April 2005 More on the US News Rankings Echo Chamber
Brian Leiter’s August 2005 NY Times Expose of How Law Schools Manipulate the US News Rankings