Rating Academic Reputation
There’s lot of talk recently about how to rate an academic’s reputation. As scholars, we’ve devoted extensive thought and discussion to the issue. Some ingenious techniques we’ve devised:
1. Count citations to a scholar’s work. Of course, for the reasons Brian Leiter documents, citation counts aren’t an indication that a particular article is any good. Jack Balkin and Sanford Levinson have a hilarious discussion of the foibles of citation counts in their article, How to Win Cites and Influence People, 71 Chi.-Kent L. Rev. 843 (1996). They write, for example:
If you want to understand how to write the sort of articles that will get cited incessantly, you first must understand why people cite articles. The most obvious reason to cite something is as authority for a legal proposition one is arguing for. But much legal scholarship is cited to represent an idea, a movement, or a memorable phrase associated with one of them. Thus, many citations to legal scholarship, and particularly to the most canonical pieces of legal scholarship, are citations to what the article symbolizes rather than acknowledgements of the truth of what the article says. The most-cited articles are less influences than icons; they are like colors of paint conveniently dabbed on the canvas because they are familiar and are easily accessible. . . .
Citations, we think, are often just such a form of public relations or impression management. They are a way of displaying information the citing author wants to convey about him-or herself, while concealing other information that would interfere with his or her desired “performance” as a legal scholar. From a public relations perspective, it may be quite irrelevant what you’ve actually read—or, much less what you’ve genuinely grappled with—, as long as you can successfully create the impression that you are the sort of person who is familiar with the cited work. Just as the insecure dinner-party host can walk into the wine shop and ask for “the wines most often bought by classy people,” the insecure legal academic—and is this not a redundancy?—can rarely go wrong by associating him-or herself, even if only through footnotes, with the articles published by classy people in classy law reviews. Citations are thus an essential part of the rhetoric of the law review article. They are not merely forms of evidence or proof; they also fall under the category of what rhetoricians call the “ethical appeal”—the demonstration, necessary in every persuasive speech, that the speaker is the sort of person who can be trusted and believed.
2. Count SSRN downloads. Gordon Smith of the Conglomerate has a post about how the SSRN system can be gamed. But gaming only matters if SSRN downloads tell you something meaningful about a scholar’s articles. The problem is that SSRN downloads aren’t any indication of article quality — they’re an indication of popularity among surfers of the Web. Post an article about Paris Hilton’s porn video and you’ll be one of the most downloaded scholars. [And by mentioning this, I'm ironically gaming our blog's Site Meter stats as I hope to get many Google hits for Paris Hilton's video.] Or write an article about computer hacking and hope to get a Slashdot link, which will bring thousands of downloads. Just because an article is downloaded a lot doesn’t mean that it is respected or that it is any good.
3. Look to the article’s law review placement. Another technique for determining whether an article is any good is to see what law review it is published in. But many crappy articles get placed well and many good articles get placed badly. Nate Oman’s astute post aptly suggests we quit blaming the law reviews and start looking at ourselves.
Perhaps we should try something else: We could actually read the articles and determine for ourselves if they are any good. This would, of course, mean actually reading the articles. But why would we want to do something that tedious when checking SSRN and citation counts is much more quick and fun?