In Praise of Complexity
posted by Corey Yung
Earlier this month, right here on this very blog, Dave Hoffman pontificated about two of my favorite subjects: empirical legal studies and baseball. Primarily, Dave wondered about whether empirical legal research was facing might face the same problem as sabermatic baseball analysis: inaccessible complexity. I won’t rehash his argument because he did a very good job of explaining it in the original post. Although I completely agree with his conclusion that empirical legal studies should seek to be more accessible (which I always note at the end of my introduction of my empirical work), I disagree with his contention that empirical legal studies are facing might face widespread incomprehensibility due to growing complexity. Because I think it is a helpful analogy, I’ll borrow Dave’s example of advanced statistics in baseball.
My initial problem with Dave’s contention is that empirical legal studies have barely scratched the surface in some areas of research. In my own neck of the woods, law and courts, there has been an immense amount of study done primarily by political science and legal scholars. Yet, it wasn’t that long ago that scholars finally recognized that judges vote differently based upon who they are sitting with (panel effects). And despite strong empirical evidence that the same models don’t work outside of the Supreme Court, most scholarship has simply transplanted models of decisionmaking by the Justices to lower courts. My personal bugaboo outlined in my most recent article is that studies of lower courts, in particular federal Courts of Appeals, continue to treat judges as a monolithic group with no diversity except for the party of their appointing President. As an example of how dominant the belief of a homogeneity among judges is, consider this quote from a prominent scholar in a University of Chicago Law Review article:
“The jurisdictional competition model assumes that judges are homogeneous. Of course, they were not. Introducing heterogeneity would complicate the model without substantially altering the conclusions.”
This quote hardly does justice to the endemic nature of the belief that it is unnecessary to consider the significant personal variations that might exist among judges. It just happens that this particular author demonstrated his awareness of the issue by discussing the assumption whereas empirical legal scholars typically let the premise go unsaid in their judicial decisionmaking research. By not including possible variations in empirical legal studies of judicial behavior (beyond the appointing President’s party), the dominant law and courts research has yet to enter the realm of complexity comparable to advanced sabermatic research. Further, as law and courts is one of the most heavily researched areas using empirical methods, the simplistic assumptions of its models is indicative that other areas might be at even more basic levels of analysis.
I think the baseball analogy is particularly helpful in explaining my second objection to Dave’s reasoning. There are numerous concepts in sabermatic work that are beyond even sophisticated connoisseurs of such information. However, those advanced studies and ideas still find a way to inform statistical analysis in the mainstream (excluding Joe Morgan). I’m guessing that most people have no idea who Voros McCracken is, but he produced one of the most significant statistical findings in baseball history. McCracked discovered that, with few exceptions, a pitcher has little control over whether a ball in play is a hit or not. Instead, a pitcher can typically only control the three true outcomes: Walk, Strikeout, and Home Run. This insight supported by statistical analysis revolutionized the way we understand pitchers. Initially there was substantial resistance among baseball insiders. Had the management of the Diamondbacks, for example, listened to McCracken, there was no way they would have signed Russ Ortiz to the contract that every informed sabermatician knew was a disaster from day one. Eventually, McCracken’s ideas have infiltrated common understanding of educated baseball fans even though few probably know who McCracken is and even fewer could read and understand his study. As it turns out, even more complexity has been added to McCracken’s initial conclusions yielding even more insight into the relationship between pitching and hitting in baseball. More recent statistics such as WAR (Wins Above Replacement) and VORP (Value over Replacement Player) have started to be used by the mainstream press and baseball announcers. Yet, while people have come to understand the concepts measured by those stats, I doubt almost anyone could tell you how those stats are actually computed. Indeed, there is an ongoing fight among sabermaticians about which measurement of WAR is actually right (the difference is based upon which defensive metrics are used). For empirical legal studies to be important, not every study needs to be accessible. That isn’t to say that accessibility shouldn’t be a central goal. However, just like with sabermatics, the advanced studies ultimately can yield commonly held ideas.
Regardless of my quibbles with Dave’s post, I think this is an important discussion for scholars, empirical and otherwise, to have. And my agreements with his normative beliefs on the subject are more significant than the disagreements.
September 28, 2011 at 11:32 pm
Posted in: Courts, Empirical Analysis of Law
Print This Post
Responses (3)
Dave Hoffman - September 30, 2011 at 12:28 pm
“I disagree with his contention that empirical legal studies are facing widespread incomprehensibility due to growing complexity.”
Corey – I said no such thing. I “wondered” whether this was a problem, and then quoted a sabermetrics post that made that claim about baseball statistics as a warning.
Corey Rayburn Yung - September 30, 2011 at 12:47 pm
Hi Dave,
I’m sorry if I misrepresented your argument. I meant “facing” in the forward-looking-concern sense of the word, but my word choice should have been better (or I should have said “might face” instead). I’ll correct my post to properly reflect your contention with no ambiguity about it.
Corey
Jason W. - October 2, 2011 at 3:42 am
Apparently the very word “sabermetrics” is inaccessible to Corey. I don’t want to be snarky, but seriously: Google.
Also, as a less well-known FYI, there are myriad differences between the different calculations of WAR. The defensive metric can be a major area of disagreement between competing calculations, but it’s just one of several.
Leave a Reply