Docketology
posted by Dave Hoffman
As I previously have discussed here and here, I’ve been working on a project examining when trial courts write opinions. With the help of statistician co-authors, I have investigated trial court dockets, trying to account for various factors that might lead a contested matter to either be explained through a traditional written opinion or issued in a brief order. Our resulting draft, “Docketology, District Courts, and Doctrine”, is now available from SSRN or from Selected Works. Here is an abstract:
Empirical legal scholars have traditionally modeled judicial opinion writing by assuming that judges act rationally, seeking to maximize their influence by writing opinions in politically important cases. Support for this hypothesis has reviewed published opinions, finding that civil rights and other “hot” topics are more to be discussed than other issues. This orthodoxy comforts consumers of legal opinions, because it suggests that opinions are largely representative of judicial work.The orthodoxy is substantively and methodologically flawed. This paper starts by assuming that judges are generally risk averse with respect to reversal, and that they provide opinions when they believe that their work will be reviewed by a higher court. Judges can control risk, and maximize leisure, by writing in cases that they believe will be appealed. We test these intuitions with a new methodology, which we call docketology. We have collected data from 1000 cases in 4 different jurisdictions. We recorded information about every judicial action over each case’s life.
Using a hierarchical linear model, our statistical analysis rejects the conventional orthodoxy: judges do not write opinions to curry favor with the public or with powerful audiences, nor do they write more when they are younger, seeking to advance their careers. Instead, judges write more opinions at procedural moments (like summary judgment) when appeal is likely and less opinions at procedural moments (like discovery) when it is not. Judges also write more in cases that are later appealed. This suggests that the dataset of opinions from the trial courts is significantly warped by procedure and risk aversion: we can not look at opinions to capture what the “Law” is.
These results have unsettling implications for the growing empirical literature that uses opinions to describe judicial behavior. It also challenges the meaning of doctrine, as we show that the vast majority of judicial work – almost 90% of substantive orders, and 97% of all judicial actions – are not fully reasoned, and are read only by the parties. Those rare orders that are explained by opinions are, at best, unrepresentative. At worst, they are true black sheep – representing moments and issues where the court is most obviously rejecting traditional patterns and analyses.
I am very interested in receiving comments on this paper, particularly before the late summer, when we plan to submit it to the law reviews!
[Nit-seekers beware: there is one typo in the SSRN abstract. (Don't go find it, just trust me, it is there.) For what it is worth, I basically agree with Kevin Heller that SSRN should give users more control over author-submitted papers to make revision easier. ]
April 23, 2007 at 11:01 am
Posted in: Empirical Analysis of Law
Print This Post
Responses (6)
James Grimmelmann - April 23, 2007 at 11:31 am
Yes, I come out of the woodwork whenever SSRN is mentioned. Editorial review is slow and expensive. Nonetheless, errors still get through. (I’ve had a paper leave my hands fine and end up on SSRN with a mistake on its download page.) A long feedback loop discourages authors from being closely attentive to details of their online papers’ presentation and makes it hard to fix errors as they crop up. Many schools also filter all of their SSRN uploads through an administrative assistant, making the author even less involved.
By way of comparison, changes to papers on the Berkeley Electronic Press Selected Works site go live within minutes.
Doug B. - April 23, 2007 at 4:55 pm
Dave: I love the topic and the term “Docketology,” and I also think your main themes are sound and VERY important. I wonder, though, about your data-set focus on (1) only federal cases, and (b) only civil cases. (You may explain these choices well in the paper, but my quick read did not lead me to these explanations.)
I doubt that your data selection choices profoundly impact your basic findings and conclusions. Moreover, my complaint may stem principally from hoping to see some examination of the criminal side of the docket in the arenas you are exploring. (Indeed, I think when and how federal district judges write sentencing opinions could/should be an incredibly rich area for empirical study.)
That’s my quick two cents along with general kudos.
dave - April 23, 2007 at 5:22 pm
Doug:
Thanks for these comments. In terms of “why federal” the answer is: data availability. The project required me (or, more precisely, RAs) to be able to read orders to learn motion types, results, and various other factors. Thus, we needed electronic docketing. State trial courts, as a general rule, don’t have e-docketing up and running out a broad scale, and those that do are not representative.
In terms of “why civil,” I think it is a matter of expertise. Even when I was a district court clerk, I had little exposure to criminal cases and criminal opinion writing. I think that to the extent law professors add value with empirical work, it is when they can offer some sense of what the world “should” look like, if the data were well-gathered. I don’t yet have that big-picture sense about trial court criminal practice. That doesn’t mean there aren’t projects yet to be done…
-Dave
Orin Kerr - April 23, 2007 at 6:49 pm
Dave,
I think you need to explain a bit more about your definition of “opinion writing.”
The paper states: “For our purposes, an ‘opinion’ is any judicial disposition on Westlaw or Lexis; an ‘order’ is any disposition that is not.” You then note that the E-Government Act of 2002 requires judges to make all opinions available on their websites. You state the conclusion that this means that these days everything that is really an opinion will end up on Westlaw or Lexis.
I’m not sure I’m convinced, at least based on your explanation in the paper. Can you break down the argument a bit more, or at least provide a bit more in the nature of evidence for it? I went to Westlaw’s ALLFEDS database and asked for every opinion in 2006 from a few well-known district court judges, and the numbers of opinions per jugde varied widely (suggesting that some comply and others don’t). Some Judges had around 150 opinions in the database that year; others had around 50; at least one had under 20. If you could justify your methodology a bit more, I think it would be helpful.
TJ Chiang - April 23, 2007 at 9:19 pm
Interesting hypothesis, but a statistical study might have cause-effect problems. Assuming that decisions that are more directly related to an appeal get more opinions, and the decisions that get opinions are more likely to be appealed, is this because: (1) judges are risk-averse and do not want to be reversed, therefore they opportunistically write opinions to get the first word; or (2) judges like to spend more time write opinions on interesting or difficult cases, that are also more likely to be appealed; or (3) a written opinion itself makes appeal more likely, because at least one person (the district judge) thought the issues sufficiently close to warrant an opinion?
David Cheifetz - April 27, 2007 at 5:19 am
Given that, in some quarters, law occupies a place in epistemology once occupied by religion (arguably even functions in a manner analogous to religion), the “Doxology” nit, unintended or not creates a world of irony the authors should consider leaving in place.
I’ve not yet finished reading the paper, so merely point out that it might be useful for the authors to look north of the US border at Canada where all judicial decisions are supposed to be adequately reasoned and judges are required to give some useful explanation for why he or she decided that the moon was made out of blue cheese, not green. Or, as occurred some years ago in Ontario, Canada, why “Martians” don’t have standing to sue for personal injury in that province’s courts. (That’s because, under Ontario’s rules of civil procedure, even
assuming that natural entities such as “Martians” exist and the plaintiff was a “Martian”, “Martians” don’t qualify as legal persons because they’re neither H. sapiens nor legal fictions such as corporations.)
That’s just my view; however, I’m a Canadian, so that may colour my reality.
Leave a Reply