Measuring Judicial Activism by Federal Appellate Judges
posted by Corey Yung
In my last post, I briefly described the major approaches to studying activism. In this post, I will outline my measure for the concept as well as the contours of my dataset.
Since judicial activism at its core is about substituting judgment of other constitutional actors, for federal appellate courts it makes the most sense to study where they do most of their work: reviewing the judgments of district courts. That ensures large sample sizes, wide coverage of different areas of law, and decisions that are still restrained so some non-activist baseline can be derived.
To that end, standards of review offer a powerful tool to understand judicial decision making at the federal appellate level. There are deferential (i.e. clear error) and non-deferential (i.e. de novo) reviews of district court judgments. As you can see from the chart below derived from my dataset, standards of review do affect reversal rates.
So, what can we learn from standards of review? I contend that by comparing a judge’s reversal rate in non-deferential cases with the reversal rate in deferential cases, we can effectively measure the concept of judicial activism. So, appellate judges are more “activist” when they reverse district court judgments under a deferential standard at a higher relative rate compared to reversals using a non-deferential standard. My exact measure (or activism “score”) for activism is reversal rate in non-deferential cases minus the reversal rate in deferential cases. This measure captures when a judge is not deferring to other constitutional actors when we would normally expect him or her to do so.
The measure has the advantage of not being based upon the substantive outcome of the case. A judge can use either a deferential or non-deferential standard and still find for either party. Since we might think that activist judges are not keen to make clear that their decisions are actually activist, looking at substantive outcomes can be tricky as judges try to mask an appearance of activism. Since standards of review are usually non-controversial (in that the parties rarely dispute over which standard applies) formal rules, we might think that there will be less ability for judges to mask their activism. Ultimately, the failure to defer by a judge over time indicates a relative propensity for activism even if we cannot say for certain that any individual decision is activist. As will be clear in my next post, judges vary quite a bit in their deference under standards of review.
My dataset thus far includes all 2008 cases that used standard of review related words (except habeas and immigration cases) for five circuits: the 2nd, 3rd, 4th, 7th, and 8th. Eventually the dataset will include all eleven numbered circuits as well as the D.C. Circuit. The dataset covers 3,873 cases and 11,583 judicial votes. Each vote is coded for, among other things, standard of review, type of vote, and type of case. I have also integrated biographical information for each judge to determine if background or demographic characteristics are related to judicial activism.
In my next post, I will detail some of my results based upon my preliminary data.
May 30, 2009 at 9:15 pm
Posted in: Empirical Analysis of Law, Legal Theory
Print This Post
Responses (6)
Aaron Titus - June 1, 2009 at 7:55 am
This project fascinating and refreshing. I look forward to seeing some of your preliminary findings.
Prolific Programmer - June 3, 2009 at 3:36 pm
Perhaps you would be kind enough to make the raw data available for those of us stat-heads who like to do our own analysis?
Matthew Sag - June 5, 2009 at 6:37 am
Hi Corey,
I have been working my way backwards through this very interesting serious of posts. Could you elaborate a little on how, if at all, the case mix differs between the cases under a deferential standard v. a non-deferential standard. What strategy did you adopt to deal with the different selection effects that might be underlying your results?
Cheers
Matt
Corey Yung - June 5, 2009 at 9:10 pm
Hi Matt,
I missed replying to this comment the first time around. The overwhelming majority of standards of review used are de novo and clear error. The other standards are used less frequently. I am coding for different areas of law although I am not making particularly fine distinctions. Criminal cases, for example, have lower reversal rates regardless of the standard of review used so I have tried to make sure the mix of criminal cases is at least reasonably close among judges.
So far, I haven’t done much to deal with selection effects for a few reasons. First, my sample sizes for each judge are large enough such that the case mix isn’t much different (at least for the areas I have coded). Second, so far my data hasn’t indicated that the activism differential varies substantially based upon area of law. Third, since I’m still gathering data, I haven’t done much to normalize small differences simply because I can’t be sure if the observed differences will continue in other circuits. Fourth, even though I have a lot of data in one sense, when I use judges as a unit of measure, I only have 52 individuals with adequate sample sizes. So, I haven’t been able to really test for statistical significance in a meaningful way to decide if I need to make some refinements.
At some point, I might have a circuit that has a very different reversal rate overall or a different case mix. In such cases, I have a couple of strategies in mind to deal with those situations. Probably the toughest case for me will be when I include the D.C. Circuit since that court has a very different docket (and I’m not even bothering to look at the Federal Circuit).
If you have any more ideas, let me know – I’d love to hear them. Thanks.
Corey
National Journal bloggers’ poll on Sotomayor - June 7, 2009 at 3:22 am
[...] I can’t vouch for the methodology, which is not one that would have occurred to me, but this analysis by Corey Yung of five federal [...]
Michael Jonathan Grinfeld - June 20, 2009 at 7:07 am
I’ve not been able to understand the merit of framing your research with a label like “judicial activism.” It seems to me that our democracy was based upon concepts of separation of powers and checks and balances, making the very underpinning of our political system one involving constructive conflict between the branches. Thus, it’s appropriate to have an “activist” executive, an “activist” legislature, and an “activist” judiciary, with each branch contending during a process that ultimately results in societal consensus.
That consensus, throughout our history, has shifted and evolved as our nation has grown more enlightened and our understanding of liberty has broadened. For instance, today every adult can vote, no person can own another, and our children are in integrated schools rather than sweatshops. These changes are the direct result of the appropriate exercise of each branch’s authority, working not only in harmony but in competition as well.
It seems you’re characterizing reversal rates as something more than they actually represent. They’re not a reflection of “activism” at all, but merely a cataloguing of outcomes during the exercise of judicial authority as it’s evolved over the centuries. The trouble with the “activism” framing is that it makes the judicial process seem sinister and negative, and as an overstepping of boundaries, when in fact, it’s not. The reason we have multiple layers of judicial scrutiny is because our democracy works hard to be just. The unfortunate label “judicial activism” should be rejected rather than “researched.”
Leave a Reply