Site Meter

Author: Corey Yung

2

Judging the Measures of Judges

In my last post, I discussed the measure I have proposed in my article for the judicial ideologies of federal appellate and district judges. That leads to the question: how do we know if my measure is “good?” Anyone can make up a bunch of numbers and formulas and declare their measure to be better than existing ones. How can it be said with any certainty that one measure of ideology is quantitatively or qualitatively better than another? That is one of the trickier questions in empirical legal studies of federal courts.

Consider the very interesting and valuable studyby Michael Heise and Gregory Sisk of judicial ideology in religious liberty cases published in the Northwestern University Law Review a few years ago. The study found, consistent with prior research, that ideology had a modest correlation with outcomes in religious liberty cases. How was ideology measured? Using Common Space Scores. What if the Common Space Scores were actually a poor measure of judicial ideology and votes in religious liberty cases were actually a “better” indicator of a judge’s ideology?  How would we know? Heise and Sisk chose to use Common Space Scores even while noting in detail the potential problems with the Scores. Of course, Heise and Sisk did so in part because they framed their study as part of a response to the firestorm created by an article by Lee Epstein and Gary King attacking empirical legal studies by legal academics. The types of inferences that can be drawn from the Heise and Sisk study would change dramatically if Common Space Scores were not strong indicators of judicial ideology. Read More

6

Judged by the Company You Keep

Last week, I tried to outline the difficulties associated with measuring judicial ideology in regards to the limited alternatives that have been offered by scholars. In this post, I hope to describe how I have measured it and attempted to overcome the various obstacles brought about by my methodology.

My idea for identifying the ideologies of federal appellate judges was to determine the rates at which such judges agree and disagree with “conservatives” and “liberals” on the bench. The assumption was that like-minded judges will vote together more often and judges with dissimilar ideologies will tend to disagree. By focusing on the agreements and disagreements among the judges, the goal was to pinpoint their respective ideologies (via “ideal points”). This is an agnostic method that necessarily faces all of the shortcomings of such an approach that I previously described.

The initial concern with such a method is that there are far too few disagreements among the judges on the Courts of Appeals. Indeed, in the 10,242 cases in my dataset, there were only 288 dissents (including partial dissents). Some judges who participated in over 100 cases were not on a panel in which there was a single dissenting vote. Looking at the Courts of Appeals alone was, thus, unlikely to offer much information. My solution was to treat the district judges being reviewed as pseudo-fourth members of the appellate panel. After all, the district judge reviewed the same legal issue as the appellate panel and rendered judgment on that very same issue. Notably, there are far more disagreements with district judges in the form of reversals. Also, by including the district judges, my methods also allowed data to be harvested from unanimous affirmances as well (as described below). Read More

1

How Can Judicial Ideology be Measured?

In my last post, I talked about the shortcomings with the leading measures of judicial ideologies. There are strong reasons, however, why those measures have dominated empirical legal research. If a scholar wants to assess the ideology of a judge, he or she is likely to try a technique that fits within one of these three categories: Case Outcome Coding, External Proxies, and Agnostic Coding. Each of these types of measures has advantages and disadvantages for federal judges not serving on the Supreme Court.

Case Outcome Coding – this category relies on a researcher going through a sample of cases and coding whether the judge’s or panel’s vote were “liberal” and “conservative” in ideological direction. While this technique can work reasonably well at the Supreme Court level (although there are shortcoming there as well), it is extremely difficult to apply to the Courts of Appeals or federal district courts. There is an enormous amount of labor required for sufficient samples to be accumulated for individual judges. Further, coding decisions are much more subjective than at the Supreme Court level as most of the federal docket is filled with cases that have little political salience. As Tonja Jacobi and Matthew Sag recently observed, “the last four decades of empirical scholarship have proceeded without a sophisticated objective measure of case outcomes.” Given the low level of disagreement among judges on the Courts of Appeals (due to consensus norms, “easy” cases, or other strategic incentives), there is an additional problem of making valid assessments without an enormous sample of data for each judge.  It is also possible that a researcher could try to code the ideological direction of methods instead of outcomes, but such a technique would tend to accentuate the difficulties described above. As a result, such measures have never been attempted on a comprehensive basis for individual judges on the federal appellate or district courts. Read More

5

“What is Judicial Ideology, and How Should We Measure it?”

That was the title of an excellent symposium piece by Josh Fischman and David Law last year that highlighted the limited scholarly work that has been done to effectively define and measure judicial ideology, particularly for judges not serving on the Supreme Court. Academics who are not engaged in empirical work relevant to the courts are often to surprised to find out just how crudely ideology is measured by law scholars and political scientists. There really are only two existing measures of the judges serving on the federal courts other than the Supreme Court: political party of the appointing President and Judicial Common Space Score. Every major study in empirical legal studies for decades examing members of the judiciary has relied upon one of those two metrics to determine the ideologies of federal judges (with most studies using the party of the appointing President).

Using the President’s party reduces ideology to a simple binary score – either a judge is “liberal” or “conservative.” Notably, using the President’s party, recent and current Justices Clarence Thomas, John Paul Stevens, Sandra Day O’Connor, Antonin Scalia, Samuel Alito, Harry Blackmun, and David Souter are ideological equals. In a nomination battles, the measure is essentially useless since whoever the President nominates is given the exact same score. Read More

5

“[Insert Judicial Nominee Here] is Out of the Political Mainstream”

I wanted to start by thanking the Concurring Opinions gang for having me as a guest this month.

The common attack of partisans in the recent judicial confirmation battles has been to brand the nominee as “out of the political mainstream.” Such accusations have been made against Justice Roberts, Justice Alito, Justice Sotomayor, and General Elena Kagan. However, the argument has also been deployed against lower court nominees as well. My favorite use of the device has been in the opposition to Ninth Circuit nominee Professor Goodwin Liu. In what typifies our modern political theater, Liu has been labeled as “out of the mainstream” in large part because he had the audacity to assert that Justice Alito was out of that very same “mainstream.” Of course, no one ever explains exactly what the “mainstream” is or what it takes to be “outside” of it. Read More

6

Mirror, Mirror on the Wall, Who is the Most Activist of Them All?

In my last post, I gave some results based upon collective groupings of appellate judges. In this post, I want to focus on the performance of individual judges. The primary reason that I am working to create a relatively large dataset is to allow for individual judge assessments. That has not been possible with the existing appellate court databases.

So, while I cannot yet tell you who the most activist judge was in 2008 because I have only reviewed data from five circuits, I can share my preliminary results for a few higher profile judges, including the most recent nominee to the United States Supreme Court. Here are the activism scores based upon my preliminary data for some of the highest profile judges in the Second, Third, Fourth, Seventh, and Eighth Circuits ranked from most activist to least activist:

notables2

Read More

5

Applying My Measure of Judicial Activism

In my previous post, I finally got around to explaining my measure of judicial activism. In this post, I will give some of the results based upon my preliminary data.

One of the most remarkable things I have found so far is that, although each of the circuits has a relatively consistent reversal rate, the activism scores vary by significant margins. The chart below shows the reversal rates and activism differentials for each of the five circuits that I have examined:

circuits

The activism scores are negative numbers with the higher numbers indicating a higher degree of activism. Thus, the 3rd Circuit is the most activist and the 4th Circuit is, by far, the least activist.

Of course, one thing that is always interesting to look at is which President’s appointees are the most activist. Here are the results I have so far:

presidents2

This chart should be taken with a very large grain of salt. Although in some senses, I already have a large amount of data, in others I have very limited sample sizes. When I start analyzing data that uses the judge as a unit of measure, I only have 52 judges with adequate sample sizes. So, there are few representatives for some presidents. That is also why I am not including any regression analysis in these posts. I include this chart simply to show one of variables that I will explore in regards to activism. Other examples of variables I will examine include background experience (district court, private practice, prosecutor, law professor, etc.), conditions at the time of the appointment (election year, unified government, etc.), and other assorted factors (law school attended, activism in particular areas of law, ABA ratings, etc.).

As this project is still ongoing, I welcome any comments and/or suggestions. In my next, and probably last, post, I will give some information about some high profile judges including Judge Sotomayor.

Update: Based upon popular demand, I have changed the second graph to a bar chart.

6

Measuring Judicial Activism by Federal Appellate Judges

In my last post, I briefly described the major approaches to studying activism. In this post, I will outline my measure for the concept as well as the contours of my dataset.

Since judicial activism at its core is about substituting judgment of other constitutional actors, for federal appellate courts it makes the most sense to study where they do most of their work: reviewing the judgments of district courts. That ensures large sample sizes, wide coverage of different areas of law, and decisions that are still restrained so some non-activist baseline can be derived.

To that end, standards of review offer a powerful tool to understand judicial decision making at the federal appellate level. There are deferential (i.e. clear error) and non-deferential (i.e. de novo) reviews of district court judgments. As you can see from the chart below derived from my dataset, standards of review do affect reversal rates.

standardsofreview

Read More

0

Studying Judicial Activism by Federal Appellate Judges

In my last post, I offered a new definition of judicial activism as applied to federal appellate judges. In this post, I will discuss some of the existing research on the subject.

The existing empirical scholarship is largely focused on two things: activism by the U.S. Supreme Court and judicial review of the other federal branches. While there is a lot of interesting work within those studies, I think the focus on Supreme Court review of other branches offers an extremely narrow view of judicial activism.

The Supreme Court is an unusual body in American courts. It is not subject to any higher review. The Justices are largely free to decide a case however they feel. As a result of the unrestrained nature of the Court, “activism” is a hard concept to measure. There is difficulty in finding a baseline against which to measure the Justices. A Justice who is an outlier on one Court might have regularly been in the majority during a different era.  The problem of measuring activism is compounded by the Court’s very small docket. Because there are so few cases, the sample sizes are not large. Further, because the docket is self-selected, the cases are not random and are concentrated in just a few areas of law.

Read More

5

Defining Judicial Activism by Federal Appellate Judges

First off, I wanted to thank Dan and the rest of the Concurring Opinions bloggers for having me back so soon. My last stint was cut a bit short by some unexpected family issues. As it turns out the subject I wanted to blog about then, judicial activism by federal appellate judges, is more timely now because of the Sotomayor nomination. She is already being attacked as an activist judge as a basis for rejecting her nomination (even if those attacks are just part of some inevitable game we play). For some time, I have been working on a project that attempts to provide an effective measure of judicial activism by federal appellate judges like Sotomayor. Over the next few posts, I will explain my measure and show some preliminary results. However, I need to do a bit of work defining the concept first.

Read More