My [...]]]> Earlier this month, right here on this very blog, Dave Hoffman pontificated about two of my favorite subjects: empirical legal studies and baseball. Primarily, Dave wondered about whether empirical legal research was facing might face the same problem as sabermatic baseball analysis: inaccessible complexity. I won’t rehash his argument because he did a very good job of explaining it in the original post. Although I completely agree with his conclusion that empirical legal studies should seek to be more accessible (which I always note at the end of my introduction of my empirical work), I disagree with his contention that empirical legal studies are facing might face widespread incomprehensibility due to growing complexity. Because I think it is a helpful analogy, I’ll borrow Dave’s example of advanced statistics in baseball.

My initial problem with Dave’s contention is that empirical legal studies have barely scratched the surface in some areas of research. In my own neck of the woods, law and courts, there has been an immense amount of study done primarily by political science and legal scholars. Yet, it wasn’t that long ago that scholars finally recognized that judges vote differently based upon who they are sitting with (panel effects). And despite strong empirical evidence that the same models don’t work outside of the Supreme Court, most scholarship has simply transplanted models of decisionmaking by the Justices to lower courts. My personal bugaboo outlined in my most recent article is that studies of lower courts, in particular federal Courts of Appeals, continue to treat judges as a monolithic group with no diversity except for the party of their appointing President. As an example of how dominant the belief of a homogeneity among judges is, consider this quote from a prominent scholar in a University of Chicago Law Review article:

“The jurisdictional competition model assumes that judges are homogeneous. Of course, they were not. Introducing heterogeneity would complicate the model without substantially altering the conclusions.”

This quote hardly does justice to the endemic nature of the belief that it is unnecessary to consider the significant personal variations that might exist among judges. It just happens that this particular author demonstrated his awareness of the issue by discussing the assumption whereas empirical legal scholars typically let the premise go unsaid in their judicial decisionmaking research. By not including possible variations in empirical legal studies of judicial behavior (beyond the appointing President’s party), the dominant law and courts research has yet to enter the realm of complexity comparable to advanced sabermatic research. Further, as law and courts is one of the most heavily researched areas using empirical methods, the simplistic assumptions of its models is indicative that other areas might be at even more basic levels of analysis.

I think the baseball analogy is particularly helpful in explaining my second objection to Dave’s reasoning. There are numerous concepts in sabermatic work that are beyond even sophisticated connoisseurs of such information. However, those advanced studies and ideas still find a way to inform statistical analysis in the mainstream (excluding Joe Morgan). I’m guessing that most people have no idea who Voros McCracken is, but he produced one of the most significant statistical findings in baseball history. McCracked discovered that, with few exceptions, a pitcher has little control over whether a ball in play is a hit or not. Instead, a pitcher can typically only control the three true outcomes: Walk, Strikeout, and Home Run. This insight supported by statistical analysis revolutionized the way we understand pitchers. Initially there was substantial resistance among baseball insiders. Had the management of the Diamondbacks, for example, listened to McCracken, there was no way they would have signed Russ Ortiz to the contract that every informed sabermatician knew was a disaster from day one. Eventually, McCracken’s ideas have infiltrated common understanding of educated baseball fans even though few probably know who McCracken is and even fewer could read and understand his study. As it turns out, even more complexity has been added to McCracken’s initial conclusions yielding even more insight into the relationship between pitching and hitting in baseball. More recent statistics such as WAR (Wins Above Replacement) and VORP (Value over Replacement Player) have started to be used by the mainstream press and baseball announcers. Yet, while people have come to understand the concepts measured by those stats, I doubt almost anyone could tell you how those stats are actually computed. Indeed, there is an ongoing fight among sabermaticians about which measurement of WAR is actually right (the difference is based upon which defensive metrics are used). For empirical legal studies to be important, not every study needs to be accessible. That isn’t to say that accessibility shouldn’t be a central goal. However, just like with sabermatics, the advanced studies ultimately can yield commonly held ideas.

Regardless of my quibbles with Dave’s post, I think this is an important discussion for scholars, empirical and otherwise, to have. And my agreements with his normative beliefs on the subject are more significant than the disagreements.

The so-called “whistleblower” panel effect occurs when a judge wants to draw attention to the actions of co-panelists that might be acting in an ideological extreme manner. In contrast, strategic incentives often point in a different direction because dissents rarely serve [...]]]> For people interested in judicial decision-making, one of the most interesting findings in the last decade was the evidence that judges on panels do not make decisions independent of one another. In fact, the political ideology of co-panelists has a strong connection to how an individual judge will vote in many cases. These “panel effects” are now well-known among scholars who are not regular readers of empirical work regarding the courts. However, the details, magnitude, and explanation for these panel effects are still disputed and ambiguous.

The so-called “whistleblower” panel effect occurs when a judge wants to draw attention to the actions of co-panelists that might be acting in an ideological extreme manner. In contrast, strategic incentives often point in a different direction because dissents rarely serve any function at the federal appellate level when en banc panels are infrequent and Supreme Court review even less likely. There are also strong incentives toward consensus among judges in the same circuits because of the limited value of dissents and the upsides to collegiality among judges that will serve on panels hundreds of times together. To these theories of panel decision-making, I wanted to share one of my recent findings.

In my studies of judicial activism and ideology at the federal appellate level, I created measures which concerned the inter-court relations of district and appellate courts. In doing so, I initially sought to control for panel effects among the judges being studied. However, as I assumed that the district court judge was essentially a hidden fourth member of the appellate panel, I decided to throw in the district judge’s ideology (as determined by the party of the appointing President). And in doing so, I found that the district court’s party connection seemed to have had an effect on the votes of the panelists.

There are three basic political arrangements of a judge’s co-panelists and the district court judge being reviewed: 1) they are all of the same party; 2) the co-panelists are of the opposite party as the district judge; and 3) the co-panelists are split with one being aligned with the district judge. As it turns out, a judge’s decision to affirm or reverse the district court was connected to which of the three arrangements existed as illustrated in the figure below.

While I included the above finding in my two recent articles in order to control for the varying alignments encountered by each judge, I did not discuss any explanations for the result. I have some ideas and have begun working on an article related to panel effects generally and the district court’s role in particular. So, as my time at Concurring Opinions has come to and end, I welcome any and all ideas you all might have about why the co-panelists having the opposite ideology of the district court increases the rate at which a judge votes to reverse.

I did my own count recently of the California Supreme Court opinions published during the past five years that relied on law reviews as authority: There were just six. This despite—or perhaps because of—the fact that law reviews have tripled in number since the 1970s. The 20 ABA-accredited law schools in California now publish a total [...]]]> It has been conventional wisdom for some time that the legal academy has become increasingly disconnected from the practice of law. And because of this, law reviews are said to be of little use to attorneys and judges. A recent piece in the California Lawyer by a law professor and former law school dean has received some attention for making these claims in the strongest terms. From the article:

I did my own count recently of the California Supreme Court opinions published during the past five years that relied on law reviews as authority: There were just six. This despite—or perhaps because of—the fact that law reviews have tripled in number since the 1970s. The 20 ABA-accredited law schools in California now publish a total of 82 law reviews. UC Berkeley’s alone publishes 14, while Stanford and UC Hastings each publish 9. Both law professors seeking tenure and law students seeking employment at elite law firms eagerly fill these volumes. But who reads them now? Surely not the judges who decide the law. And not practicing lawyers either.

As Adam Liptak of the New York Times observed a few years ago, “Articles in law reviews have certainly become more obscure in recent decades. Many law professors seem to think they are under no obligation to say anything useful or to say anything well. They take pride in the theoretical and in working in disciplines other than their own. They seem to think the analysis of actual statutes and court decisions—which is to say the practice of law—is beneath them.”

I’m not quite sure the method that the author used to identify just six citations to law reviews, but I performed a quick and dirty Lexis search and turned up far more. The bigger question, though, is whether the author is right that judges are not reading law reviews. A new article by David Schwartz and Lee Petherbridge indicates that, at least for the federal appellate courts, the conventional wisdom seems to be flat wrong. In fact, according to their study, law review citations have increased dramatically in the last twenty years (even when accounting for the increased number of journals). I couldn’t cut and paste the tables and graphs from the article, but the results regarding the proportion of court opinions that cite law reviews are clear.

So, why is the conventional wisdom so completely wrong on this point? Maybe citations are clerk driven. Or it could be that judges are misremembering the golden age of law reviews. I want to offer a different explanation, though. Judges and commentators have argued that because law review articles are rarely concerned with legal doctrine, they are of no use to those practicing law and judging cases. I think there is a strong argument that it is precisely because law reviews are unconcerned with doctrine, they are much more valuable to judges and are cited as a result. Articles that merely outline, discuss, or analyze doctrinal areas do little to advance the judge’s knowledge beyond what he or she could establish alone. Instead, it is the broader theoretical points and empirical studies that are outside of a judge’s metaphorical wheelhouse. Just as judges will not allow an expert on law to testify, it makes little sense for law reviews to inform judges about topics which they are comfortable. Instead, like the expert on ballistics evidence or biological sciences, law reviews that are increasingly removed from “law” actually educate judges where they are weakest. And, as a result, law reviews are increasingly being cited by the very judges who proclaim their uselessness.

Baudrillard’s view of the Gulf War [...]]]> In 1991, Jean Baudrillard provocatively titled a collection of essays as The Gulf War Did Not Take Place. In the book, Baudrillard did not embrace some bizarre conspiracy theory that the Gulf War was staged like many have contended about the Apollo moon landing. Instead, Baudrillard argued that the invasion of Iraq was not a “war” in any meaningful sense of the word. Although the media treated the run up to the conflict as the beginning of a possible global conflagration, the outcome was preordained. I remember the repeated statement that Iraq had the fourth largest army in the world as a particularly odd statement of propaganda. The death and environmental damage were real, but, to Baudrillard, the non-event could not be termed a “war.”

Baudrillard’s view of the Gulf War is perhaps more applicable to the so-called “federalism revolution” that was led by Justice Rehnquist. As a result of the Court’s opinions in Lopez and Morrison, many legal scholars felt that the Commerce Clause was seemingly reinvigorated as a means of limiting federal power. As hundreds of law reviews embraced the revolutionary narrative, the popular press joined them. To many, the revolution came to a screeching halt with the Court’s ruling in Raich. However, it was still possible to reconcile the doctrine in Raich (as a logical extension of Wickard v. Filburn) with Lopez and Morrison. Whereas marijuana was an economic good, sexual violence and guns were not (although the tension with Lopez was clear). However, with the Court’s recent decision in United States v. Comstock, the legacy of Justice Rehnquist in regards to the Commerce Clause seems to have vanished.

Although the district court and Fourth Circuit panel thought it was clear under Morrison and Lopez that the Commerce Clause could not justify federal civil commitment of certain persons believed to be a threat to commit future sexual violence, the Court embraced the government’s framing of the basis for federal jurisdiction. As there was no economic good at issue in Comstock and the parallel to the VAWA statute in Morrison was strong because of the regulation of sexual violence, the Court treated the case as involving the Necessary and Proper Clause. Justice Breyer created a multi-part test that seemed to remove the word “Necessary” from the Clause and embraced a very broad view of proper exercise of federal power. And how the statutes in Morrison and Lopez would not be justified by the newly crafted test was never explained.

One could simply view the Roberts’ Court’s counter-revolution as having shutdown Justice Rehnquist’s efforts. However, I think it is simpler and more accurate to say that there was never any revolution. The Court never wrote in terms of revolution. Even when deciding Lopez and Morrison, the Court left intact expansive precedents like Wickard. The government’s winning brief in Comstockrelied almost exclusively on precedents before the opinions of Rehnquist’s “revolution.” Like the media in the period before the first Gulf War, legal scholars saw a radical event unfolding. But just like the outcome in the Gulf War was preordained, none of us should be surprised that the hiccup in Commerce Clause jurisprudence represented by Lopez and Morrison was a non-event. And the collective focus of legal scholars on those decisions was, in hindsight, misguided.

And yet the belief in the enduring power of the Commerce Clause as a check on federal power has continued in regard to health care reform. Noted scholars, pundits, and other media have argued that existing doctrine in the area makes the recent federal health care law unconstitutional. Given Raich and Comstock, I admit to being baffled by this belief. The distinction that the government is regulating inaction seems wholly unconnected from the Commerce Clause. That Clause only determines who has the proper jurisdiction to deny liberty (the Feds or the States) – it has not offered a shelter for individuals from government regulation. Of course, no one saw Lopez and Morrison coming. Still, it seems that after Comstock, as a wholly doctrinal matter, the two most famous federalism decisions of the Rehnquist Court should be given their proper stature as minor opinions of little ongoing significance. And claims of a federalism revolution should be treated with skepticism.

Consider the very interesting and valuable studyby Michael Heise and Gregory Sisk of judicial ideology in religious liberty cases published in the Northwestern University Law Review a few years ago. The study found, consistent [...]]]> In my last post, I discussed the measure I have proposed in my article for the judicial ideologies of federal appellate and district judges. That leads to the question: how do we know if my measure is “good?” Anyone can make up a bunch of numbers and formulas and declare their measure to be better than existing ones. How can it be said with any certainty that one measure of ideology is quantitatively or qualitatively better than another? That is one of the trickier questions in empirical legal studies of federal courts.

Consider the very interesting and valuable studyby Michael Heise and Gregory Sisk of judicial ideology in religious liberty cases published in the Northwestern University Law Review a few years ago. The study found, consistent with prior research, that ideology had a modest correlation with outcomes in religious liberty cases. How was ideology measured? Using Common Space Scores. What if the Common Space Scores were actually a poor measure of judicial ideology and votes in religious liberty cases were actually a “better” indicator of a judge’s ideology?  How would we know? Heise and Sisk chose to use Common Space Scores even while noting in detail the potential problems with the Scores. Of course, Heise and Sisk did so in part because they framed their study as part of a response to the firestorm created by an article by Lee Epstein and Gary King attacking empirical legal studies by legal academics. The types of inferences that can be drawn from the Heise and Sisk study would change dramatically if Common Space Scores were not strong indicators of judicial ideology.

In the article that I previously discussed, Josh Fischman and David Law tackled this sticky problem by assessing the leading measures of ideology (as well as Fischman’s consensus model) based upon their ability to predict case outcomes. Specifically, Fischman and Law looked at whether Common Space Scores of the appointing President’s party were good predictors of panel votes in Ninth Circuit asylum cases. The assumption was made that a pro-asylum vote was indicative of liberalism. They found that Common Space Scores offered very little improvement over the appointing President’s party.

Following a similar methodology as Fischman and Law, I tested my Ideology Scores against the two leading measures in criminal cases. I used 1,000 criminal cases (selected using a pseudo-random number generator) from my dataset and recalculated my Ideology Scores without those cases to prevent circularity problems. I assumed that on average a vote for a criminal defendant was a more liberal outcome. I then performed logistic regressions on the panel median and net ideology scores using the three measures against outcomes in favor of criminal defendants. Notably, and perhaps surprisingly, neither Common Space Scores nor the party of the appointing President had a statistically significant relationship with votes for criminal defendants. In contrast, my Ideology Scores showed such a relationship for both the panel median and net scores (p=0.004 and p=0.013, respectively). I also performed goodness-of-fit tests (pseudo R² tests) which assessed whether my measure was a better fit for the data in the 1,000 criminal cases. In every test, my measure had a better goodness-of-fit than the leading measures. The figures below shows the relationship between a criminal defendant’s winning percentage before a judge and that judge’s Ideology Scores and Common Space Scores.
The first graph shows that a liberal under the Common Space Score measure is only slightly more likely to vote for a criminal defendant than a conservative. In contrast, a liberal as defined by the Ideology Scores is much more likely to vote for a criminal defendant. This difference is largely consistent with anecdotal evidence of Clinton appointees. President Clinton ran for President contending that he would be tough on crime. He signed into law some of the harshest criminal laws in history. And according to many observers, his appointments to the federal judiciary were not particularly concerned with defendant rights or sympathetic to defendants in general. The Common Space Scores and appointing President’s party measures cannot account for the priorities of the Clinton administration because every appointment he made is assumed to be liberal. The Ideology Scores, being based upon performance of the judge, allow for party switching. As a result, a clearer trend of liberals voting for defendants emerged. A similar test was performed with civil rights (Title 7, ADA, and Section 1983) cases and the results were similar. Overall, the various indicia of being a “better” measure of judicial ideology favor the Ideology Scores against both the Common Space Scores and President’s party.

So, if there is reason to believe the Ideology Scores are a useful tool for measuring ideology, what can be done with that information? In my next few posts, I will talk about some of the findings in my paper regarding judicial ideology among the circuits, based upon law school attended, and related to prior work experience.

My idea for identifying the ideologies of federal appellate judges was to determine the rates at which such judges agree and disagree with “conservatives” and “liberals” on the bench. The assumption was that like-minded judges will vote together more often and judges with dissimilar ideologies will tend to disagree. By focusing on the agreements and disagreements among the judges, the goal was to pinpoint their respective ideologies (via “ideal points”). This is an agnostic method that necessarily faces all of the shortcomings of such an [...]]]> Last week, I tried to outline the difficulties associated with measuring judicial ideology in regards to the limited alternatives that have been offered by scholars. In this post, I hope to describe how I have measured it and attempted to overcome the various obstacles brought about by my methodology.

My idea for identifying the ideologies of federal appellate judges was to determine the rates at which such judges agree and disagree with “conservatives” and “liberals” on the bench. The assumption was that like-minded judges will vote together more often and judges with dissimilar ideologies will tend to disagree. By focusing on the agreements and disagreements among the judges, the goal was to pinpoint their respective ideologies (via “ideal points”). This is an agnostic method that necessarily faces all of the shortcomings of such an approach that I previously described.

The initial concern with such a method is that there are far too few disagreements among the judges on the Courts of Appeals. Indeed, in the 10,242 cases in my dataset, there were only 288 dissents (including partial dissents). Some judges who participated in over 100 cases were not on a panel in which there was a single dissenting vote. Looking at the Courts of Appeals alone was, thus, unlikely to offer much information. My solution was to treat the district judges being reviewed as pseudo-fourth members of the appellate panel. After all, the district judge reviewed the same legal issue as the appellate panel and rendered judgment on that very same issue. Notably, there are far more disagreements with district judges in the form of reversals. Also, by including the district judges, my methods also allowed data to be harvested from unanimous affirmances as well (as described below).

However, my solution faced numerous difficulties which necessitated adding nuance to the general concept for the measure. Importantly, I incorporated these details into my Ideology Scores:

• Case Mixes – different judges hear different types of cases. In my dataset, the most significant difference that was relevant to the measure was in relation to criminal and civil cases. District judges were reversed twice as often in civil cases. Accordingly, each judge (or other unit of measure) was analyzed assuming an average distribution of civil and criminal cases.
• Standards of Review – because appellate judges review district court judges with deferential and non-deferential standards of review, there is a need to adjust for standards used. For example, a reversal in a criminal case with a deferential standard of review is far less common than a reversal in a civil case with a non-deferential standard. So, in addition to the case mix adjustment, judicial votes were further weighted based upon the standard of review used.
• Rates, not Frequencies – judges have differing co-panelists and judges reviewing their decisions. As a result, it was essential to focus on the percentages of agreements rather than the absolute numbers. So, judges were evaluated for the percentage with which they agreed with judges appointed by Republicans and by Democrats in each of the relevant sub-categories (i.e., criminal case with a non-deferential standard). The use of rates also provides a stable baseline for comparisons so that judges who are in the minority ideology on their court are measured in the same manner as those in the majority group. Rates also incorporate affirmances into the measure – something that previous agnostic measures have been unable to do.
• Panel Effects – one of the most significant results in recent empirical legal studies has been that the composition of the panel determines to a large degree how an individual judge votes. Judges who sit with more Republican appointees will tend to vote in a more conservative direction than if they sat with Democratic appointees. Accordingly, the measure was adjusted to assume that judges had the same co-panelists. This was done by determining the differential in co-panelists arrangements between two Republican appointees and two Democratic appointees.
• Inter-Circuit Adjustments – the Courts of Appeals in my dataset have their jurisdiction determined by geography. As a result, they review differing sets of cases and differing district judges. This would tend to make inter-circuit comparisons impossible. However, the circuits are not entirely insular. Judges who have taken senior status often travel to different circuits and sit by designation. In my dataset, there were 2,472 votes issued by 26 “traveling judges” in either their home or away circuits. Using data from how those judges voted at home and away, adjustments were made to the Ideology Scores of judges on each circuit. This adjustment assumed that a judge remains constant in his or her ideology regardless of circuit. Again, accounting for all of the above adjustments, it was determined whether a particular circuit made travelling judges vote more liberally or conservatively.

Based upon all of those adjustments, individual Ideology Scores were determined for every federal appellate judge and district judge who issued an opinion in 2008 or had their opinion reviewed in 2008. However, due to sample size concerns, the article focused on 138 judges who sat on the Courts of Appeals that year. In my next post, I will discuss how my measure does against the two leading measures. Below are some example scores (in comparisons to the other leading measures) from notable judges studied in my article. Total interactions refers to the total number of co-panelists and district judges included the sample. Higher scores are more conservative and lower scores are more liberal.

Case Outcome Coding – this category relies on a researcher going through a sample of cases and coding whether the judge’s or panel’s vote were “liberal” and “conservative” in ideological direction. While this technique can work reasonably well at the Supreme Court level (although [...]]]> In my last post, I talked about the shortcomings with the leading measures of judicial ideologies. There are strong reasons, however, why those measures have dominated empirical legal research. If a scholar wants to assess the ideology of a judge, he or she is likely to try a technique that fits within one of these three categories: Case Outcome Coding, External Proxies, and Agnostic Coding. Each of these types of measures has advantages and disadvantages for federal judges not serving on the Supreme Court.

Case Outcome Coding – this category relies on a researcher going through a sample of cases and coding whether the judge’s or panel’s vote were “liberal” and “conservative” in ideological direction. While this technique can work reasonably well at the Supreme Court level (although there are shortcoming there as well), it is extremely difficult to apply to the Courts of Appeals or federal district courts. There is an enormous amount of labor required for sufficient samples to be accumulated for individual judges. Further, coding decisions are much more subjective than at the Supreme Court level as most of the federal docket is filled with cases that have little political salience. As Tonja Jacobi and Matthew Sag recently observed, “the last four decades of empirical scholarship have proceeded without a sophisticated objective measure of case outcomes.” Given the low level of disagreement among judges on the Courts of Appeals (due to consensus norms, “easy” cases, or other strategic incentives), there is an additional problem of making valid assessments without an enormous sample of data for each judge.  It is also possible that a researcher could try to code the ideological direction of methods instead of outcomes, but such a technique would tend to accentuate the difficulties described above. As a result, such measures have never been attempted on a comprehensive basis for individual judges on the federal appellate or district courts.

External Proxies – this category includes techniques which rely on a party external to the judges reviewing the cases to assess the ideology of the judges. The President’s political party and Common Space Scores are external proxies. Such measures are often easy to implement, but rely on sometimes questionable assumptions about the quality of the proxy utilized. Especially for the lower court judges, it is unclear to what degree ideology is the driving force in a President’s appointment of a judge. I have addressed the general shortcomings with this type of measure in my previous post.

Agnostic Coding – this category of measures is based upon identifying voting blocs among judges. I believe the term “agnostic coding” was created by Fischman and Law and, in my mind, perfectly captures how these metrics work. Agnostic coding relies on the premise that like-minded judges will vote together more often than judges of opposing ideology, but does not require any ideology coding of the case outcome. By looking at who an individual judge votes with and against most often, assessments of that judge’s ideology can be made. The Martin-Quinn scores, which are becoming the leading measure of ideology at the Supreme Court level, rely on an agnostic technique. Agnostic coding allows for analysis of large amounts of data because of the limited, typically objective, coding involved. However, such a technique has generally been thought to be impossible to apply to the Courts of Appeals and federal district courts. For district judges, there are no “voting blocs” because each judge sits alone. For appellate judges, simulation techniques like those used by Andrew Martin and Kevin Quinn are largely hindered by the very low dissent rate on the Courts of Appeals (less than 2% in my data). Since agnostic coding does not include the ideological direction of cases, techniques in that category have not been able to harvest data from unanimous panels. Beyond ignoring the overwhelming majority of cases, agnostic coding using current methods also requires strong prior assessments of what constitutes “liberal” or “conservative” ideology. Without such assessments, the results may indicate different groups of judges, but there won’t be any clarity as to what ideologies that those groups embody.

Given the numerous obstacles for each possible methodology technique, it is not surprising that scholars have not added to the arsenal of measures for judges on the Courts of Appeals and federal district court judges. In my next post, I will attempt to outline my method for measuring judicial ideology of federal appellate and district judges and address how I confront the various difficulties that I have discussed.

The common attack of partisans in the recent judicial confirmation battles has been to brand the nominee as “out of the political mainstream.” Such accusations have been made against Justice Roberts, Justice Alito, Justice Sotomayor, and General Elena Kagan. However, the argument has also been deployed against lower court nominees as well. My favorite use of the device has been in the opposition to Ninth Circuit nominee Professor Goodwin Liu. In what typifies our modern political theater, Liu has been labeled as “out of the mainstream” in large part because he had the audacity to assert that Justice Alito was out of that very same “mainstream.” Of course, no [...]]]> I wanted to start by thanking the Concurring Opinions gang for having me as a guest this month.

The common attack of partisans in the recent judicial confirmation battles has been to brand the nominee as “out of the political mainstream.” Such accusations have been made against Justice Roberts, Justice Alito, Justice Sotomayor, and General Elena Kagan. However, the argument has also been deployed against lower court nominees as well. My favorite use of the device has been in the opposition to Ninth Circuit nominee Professor Goodwin Liu. In what typifies our modern political theater, Liu has been labeled as “out of the mainstream” in large part because he had the audacity to assert that Justice Alito was out of that very same “mainstream.” Of course, no one ever explains exactly what the “mainstream” is or what it takes to be “outside” of it.

Even though it would likely make little difference in the political arena, one might think this is an area where empirical legal studies might be able to offer substantial assistance to the debate. After all, with the explosion of such research by legal academics in recent years, one might think that we would have a firm grasp on what is the mainstream of judicial ideology and where a nominee fits within that schema. At the very least, one might think that for judges being nominated to a higher position in the judiciary, scholars might be able to effectively pinpoint a judge’s ideology based upon past behavior on the bench. However, there simply is no such ability based upon existing empirical research.

Justice Sotomayor’s nomination battle is highly illustrative on this point. Despite the nearly seventeen years on the bench at the time of her nomination, Justice Sotomayor’s ideology was reduced to three words: “Ricci” and “wise Latina.” The rest of her judicial record (and the fact that she was originally appointed to the federal bench by a Republican President) drifted into the background. While it seemed more than a tad ridiculous, scholars could offer little to dispute the very narrowly focused attacks on then-Judge Sotomayor’s ideology beyond subjective assessments of her record.

Last time I was a guest here, we were in the midst of President Obama’s attempts to have Justice Sotomayor confirmed and I was discussing my method for assessing whether she was a judicial activist. This time around, I want to move from the concept of “activism” to “ideology.” In a new article that I have been working on, I discuss why empirical legal studies have failed in this area and offer a new method for addressing such conflicts in the future. I will be blogging about the problems with measuring ideology in a few posts this month.

So, while I cannot yet tell you who the most activist judge was in 2008 because I have only reviewed data from five circuits, I can share my preliminary results for a few higher profile judges, including the most recent nominee to the United States Supreme Court. Here are the activism scores based upon my preliminary data for some of the highest profile judges in the Second, [...]]]> In my last post, I gave some results based upon collective groupings of appellate judges. In this post, I want to focus on the performance of individual judges. The primary reason that I am working to create a relatively large dataset is to allow for individual judge assessments. That has not been possible with the existing appellate court databases.

So, while I cannot yet tell you who the most activist judge was in 2008 because I have only reviewed data from five circuits, I can share my preliminary results for a few higher profile judges, including the most recent nominee to the United States Supreme Court. Here are the activism scores based upon my preliminary data for some of the highest profile judges in the Second, Third, Fourth, Seventh, and Eighth Circuits ranked from most activist to least activist:

As noted in the chart, my average activism score is -10.40%. That means Judge Sotomayor, who is highlighted in green, is less activist than the average judge in my dataset. I have highlighted in red the other judges that were rumored to be on President Obama’s shortlist. As I have noted elsewhere, Judge Sotomayor is slightly more activist than average in criminal cases, perhaps owing to her extensive experience in that area of law. Although one of the primary attacks on Judge Sotomayor is her alleged judicial activism, I do not think such an attack is supported by her record in 2008. I would be interested in seeing any data which actually supports a charge of activism. Simply cherry picking a few cases over the long period she has been on the appellate judge or relying on sentences out of public statements gives almost no insight into Judge Sotomayor’s overall judging philosophy and technique. Further, such selective review of cases and speeches offers no information about how similarly situated judges have performed during the same period.

Outside of Judge Sotomayor, I wanted to highlight a couple other judges in my chart: Judges Posner, Easterbrook, and Wilkinson. Those three are notable because they have taken the unusual step of writing generally about judging and specifically about activism. Based upon my reading of their writings, I think all three judges perform as you might expect. I think a fair, but crude, assessment of Judge Posner’s perspective on how a judge should decide a case is: “if it’s broke, fix it.” We would expect that Judge Posner would not be particularly deferential to the opinions of others if he thinks that they are in error. Thus, we might expect the data to show, relatively, that Judge Posner is more activist. Judge Easterbrook has openly lamented the activism of judges. We might then expect him to be substantially less activist than an average judge. Judge Wilkinson has been perhaps the most aggressive judge in attacking judicial activism. He even vocally targeted the United States Supreme Court decision in Heller even though that opinion probably fits with his policy ideology. Judge Wilkinson, then, might be expected to be among the least activist judges. That all three judges fit with some of the information we have about each of them might indicate that the data I have collected is a valid indicator of an individual judge’s activism.

I should note that activism by judges is not inherently a negative quality. On the other extreme, a judge who is too deferential to other constitutional actors might not be adequately fulfilling his or her responsibilities under the Constitution. Such a judge may simply be acting as a rubber stamp by failing to exercise proper judgment. Further, since my measure is only relative between judges, I do not attempt to describe what the “ideal” amount of activism by a judge is. So, nothing here should be taken as a disparagement of a particular judge. My hope is simply to add more data to understanding judges and judicial behavior.

As I mentioned previously, this is an ongoing project so I welcome comments and suggestions. As this will be my last post, I wanted to thank Dan, Dave, and the rest of Concurring Opinions for having me here.

One of the most remarkable things I have found so far is that, although each of the circuits has a relatively consistent reversal rate, the activism scores vary by significant margins. The chart below shows the reversal rates and activism differentials for each of the five circuits that I have examined:

The activism scores are negative numbers with the higher numbers indicating a higher degree of activism. Thus, the 3rd Circuit is the most activist and the 4th Circuit is, by far, the least activist.

Of course, one thing that is always interesting to look at is which [...]]]> In my previous post, I finally got around to explaining my measure of judicial activism. In this post, I will give some of the results based upon my preliminary data.

One of the most remarkable things I have found so far is that, although each of the circuits has a relatively consistent reversal rate, the activism scores vary by significant margins. The chart below shows the reversal rates and activism differentials for each of the five circuits that I have examined:

The activism scores are negative numbers with the higher numbers indicating a higher degree of activism. Thus, the 3rd Circuit is the most activist and the 4th Circuit is, by far, the least activist.

Of course, one thing that is always interesting to look at is which President’s appointees are the most activist. Here are the results I have so far:

This chart should be taken with a very large grain of salt. Although in some senses, I already have a large amount of data, in others I have very limited sample sizes. When I start analyzing data that uses the judge as a unit of measure, I only have 52 judges with adequate sample sizes. So, there are few representatives for some presidents. That is also why I am not including any regression analysis in these posts. I include this chart simply to show one of variables that I will explore in regards to activism. Other examples of variables I will examine include background experience (district court, private practice, prosecutor, law professor, etc.), conditions at the time of the appointment (election year, unified government, etc.), and other assorted factors (law school attended, activism in particular areas of law, ABA ratings, etc.).

As this project is still ongoing, I welcome any comments and/or suggestions. In my next, and probably last, post, I will give some information about some high profile judges including Judge Sotomayor.

Update: Based upon popular demand, I have changed the second graph to a bar chart.

Since judicial activism at its core is about substituting judgment of other constitutional actors, for federal appellate courts it makes the most sense to study where they do most of their work: reviewing the judgments of district courts. That ensures large sample sizes, wide coverage of different areas of law, and decisions that are still restrained so some non-activist baseline can be derived.

To that end, standards of review offer a powerful tool to understand judicial decision making at the federal appellate level. There are deferential (i.e. clear error) and non-deferential (i.e. de novo) reviews [...]]]> In my last post, I briefly described the major approaches to studying activism. In this post, I will outline my measure for the concept as well as the contours of my dataset.

Since judicial activism at its core is about substituting judgment of other constitutional actors, for federal appellate courts it makes the most sense to study where they do most of their work: reviewing the judgments of district courts. That ensures large sample sizes, wide coverage of different areas of law, and decisions that are still restrained so some non-activist baseline can be derived.

To that end, standards of review offer a powerful tool to understand judicial decision making at the federal appellate level. There are deferential (i.e. clear error) and non-deferential (i.e. de novo) reviews of district court judgments. As you can see from the chart below derived from my dataset, standards of review do affect reversal rates.

So, what can we learn from standards of review? I contend that by comparing a judge’s reversal rate in non-deferential cases with the reversal rate in deferential cases, we can effectively measure the concept of judicial activism. So, appellate judges are more “activist” when they reverse district court judgments under a deferential standard at a higher relative rate compared to reversals using a non-deferential standard. My exact measure (or activism “score”) for activism is reversal rate in non-deferential cases minus the reversal rate in deferential cases. This measure captures when a judge is not deferring to other constitutional actors when we would normally expect him or her to do so. 

The measure has the advantage of not being based upon the substantive outcome of the case. A judge can use either a deferential or non-deferential standard and still find for either party. Since we might think that activist judges are not keen to make clear that their decisions are actually activist, looking at substantive outcomes can be tricky as judges try to mask an appearance of activism. Since standards of review are usually non-controversial (in that the parties rarely dispute over which standard applies) formal rules, we might think that there will be less ability for judges to mask their activism. Ultimately, the failure to defer by a judge over time indicates a relative propensity for activism even if we cannot say for certain that any individual decision is activist. As will be clear in my next post, judges vary quite a bit in their deference under standards of review.

My dataset thus far includes all 2008 cases that used standard of review related words (except habeas and immigration cases) for five circuits: the 2nd, 3rd, 4th, 7th, and 8th. Eventually the dataset will include all eleven numbered circuits as well as the D.C. Circuit. The dataset covers 3,873 cases and 11,583 judicial votes. Each vote is coded for, among other things, standard of review, type of vote, and type of case. I have also integrated biographical information for each judge to determine if background or demographic characteristics are related to judicial activism.

In my next post, I will detail some of my results based upon my preliminary data.

The existing empirical scholarship is largely focused on two things: activism by the U.S. Supreme Court and judicial review of the other federal branches. While there is a lot of interesting work within those studies, I think the focus on Supreme Court review of other branches offers an extremely narrow view of judicial activism.

The Supreme Court is an unusual body in American courts. It is not subject to any higher review. The Justices are largely free to decide a case however they feel. As a result of the unrestrained nature of the Court, “activism” [...]]]> In my last post, I offered a new definition of judicial activism as applied to federal appellate judges. In this post, I will discuss some of the existing research on the subject.

The existing empirical scholarship is largely focused on two things: activism by the U.S. Supreme Court and judicial review of the other federal branches. While there is a lot of interesting work within those studies, I think the focus on Supreme Court review of other branches offers an extremely narrow view of judicial activism.

The Supreme Court is an unusual body in American courts. It is not subject to any higher review. The Justices are largely free to decide a case however they feel. As a result of the unrestrained nature of the Court, “activism” is a hard concept to measure. There is difficulty in finding a baseline against which to measure the Justices. A Justice who is an outlier on one Court might have regularly been in the majority during a different era.  The problem of measuring activism is compounded by the Court’s very small docket. Because there are so few cases, the sample sizes are not large. Further, because the docket is self-selected, the cases are not random and are concentrated in just a few areas of law.

Studying federal appellate courts is more informative because the courts are restrained and are responsible for defining much more of federal law and other issues in federal courts. As the Supreme Court’s docket as shrunk, the Courts of Appeals have increased in importance. Further, as is the case with Sotomayor, studying appellate judges can give you more information about a Justice before they join the Court rather than waiting until someone has life tenure and is unreviewable. Because federal appellate courts are limited by the decisions of other panels, en banc review, and U.S. Supreme Court law and review, it is much easier to establish a baseline against which to measure activism.

The emphasis on inter-branch relations in activism is also problematic. Again, there are very small sample sizes since only rarely do courts review the constitutionality of legislation or other branch actions. There is also a problem in deciding which acts of judicial review are “really” activist. Not all Court actions striking down legislation are created equal. The focus on inter-branch review is particularly problematic for federal appellate courts since those cases are an incredibly small percentage of the overall docket.

As a result, to better appreciate and understand the concept of judicial activism, I believe it is helpful to move beyond the Supreme Court and to consider actions by the courts other review of other branches. 

In my next post, I’ll describe my measure and my existing dataset.