Home | About | RSS Feed | Contact and Publicity Guidelines | Comment Policy the Law, the Universe, and Everything 


advertise-here4


Slip Opinions


University governance as a new topic of public discussion.

An unusual profile of Mary Anne Franks (kw)

Aggressive copyright litigation run amok. (fp)

USA Today's Matt Krantz quoting me on Warren Buffett joining Twitter.  (LAC)

Private prisons? Why, sure! What could possibly go wrong? (kw)

TNR profiles Susan Crawford (kw)

Berkshire Hathaway is bigger than Warren Buffett.  Manual of Ideas (LAC).

Guns don't shoot people, kitchen appliances shoot people (kw)

Via Glom, Sat Eve Post review of The Essays of Warren Buffett.

Jack Coffee on Bad Plaintiffs' Counsel in M&A Deals and What Must Be Done to Break Them


Our Podcast

Subscribe to Law Talk


  • Posts by Author

  • Categories

  • Archives


  • Recent Comments


    • Marty Lederman on Copyright’s Constitutional Chameleon

    • Brett Bellmore on Copyright’s Constitutional Chameleon

    • Ryan Calo on Franks on "How to Feel Like a Woman, or Why Punishment Is a Drag"

    • Anon on Wachtell Lipton's Errors on Shareholder-Paid Director Bonuses

    • Sean Croston on Copyright’s Constitutional Chameleon

    • Shag from Brookline on Kentucky: Boy, 5, Kills Sister, 2

    • jdgalt on Wrongful Birth and Adoption

    • Sub Specie AEternitatis on The Pervasive Effect of Priors: Part Four

    • victim on Criminal Prosecution for Scientific Fraud

    • jdgalt on Kentucky: Boy, 5, Kills Sister, 2

    • Brett Bellmore on Kentucky: Boy, 5, Kills Sister, 2

    • Christine Hurt on Kentucky: Boy, 5, Kills Sister, 2

    • Kaimi on Welcome to Wills Lab

    • A.P. on Kentucky: Boy, 5, Kills Sister, 2

    • Malcolm on Wrongful Birth and Adoption
  •  

    Site Meter

    About the Blog

    Concurring Opinions is a multiple authored, general interest legal blog.

    (Image: Wikicommons)

What Should a Judge’s Reversal Rate Be?

posted by Dave Hoffman

Via TPM, I found that critics of Judge Sotomayor have made some hay of her reversal rate in the Supreme Court (50% of 6 cases heard).   Though I like data more than the next guy, I will freely admit that this use of quantitative empirical legal studies is demonstrably silly.  I’m going to try to demonstrate to you in this post that a reversal rate of fifty percent is exactly what the state-of-science would predict to be ordinary [or even below average], and a rate of zero (which some might think of as emblematic of a law-abiding judge) tells you that the judge in question might be departing quite severely from precedent.

Let’s start with the obvious.  Most appellate court opinions aren’t subject to a cert petition; almost no cert petitions are granted; only some granted cert petitions end up in full argument and a written opinion by the Supreme Court.  Throughout this long process, parties may settle their cases and exit the system.  They may do soduring briefing before the appellate court or after argument (with some limitations), they may settle after an opinion issues but before a cert petitition is filed, and they may settle thereafter until the Court rules (again, with some limitations based on mootness doctrine).  This potential for settlement after the appellate court issues its mandate creates selection effects.

Though such selection effects are likely less predictable & more dominated by wealth & party characteristics than the immense selection that occurs in the district court, it remains that case that the universe of cases that survive the decision to appeal and the decision to grant certiorari is significantly winnowed.  That winnowing produces a distinct set of cases.   Cases before the Supreme Court contain legal & factual issues more finely balanced than those that issued from the courts below.  To put it another way, cases are argued (usually) because the parties both believe they are going to win.  If the parties are rational & wealth neutral, an assumption that sometimes holds, we should expect that the resulting decisions from the Court will be a bit of a random walk.  (See my earlier post on bankruptcy scholarship for more on this hobby-horse of mine.  Also, note that much of this applies to civil cases, not criminal cases, which create unique settlement patterns.  The actual rate of reversal, over all cases, ranges between 60 and 75%.)

What’s the upshot?  An appellate judge’s “reversal statistic” tells you less than you think about the “merits” of her opinions, or even how such opinions stacked up against governing Supreme Court precedent.  Ironically, when a judge significantly departs from precedent either for or against the plaintiff, settlement may be particularly likely, as the parties’ chances above are quite clear: thus a judge who convinces her colleagues to depart from precedent often will almost never be reviewed or reversed by the Supreme Court. Cases where the judge stuck with precedent, by contrast, may face appeal and reversal, especially if the precedent shifted due to ideological change on the Court itself.

There’s lots of good work on this, much of it recent.  And what it teaches me is that we have no idea what an appropriate reversal rate for an appellate judge ought to be.  It also suggests that there is no way to evaluate the quality of an appellate judge’s work except to read her opinions and decide for yourself what you think of them.  This is a clear instance where statistics mislead.


 May 26, 2009 at 3:20 pm   Posted in: Empirical Analysis of Law, Supreme Court   Print This Post Print This Post

Responses (6)

  1. Steven Lubet - May 26, 2009 at 3:28 pm

    Doesn’t SCOTUS reverse slightly more than 60% of all cases?

  2. Dave Hoffman - May 26, 2009 at 3:29 pm

    Steve, I think that’s a blended crim/civ statistic. But you are of course right that the overall reversal rate also strongly suggests that selection & case selection by the justices is operating pretty strongly.

  3. Howard Wasserman - May 26, 2009 at 4:22 pm

    There also is the simple fact that being reversed does not tell us anything about whether the lower-court judge was right or wrong, only that her views were or were not in step with those of the reviewing court. We are back to the old saw: We are not final because we are infallible, we are infallible because we are final.

  4. Jake - May 26, 2009 at 6:22 pm

    Does it make sense to evaluate a federal circuit court judge’s reversal/affirmance rate by limiting the analysis to cases that are granted cert based on circuit splits?

  5. Sam Baumgartner - May 27, 2009 at 8:50 am

    Apart from selection effects, a “sample” of six is not exactly large enough to draw valid descriptive inference, in this case the implied inference from her reversal rate to her level of fidelity to precedent.

  6. Glenn Johnston - June 22, 2011 at 4:22 pm

    Please review the following stats on reversals:
    2009*: http://www.scotusblog.com/wp-content/uploads/2010/07/Final-Charts-070710-10.pdf

    Partial 2008 Stats: http://www.scotusblog.com/wp-content/uploads/2010/06/Preliminary-Stats-OT09_062610-8.pdf

    All else available on Scotusblog.

Leave a Reply

Spam protection by WP Captcha-Free


  • « Previous post
  • Next post »

Authors

Daniel J. Solove
Kaimipono Wenger
Dave Hoffman
Frank Pasquale
Deven Desai
Danielle Citron
Lawrence Cunningham
Sarah Waldeck
Jaya Ramji-Nogales
Solangel Maldonado
Gerard Magliocca

Guests

Kelli A. Alces
Taunya Lovell Banks
Ryan Calo
Claire Hill
Jay Kesten
William McGeveran
Meredith Render
Aaron Saiger
David L. Schwartz
Olivier Sylvain
Charles K. Whitehead
Aaron Zelinsky


















Previous Guests

Michael Abramowicz
Michelle Adams
Robert Ahdieh
Marvin Ammori
Michelle Anderson
Laura Appleman
Derek Bambauer
Taunya Lovell Banks
Ann Bartow
Steven Bellovin
Adam Benforado
Gaia Bernstein
Francesca Bignami
Josh Blackman
Joseph Blocher
Jeremy Blumenthal
Kathleen Boozang
Bruce Boyden
Donald Braman
Khiara Bridges
Al Brophy
Neil H. Buchanan
Bill Burke-White
Scott Burris
Paul Butler
Ryan Calo
Naomi Cahn
Anupam Chander
Miriam Cherry
Jack Chin
Glenn Cohen
Gabriella Coleman
Jennifer Collins
Caroline Mala Corbin
Thomas Crocker
andré douglas pond cummings
Allison Danner
Laura DeNardis
Brannon Denning
Deven Desai
Mike Dimino
Mark Edwards
Maxine Eichner
Jessica Erickson
David Fagundes
Lisa Fairfax
Joshua Fairfield
Christine Haight Farley
Kim Ferzan
Dan Filler
Mary Anne Franks
Susan Freiwald
Michael Froomkin
Amanda Frost
Brian Frye
Timothy Glynn
Rachel Godsil
Eric Goldman
Kyle Graham
David Gray
Craig Green
Tristin Green
Jonathan Hafetz
Vivian E. Hamilton
Meredith Harbach
Michelle Harner
Angela Harris
Jeffrey Harrison
Hosea Harvey
Erica Hashimoto
Jennifer Hendricks
Carissa Hessick
Laura Heymann
Robert Hillman
Gilbert A. Holmes
Nicole Huberfeld
Christine Hurt
Darian Ibrahim
Sherrilyn Ifill
John Ip
Shavar Jeffries
Kevin Johnson
Kristin Johnson
Jeff Jonas
Courtney Joslin
Dan Kahan
Jeffrey Kahn
Brian Kalt
Sam Kamin
Michael Kang
Chimène Keitner
Alicia Kelly
Orin Kerr
Nancy Kim
Heidi Kitrosser
Adam Kolber
Russell Korobkin
Alex Kreit
Anita S. Krishnakumar
Susan Kuo
Greg Lastowka
Sarah Lawsky
Youngjae Lee
Margaret Lewis
Erik Lillquist
Jeff Lipshaw
Jonathan Lipson
Jacqueline Lipton
Matthew Lister
Joseph Liu
Michael Madison
Tayyab Mahmud
Kevin Noble Maillard
Solangel Maldonado
Jason Mazzone
Linda McClain
William McGeveran
Salil Mehra
Carrie Menkel-Meadow
Max Minzner
Viva Moffat
Scott Moss
Eric Muller
Janai Nelson
Jaya Ramji-Nogales
Helen Norton
Elizabeth Nowicki
Paul Ohm
Angela Onwuachi-Willing
David Opderback
David Orentlicher
Michael O'Shea
Kristen Osenga
Mary-Rose Papandrea
Rafael Pardo
Marcy Peek
Eduardo Peñalver
Robert Percival
Michael J. Pitts
Marc Poirier
David Post
Amanda Pustilnik
Shruti Rana
Geoffrey Rapp
William Reynolds
Neil Richards
Lori Ringhand
Alice Ristroph
Marc Roark
Brishen Rogers
Sasha Romanosky
Tuan Samahon
Susan Scafidi
David Schleicher
David Schraub
Paul Secunda
Lea Shaver
Jonathan Siegel
Jessica Silbey
Peter Smith
Judd Sneirson
Adam Steinman
Charles Sullivan
Rick Swedloff
Peter Swire
Olivier Sylvain
Steph Tai
Andrew Taslitz
Robert Tsai
Jenia Turner
Joseph Turow
Steve Vladeck
Ari Waldman
Spencer Weber Waller
Howard Wasserman
Melissa Waters
Elizabeth A. Wilson
Frank Wu
Alfred Yen
Corey Yung
David Zaring
Timothy Zick
Michael Zimmer
Jonathan Zittrain

Ownership

Concurring Opinions is a
general-interest legal blog
operated by Concurring
Opinions LLC, a Pennsylvania
Limited Liability Corporation.

Blogroll

Above the Law
Access to Justice
ACS Blog
Althouse
Balkinization
Becker-Posner Blog
BlackProf
BoingBoing
Chicago Law Faculty Blog
Conglomerate
CrimLaw
Crime & Federalism
CrimProf Blog
Crooked Timber
Derechoalderecho
Discourse.net
Dorf on Law
Election Law
Emergent Chaos
The Faculty Lounge
Feminist Law Profs
43(B)log
Freakonomics Blog
Freedom to Tinker
Google Blogoscoped
How Appealing
Ideoblog
Info/Law
Instapundit.com
Juris Novus
Jurisdynamics
Just Books
Law and Humanities Blog
Law and Letters
Law Librarian Blog
Legal Profession Blog
Legal Theory Blog
Legal Times Blog
Leiter Reports
Brian Leiter's Law School Reports
Lessig Blog
Madisonian Theory
Media Law Blog
Mirror of Justice
The Moderate Voice
National Security Advisors
Opinio Juris
Point of Law
PrawfsBlawg
Privacy and Security Training
ProfessorBainbridge.com
Property Prof Blog
Red Tape Chronicles
The Right Coast
Schneier on Security
SCOTUSBlog
Security Dilemmas
Sentencing Law and Policy
Simple Justice
Sivacracy.net
The Situationist
Susan Crawford
TalkLeft
Talking Points Memo
TaxProf Blog
TeachPrivacy Blog
Tech & Marketing Law
Truth on the Market
Volokh Conspiracy
WorkPlace Prof Blog
WSJ Law Blog
Wonkette
The Yin Blog


© Concurring Opinions

Powered by WordPress