Site Meter

Category: Law Practice


Brian Tamanaha’s Straw Men (Overview)

(Cross posted from Brian Leiter’s Law School Reports)

Brian Tamanaha previously told Inside Higher Education that our research only looked at average earnings premiums and did not consider the low end of the distribution.  Dylan Matthews at the Washington Post reported that Professor Tamanaha’s description of our research was “false”. 

In his latest post, Professor Tamanaha combines interesting critiques with some not very interesting errors and claims that are not supported by data.   Responding to his blog post is a little tricky as his ongoing edits rendered it something of a moving target.  While we’re happy with improvements, a PDF of the version to which we are responding is available here just so we all know what page we’re on.

Stephen Diamond explains why Tamanaha apparently changed his post: Ted Seto and Eric Rasmusen expressed concerns about Tamanaha’s use of ad hominem attacks.

Some of Tamanaha’s new errors are surprising, because they come after an email exchange with him in which we addressed them.  For example, Tamanaha’s description of our approach to ability sorting constitutes a gross misreading of our research.  Tamanaha also references the wrong chart for earnings premium trends and misinterprets confidence intervals.  And his description of our present value calculations is way off the mark.

Here are some quick bullet point responses, with details below in subsequent posts:

  • Forecasting and Backfilling
    • Using more historical data from SIPP would likely have introduced continuity and other methodological problems
    • Using more years of data is as likely to increase the historical earnings premium as to reduce it
    • If pre-1996 historical data finds lower earnings premiums, that may suggest a long term upward trend and could mean that our estimates of flat future earnings premiums are too conservative and the premium estimates should be higher
    • The earnings premium in the future is just as likely to be higher as it is to be lower than it was in 1996-2011
    • In the future, the earnings premium would have to be lower by **85 percent** for an investment in law school to destroy economic value at the median
  • Data sufficiency
    • 16 years of data is more than is used in similar studies to establish a baseline.  This includes studies Tamanaha cited and praised in his book.
    • Our data includes both peaks and troughs in the cycle.  Across the cycle, law graduates earn substantially more than bachelor’s.
  • Tamanaha’s errors and misreading
    • We control for ability sorting and selection using extensive controls for socio-economic, academic, and demographic characteristics
    • This substantially reduces our earnings premium estimates
    • Any lingering ability sorting and selection is likely offset by response bias in SIPP, topcoding, and other problems that cut in the opposite direction
    • Tamanaha references the wrong chart for earnings premium trends and misinterprets confidence intervals
    • Tamanaha is confused about present value, opportunity cost, and discounting
    • Our in-school earnings are based on data, but, in any event, “correcting” to zero would not meaningfully change our conclusions
  • Tamanaha’s best line
    • “Let me also confirm that [Simkovic & McIntyre’s] study is far more sophisticated than my admittedly crude efforts.”

Nonrespondent law graduates and other sampling questions

The Washington Post reports one possible concern with estimates of the Economic Value of a Law Degree:

“[Paul] Campos argues that low-earning lawyers may be less likely to participate in SIPP in the first place because of the stigma involved in admitting that, even anonymously.”

By email, Jerry Organ asks related questions about the representativeness of our sample.

“SIPP” is the United States Census Bureau’s Survey of Income and Program Participation, and is one of the primary data sources used in The Economic Value of a Law Degree.  Campos worries about stigma and non-response.  Thankfully SIPP is specifically designed to deal with these problems and to include impoverished and stigmatized members of the population, including those who receive government aid.

The Census Bureau explains SIPP’s purpose as follows:

 “To collect source and amount of income, labor force information, program participation and eligibility data, and general demographic characteristics to measure the effectiveness of existing federal, state, and local programs; to estimate future costs and coverage for government programs, such as food stamps; and to provide improved statistics on the distribution of income and measures of economic well-being in the country.”

The Census Bureau elaborates on the use of SIPP to analyze participation in Food Stamps and other anti-poverty programs here.

Census explains in greater detail how SIPP handles issues related to response bias, non-response bias, and weighting here.  SIPP oversamples in poor neighborhoods, imputes when necessary, and adjust the sample weights to approach a nationally representative sample.

It is about a good a survey as one is likely to find conducted by people who care a great deal about nonresponse and accurate estimates.

Additionally, to the extent that any lingering nonresponse bias may cause those with low earnings to be less inclined to participate, this bias will affect both law graduates and bachelor’s degree holders.  What we measure in the Economic Value of a Law Degree is the earnings premium, or difference in earnings that is attributable to the law degree.  The biases should wash out, or more likely, bias down our estimates of the law degree earnings premium, because bachelors are far more likely than law graduates to live in poverty.

Indeed studies that have compared earnings reported in SIPP to earnings from administrative data (tax and social security administration data) find that SIPP data underestimates earnings premiums because more highly educated and higher income individuals tend to underreport earnings, while less educated and lower income individuals tend to over report.  We make no attempt to correct for this downward bias in our earnings premium estimates to offset any lingering selection on unobservables.

Individual response bias issues also won’t affect federal student loan default data, which is administrative data from the Department of Education.  As noted in the article and in previous blog posts, former law students default on their student loans much less frequently than former students of bachelor’s degree or other graduate degree programs.



Snapshot Of The Legal Market

Like a blog barnacle, I continue to hang on to the good ship Concurring Opinions.  At least for another month.  Thank you for inviting me to stay on.

Today, an observation about the legal market (and a plug for a friend’s start up). Len Gray is a former Latham & Watkins associate who, prior to law school, worked as a headhunter in Atlanta.  Even so, Len was turned off by legal headhunters, whom he regarded as too aggressive and often insensitive to finding the right fit. Read More


ReInvent Law! How Technology and New Business Models Are Affecting Legal Practice

Anyone interested in where legal practice may beheaded should check out ReInvent Law Silicon Valley 2013 on March 8 at teh Computer History Museum in Mountain View, CA (disclosure I am a speaker). The conference is devoted to law, technology, innovation, and entrepreneurship in the legal services industry. Dan Katz gave and excellent talk at the mid-year AALS conference. He talked about how automated system, machine learning, and more are defeating outsourcing and changing the face of legal practice. I nodded as what he said mapped to what I learned while I was at Google. In 2008 I started writing about problems with the structure of legal education. Those issues are now with us in full force. I think Dan and this project get to issues within the legal industry that may make the what about firm jobs question obsolete (which it may already be for a host of reasons) but present opportunities going forward.

Here is how he sums up the idea:

At all price points, the legal services market is rapidly changing and this disruption represents peril & possibility. This meeting is about the possibility … about the game changers who are already building the future of this industry. This is a 1 day event featuring 40 speakers in a high energy format with specific emphasis on technology, innovation and entrepreneurship. It will inspire you to consider all of the possibilities.

In that Silicon Valley way, it will be a blitz of 40 speakers covering LegalTechStartUp, Lawyer Regulation, Business of Law, Quantitative Legal Prediction, Design, 3D Printing, Driverless Cars, Legal Education, Legal Information Engineering, New Business Models, Lean Lawyering, Legal Supply Chain, Project Management, Technology Aided Access to Justice, Augmented Reality, Legal Process Outsourcing, Big Data, New Markets for Law, Virtual Law Practice, Information Visualization, E-Discovery, Legal Entrepreneurship, Legal Automation … and much more.

Tickets are Free but registration is required.
Please feel free to sign up today.


The Ghost of Louis Brandeis on How to Teach Law School

Hello again Co-Op! I’m happy to be back for a short guest-blogging stint that was, er, supposed to start in January but Danielle graciously allowed me to postpone into February. I’m hoping to make up for the radio silence in the last couple of weeks of the month. Anyway, without further adieu, today’s topic: Over at Prawfsblawg, a vibrant debate is going on about the perennial subject of how to best teach law school. There’s a lot of good things to be said on both sides of the that debate. I’d like to call attention in particular to the comment by Ray Campbell, which is devoid of the absolutes that tend to abound in this area. I’ve expressed my own thoughts on this topic during previous go-rounds here and here and here.

But by “perennial,” I meant that this debate is really ancient. It far pre-dates the recent financial crisis and downturn in the legal market. It pre-dates the Carnegie Report in 2007. It pre-dates the MacCrate Report in 1992. It pre-dates the 1921 Carnegie Report. Indeed, it pre-dates most law schools altogether. Benjamin Spencer’s recent article on the skills vs. doctrine debate — which includes the question of who would be the best teachers for whatever it is the students should be learning — shows that it goes back to the 1870s, and an ABA Report that concluded that the existing method of study — one taught mainly by professors with substantial practice experience — was “too brief for useful purposes,” and that the schools were inviting “unfit” and unprepared students to fill their seats, were giving “examinations, which are such only in name,” and were allowing “degrees [to be] thrown away on the undeserving and the ignorant.”

I was reminded of the length of time these sorts of discussions have been going on when I recently stumbled across a letter from the man pictured above, Louis Brandeis, to Dean Christopher Columbus Langdell of Harvard Law School. Langdell, of course, is possibly the single person most responsible for the form of legal education we have today. It was his idea at Harvard to replace classes taught by practicing lawyers with classes taught by academic law professors, hired soon after graduation after perhaps only a short judicial clerkship, and to extend the length of the program from eighteen months to three years. In particular, it was Langdell’s idea to teach law as a science, devoted to learning the general principles that pervade the law as revealed in cases, but not necessarily constituting the law of any particular jurisdiction. That is, Harvard would focus on a generalized notion of tort law, contracts law, etc., one that had the advantage, as Charles Whitebread used to say about the Model Penal Code, of being equally the law nowhere.

Brandeis was a product of that model. He graduated from Harvard Law School in 1878, eight years after Langdell had started reforming Harvard and the first year the program was extended to three years. But a little more than ten years later he thought substantial alterations should be made to the curriculum. Brandeis worried, in effect, that Harvard Law students were not learning enough actual law:

To Christopher Columbus Langdell

December 30, 1889 Boston, Mass.

My Dear Prof. Langdell: My experience as one of the examiners for admission to the Suffolk bar has impressed upon me the importance of adding to the instruction at the School a thorough course on the peculiarities of Massachusetts law. I am aware that the introduction of such a course involves apparently a departure from the present policy of the School, but my experience and observation have convinced me that such a course would increase the usefullness as well as the membership of the School, and I therefore venture to submit to you with some detail my views of the proposed course, and the reasons which induce me to advocate it. Read More


Justice, Law, and Fellowship: From Coordination to Collaboration

“True peace is not the absence of tension: it is the presence of justice and brotherhood.” — Martin Luther King, Jr. (1955, 1958, 1961)

At the Martin Luther King Jr. Memorial, Washington DC

Dr. King spoke these words or similar ones on a number of occasions, usually when explaining the relationship between love, law, and civil disobedience. I invoke them here because of their affinity with the idea that law that successfully promotes the common good will not yield simply the absence of anarchy but the presence of fellowship.

In the first major chapter of Normative Jurisprudence, “Revitalizing Natural Law”, Robin West argues for “a reengagement of liberal and progressive lawyers with … the ethical inquiry into the nature of the common good furthered by just law.” This is a terrific project. But it is a more complicated project than either a casual reader or a sophisticated scholar might notice. There are at least two major kinds of complexity involved. One, to which West devotes some attention in the chapter, involves how to specify human good, common or individual. The other, which receives less attention, at least at this phase of the book, involves figuring out what is distinctively legal about a project to promote the common good. In this post, a bit about this second area of complexity. This is not to say that West herself does not appreciate the complexity of and need for sorting out the role of law in a quest for the common good.

West persuasively explains that just because the project of promoting the common good might also be a political one or an overall ethical one, that does not mean it is not also a legal one, a distinctively legal one, or one in which law plays a distinctive role. Throughout “Revitalizing Natural Law”, West emphasizes that achieving the common good, understood as arising from the demands of individual good, necessitates coordinated social action, of the sort law is uniquely positioned to bring about.

Individuals going it alone will not get very far in achieving their own good, notes West. A group of uncoordinated individuals who realize this problem need state-sponsored coordination, in the form of law, to ensure that each of them do better, which means that all of them will do better. Fair enough, as far as it goes. But there is a lot more to coordination, and to coordination implemented by law, than meets the eye.

“Coordination” can be understand more or less thickly. A law dictating whether to drive on the left or the right coordinates thinly. It solves a problem whose solution does not impact the good in question: keeping traffic flowing. The content of the law does not matter, what matters is having one. The activities and instrumentalities involved are understood, practically speaking, largely similarly by all the participants.

Most of the time, though, there is a thicker connection between laws governing collective action or social activity and the content of the laws themselves. Laws against polluting the environment presuppose or stipulate agreement on foundational matters, including what constitutes pollution and how to demarcate the polluters from the environment. Laws regulating research on human subjects presuppose or stipulate agreement on what is research, who is human, and what it means to be a subject of another’s study.

To approach jurisprudence as West urges means noticing and taking quite seriously the role law and legal institutions – all of them, not just legislatures, but courts and agencies and review boards and prosecutors and juries and so on – play in coordinating both the understanding and the lived actuality of the activities and instruments law references. The good is rich stuff, and to get us to it, law must make it possible for us to proceed from strategic interaction in a coordinated setting (e.g. driving on the highway) to substantive cooperation (e.g. creating a functional and legitimate banking system). That sort of cooperation rests on shared background understandings of matters basic, diverse, and particularistic. To enable such cooperation law must not only invite and permit, but also foster, collaboration on a worldview sufficiently shared so that law has a shared meaning for law makers, law appliers, law enforcers, and law abiders (not that these four actors are always distinct and separate).

The flight from ethical normativity that West identifies in Normative Jurisprudence is part of a larger flight from normativity in general – including the normativity of meaning. How much agreement on meaning do we need in order to achieve just law that furthers the common good? What sort of legal actors and institutions do we need to get that agreement? In future posts during this celebration of Normative Jurisprudence, I will continue to examine these questions. I take inquiry into them to be part of the project West urges. I also expect that there will be sharp disagreement among liberal and progressive scholars about how much shared meaning we need and what we are willing to do get it.


Beyond Open: Cultures of Challenge

One of the people I follow on Google+ posted the TED Talk by Margaret Heffernan. Her desire for better training on how to challenge authority is laudable, but I think misses that corporate and other institutional cultures often squash and punish those who speak out. Her basic point is strong: seeking out those who will challenge your views and avoiding echo chambers is the best way to ensure your ideas are solid. Dr. Heffernan tells about a researcher who managed to stand up to established medical practice and change it. She tells of a colleague who managed to voice concern at his biotech company who was worried about a new product but afraid to challenge the status quo. When he did, he found that others shared his belief. He was a hero whistleblower of sorts. The later example is quite rare. Just think of Enron and the host of other debacles. I agree that it takes courage to challenge, but a corporate culture that does not punish free thinkers is important too. As the literature on scenario planning shows, sustaining a group that is permitted to think about and challenge company goals is quite difficult. And that is for a group designed to advance corporate profits. Finding room those who would, out of loyalty to a company, ask questions about plans is a deeper problem. The current focus on teamwork, loyalty, execution, speed, and results no matter what the consequences, means that lip service to openness, out of the box thinking, and pick any other management cliche you want, rule so much that it is no surprise that 85% of managers fear speaking up as Dr. Heffernan notes. So I praise the idea, but think in addition to training people to challenge, we need to build a culture of questioning.


Stanford Law Review Online: Regulating Through Habeas

Stanford Law Review

The Stanford Law Review Online has just published a Note by Doug Lieb entitled Regulating Through Habeas: A Bad Incentive for Bad Lawyers?. The author discusses the potential pitfalls in a pending DOJ rule that provides for fast-track review of a state’s death row habeas petitions if the state implements certain sanctions for lawyers found to be legally ineffective:

The most important—and most heavily criticized—provisions of the Antiterrorism and Effective Death Penalty Act restricted federal courts’ ability to hear habeas petitions and grant relief to prisoners. But the 1996 law also included another procedural reform, now tucked away in a less-traveled corner of the federal habeas statute. It enables a state to receive fast-track review of its death row prisoners’ federal habeas petitions if the U.S. Attorney General certifies that the state provides capital prisoners with competent counsel in state postconviction proceedings.

Now, a pending Department of Justice (DOJ) rule sets forth extensive criteria for states’ certification for fast-track review. Piggybacking on a federal statute that does the same, the proposed DOJ rule encourages states to adopt a seemingly commonsense measure to weed out bad lawyers: if an attorney has been found legally ineffective, remove him or her from the list of qualified counsel eligible for appointment. Unfortunately, such removal provisions may do more harm than good by jeopardizing the interests of ineffective lawyers’ former clients. This Note explains why removal provisions can be counterproductive, argues that rewarding the implementation of these provisions with fast-track habeas review is especially unwise, and offers a few recommendations.

He concludes:

The lesson, at a minimum, is that policymakers should be wary of one-off regulatory interventions into indigent defense, considering the hydraulic pressure that a new requirement might exert elsewhere in the system. Leaders within the public defense bar might also wish to think carefully about their expressions of support for ineffective-attorney-removal provisions. And, while some scholars have considered the ethical obligations of predecessor counsel when faced with an ineffectiveness claim, rigorous empirical study of lawyers’ actual responses to allegations of ineffectiveness may be needed to develop sound policy. Do most attorneys actually understand themselves to owe continuing duties to former clients, or do most do what they can to protect their professional reputations against charges of deficient performance? (And are those with the latter attitude more likely to be ineffective in the first place?) The practical effect of regulatory interventions, including removal provisions, turns on the answer to these questions.

None of this is to suggest that it’s in any way acceptable for an ineffective lawyer, let alone an incorrigibly awful one, to represent a capital—or non-capital—defendant or prisoner. The point is the opposite. Even a well-intentioned patchwork of regulation through habeas is no substitute for an adequately funded system that trains, compensates, and screens counsel appropriately. If kicking ineffective lawyers off the list may do more harm than good, the goal should be keep them off the list to begin with.

Read the full article, Regulating Through Habeas: A Bad Incentive for Bad Lawyers? by Doug Lieb, at the Stanford Law Review Online.


Charles C. Burlingham

I just finished a biography of Charles C. Burlingham, who was a remarkable lawyer and civic activist in New York during the first half of the twentieth century.  Although the book wasn’t well-written (and I can’t find a good picture of him to post here), I was eager to read about Burlingham.  He keeps popping up in my research, even though he held public office only briefly on the New York City Board of Education.  Here are some of his accomplishments:

1.  He defended White Star against the admiralty suits that followed the sinking of the Titanic.

2.  He was influential in getting Learned Hand appointed to the Federal District Court in New York.

3.  He almost single-handedly convinced local politicians to nominate Benjamin Cardozo for his first judgeship.

4.  He corresponded regularly with Felix Frankfurter and Franklin D. Roosevelt about all sorts of issues.

5.  He was one of Fiorello LaGuardia’s closest (albeit informal) advisors when LaGuardia was the Mayor.

6.  He lived to 101.  (Seems remarkable to me.)

I find his story interesting, in part, because there are so few great practicioner-statesmen left.  Lloyd Cutler was probably the last one, though perhaps I’m overlooking someone.

UPDATE:  Here’s a great anecdote.  When Burlingham was 100, he went to an awards dinner and sat with Learned Hand, Felix Frankfurter, and Earl Warren.  A young lawyer drove him home and asked “Did you have a good time?”  Burlingham replied:  “Yes, but I had hoped to meet some new people.”