Site Meter

Setting the Bar, and the Limits of Empirical Research

Dave Hoffman

Dave Hoffman is the Murray Shusterman Professor of Transactional and Business Law at Temple Law School. He specializes in law and psychology, contracts, and quantitative analysis of civil procedure. He currently teaches contracts, civil procedure, corporations, and law and economics.

You may also like...

8 Responses

  1. John Armstrong says:

    My big problem with Ribstein’s position is where he states

    Lawyers can be certified by private organizations, including existing bar associations, which can compete with each other by earning reputations for reliability

    So, if I read this correctly he’s saying there would be a number of independent groups willing to attest to the quality of lawyers, and they would gain reputations for reliability and trustworthiness in their assessments. I’m sure they would gain such reputations within the legal community, but I have a feeling quite a large number of people pick a lawyer from outside that community, and in times of rather dire need and stress. As it stands, any practicing lawyer is attested to have some basic level of competance, and a distraught prospective client can at least have that assurance.

  2. Seth R. says:

    I don’t have time to comment on the entire post. But one comment on Ribstein’s seeming proposition that deregulation would strike a blow against biglaw …

    Biglaw is exactly the segment of the Bar that would be least affected by deregulation.

    Can you imagine trying to practice family law as a solo practitioner with the added problem of having to reduce your fees to compete with all the local bargain basement offerings?

    Family lawyers already don’t make a lot of money and struggle to stay in business. Turning the whole practice area into glorified paralegal work would absolutely annihilate the family law solo practitioner. The work would be taken over instead, by larger commercial “divorce mills.” Same with immigration law, same with bankruptcy law, same with criminal law.

    I guess whether you prefer talking to your friend’s personal lawyer about your marital problems over calling 1-800-DIVORCE is a matter of personal preference.

    But biglaw would be largely immune to this. They’re clients tend to have the financial resources to pay for “the best.” Exxon Mobile isn’t about to drop Holland and Hart for representation of its Wyoming oil interests simply because Congress decides to “deregulate the bar.”

    But plenty of Main Street Americans would drop their local solo bankruptcy attorney in favor of a cheaper online “debt counselor” financed by credit card companies.

    As usual, calls for deregulation take aim at the fat cats with a bazooka, miss entirely, and annihilate the small bystanders.

  3. Ted says:

    Seth’s comment is effectively saying that deregulation won’t affect those who have specialized skills in high demand, while will hurt those who are able to charge above-market rates to consumers without offering anything special that a paralegal couldn’t do. I fail to see why that’s a bug, rather than a feature.

  4. Dave, you are surely right that measures of quality will be imperfect. But I am not quite willing to surrender the utility of empirical analysis here. One might imagine that consumer complaints or formal disciplinary actions might be relevant measures, malpractice insurance rates, and there could certainly be others. I recall a paper by Muris and McChesney which used (among other things) direct measures such as dollar amounts in child support awards (they were comparing lawyers who did and did not advertise). My point is not to say that any of these measures, and I am sure they are others which are superior to those I mention here as examples, are perfect. To the extent that your point was that empirical analysis of quality is tough, Im on board.

    But, building on the some of the earlier comments and Wilson’s theory that it is the individual consumer will be hurt, perhaps it would be useful to look at transactions like real estate closings, divorce settlements, etc. These are the types of transactions where it is theorized consumers of legal services will be harmed. I believe a study, even one using imperfect quality measures, exploiting the natural experiments in UPL laws would provide some useful meat for policy analysis.

    I should also be clear that I have not completely surveyed the literature on the relationship between licensure and quality, but a survey I came across in an Enyclopedia of Law and Economics, at http://www.econlib.org/library/Enc/OccupationalLicensing.html states the following: “Most of the evidence on this issue, however, suggests that licensing has, at best, a neutral effect on quality and may even harm consumers.”

  5. Dan Spevak says:

    What is up with Dave Hoffman’s insistance to measure everything scientifically or empirically? Some things don’t lend themselves to being measured, as the example with the surgeon illustrates: there are too many factors and variables. Same for lawyer “quality.” What exactly is that? Isn’t that sort of like an empirical measurement of beauty? Well, Anjelina Jolie has full sexy lips which we all would love to kiss, but Jennifer Anistor has that personableness to which, at the end of the day, or at the end of the sex, is would be just so great to relax next to… Some consumers value cheap lawyers, and for those lawyers offering lowest fees are of best quality; other consumers want personal attention, so lawyers who return calls regularly are best; yet other consumers want to see their enemies bleed, so most vicious lawyers are best. Eye of the beholder, anyone? Measuring quality involves making the choice about what is valued, and the components going into that choice will dictate your results, even if you had the perfect ‘emprical test tube’ for gathering and analyzing the relevant data. The whole point is that you cannot delineate the scope of relevant factors without sacrificing objectivity, empiricism, and scientificism, so pleez, let’s not pretend that the exercise is anything but a change in form without any change in substance. To change substance, the only experiment that will tell you what consumers truly prefer, and what the market truly will bear is a wholesome deregulation of every hoop that one must jump through for the “honor” to practice law. This includes not only getting rid of the bar, but getting rid of law schools as prerequisites for entering the profession. My 8-year old son, frankly, can copy and paste the contracts clauses I spend 14 hours a day copying and pasting, and I’d really he rather not spend money on 3 years of bs and torture, crowned by the thorn to the profession that is the bar exam. I wonder if there is even a single person out there who would willingly submit that the bar exam was personally useful, and a good measure of legal skill or promise.

    You want an experiment? Here’s one: ask practicing attorneys what requirements imposed on them they found valuable in the practice of law. Please include those lawyers who started off not at law schools, but as apprentices. Also, ask every consumer of family law whether they’d be willing to forego the “quality controls” of the regulated bar in favor of actual competition and yes, clinics staffed by non-attorneys who don’t have to charge prices high enough to recoup the absurd costs of legal education. As for the comment regarding the decrease in quality, and the fear that ‘debt attorneys will be bought up by credit card companies,’ any practitioner who knows anything about consumer protection laws knows this will not happen so long as there are fee shifting statutes allowing the plaintiffs’ attorneys to retain their independence. You want quality for the poor? Write to your congressman and urge more fee shifting statutes passed in a variety of other contexts. Or, you could write a fee shifting statute onto a legal malpractice, how’s that for direct regulation of quality?

    As for family law, frankly, if it can be done cheaper (by improving competition and abolishing the monopoly on lawyers doing dumb paperwork), then it should be done cheaper. I am sickened by the snobbery of our profession, perhaps even more then by our pretenses. If I could rule by fiat, I’d ban the cartel that our profession is run by.

  6. Dave Hoffman says:

    Josh says: “consumer complaints or formal disciplinary actions might be relevant measures, malpractice insurance rates, and there could certainly be others. I recall a paper by Muris and McChesney which used (among other things) direct measures such as dollar amounts in child support awards (they were comparing lawyers who did and did not advertise).”

    I know you don’t mean to be exclusive, but all of these measures raise immediate objections.

    Consumer complaints and disciplinary actions are extremely weak proxies for quality. In the current system, the likelihood of such grassroots generated action is vanishingly small. Unlike, say, citizen review boards over police departments, the obscurity and timing problems of legal malpractice make it unlikely that complaints are going to be sufficiently nonrandom to measure much besides lawyer personality.

    Malpractice rates as a quality proxy have been tried with respect to doctors, and the sense I have from the literature is that insurers don’t spend the money monitoring and thus do not target rates well to care levels.

    Child support awards – like win rates in criminal trials – are useful (if at all, see post) for those small set of lawyers who litigate in court.

    I agree with Josh that this isn’t an impossible project. But I think I’m more skeptical than he that it is solvable in the near future. Thus, Dan Spevak, I think you’ve misread me.

  7. Dave, I meant neither to be exclusive, nor necessarily endorse or criticize these measures. I certainly agree with you that they are imperfect. That said, any empirical evidence would be a welcome addition and represent an improvement over hand waving and mere speculation as to future effects, a point upon which I think we both agree.

  8. shamu says:

    can one design an empirical project that measures both litigation and transactional practice quality as a function of licensing?

    well, i suppose that in addition to the numerical measures already suggested (verdicts, malpractice judgments, etc.), one could conduct exit surveys of consumers of a particluar legal service (e.g., divorce or bankruptcy), to determine reasons, expectations, and satisfaction levels for a client’s selecting of and experience with a legal services provider (lawyer or non). this would essentially be a qualitative measure, and may capture the one quality-control measure currently operating in the legal market: “reputation,” a measure on which the us news law school rankings heavily depend, and on which chambers & partners has ranked lawyers & firms. perhaps research as to the quality of non-licensed entities would lead to an exception to the bar exam, which i agree creates unnecessary barriers and costs to practicing, without corresponding benefits of quality control. nobody practices law the way they take bar exams — answering questions from memory, as opposed to researching them.