The ABA’s Ugly Table Fetish
posted by Dave Hoffman
The ABA's Proposed Table Is About As Ugly As This One
TaxProf reports on the ABA’s possible move toward more substantively revealing employment statistics. Key to the change would be a table that each law school would have to post on its website. The Table would list salaries and employment of law school graduates, and break it down into quartile percentiles.
What’s useful about this kind of disclosure is that it avoids the problem of misleading means and medians in salary– itself caused by missing data. That is, law schools aren’t able to collect salary information for each graduate. When schools calculate mean/median salaries, they exclude missing graduates, and thereby overemphasize the importance of high earners. As the relevant committee explained:
“Schools receive salary information from a fairly small percentage of graduates. Graduates reporting their salaries are skewed towards those earning the most . . . A school that touts median salary information, without appropriate qualifiers, is misleading prospective students. We propose that all salary information clearly indicate the number of respondents and percentage of all graduates . . . We would not require schools to disclose any salary information for a given category unless there are at least five respondents.”
To the extent you think that law schools are bad actors, and that the market won’t motivate disclosure, this is a good reform. I have a different perspective: it is a bad idea for an accreditation agency to micromanage the internal workings of law school business, especially when the data is then connected to the machinations of a deeply flawed, secretive and corrupting ranking magazine. I have particular doubts about disclosure of salary data. It seems to me that this is a classic example of a liberal policy that might have unintended consequences – salary collusion between market makers in the entry level job market. It’s also the case, as I am about to discuss in a Conglomerate Masters Forum, that the ABA’s meddling in the internal affairs of the schools can be seen as (yet another) attempt to increase the price of legal education & resulting consumer costs, while protecting incumbents. To be clear: law schools should be pressured to disclose more about outcomes (and inputs). But when that pressure comes from an accrediting agency that happens to be a guild, you have to worry about what’s happening behind the scenes.
However, let’s pretend like this exact disclosure requirement is welfare maximizing. How should it proceed? The ABA wants to mandate that schools produce & display a very, very ugly table. But, for Tufte’s sake, why? All of the information in that table could be better displayed in figures – Henderson’s bimodal distribution (using # of graduates on the y-axis), and then a few bar charts displaying where people work. I am quite confident that this is possible because I am putting the finishing touches on Temple’s self-study accreditation document, which liberally uses figures – instead of tables – to display data on employment, revenues, debt, and admissions. As Epstein, Martin and Boyd explained, Tables are ugly and are terrible tools to communicate data, especially summary statistics. If the ABA thinks that communicating this data is important, it should mandate that the data be presented in a clear way, not in an ominous, busy, 11 (!) column Table that no one, ever, will read.
Shucks, it is as if the ABA wants to pretend to care about disclosure, so as to maintain its accreditation monopoly, but to implement sunshine in a form that effectively destroys its utility.
March 20, 2011 at 2:17 pm
Posted in: Law School, Law School (Teaching)
Print This Post
Responses (9)
Gato Points - March 21, 2011 at 4:26 am
I really need to read these post before I forward them to everyone. Just reading the headline and seeing the picture I though the story was that ABA was literally changing their decor. Like the TSA, I should’ve checked more before I jumped to my conclusion: http://jonathanturley.org/2011/03/20/irs-to-conduct-abortion-audits/#comment-214159
Apologies…
Gato Points - March 21, 2011 at 4:26 am
http://lawblog.legalmatch.com/2011/03/14/why-going-through-airport-security-is-still-better-than-being-muslim-in-america/
Sorry wrong link…
James Grimmelmann - March 21, 2011 at 6:29 am
I agree that the table is ugly. But it is a standard format. If every law school fills it out in this form, then it becomes much easier for anyone to take the raw data and create good visualizations of it. It’s better to put the simple informational mandate on the law schools, and leave it up to the wider “market” of interested people to come up with the compelling visualizations that make it meaningful.
Even better would be if the ABA required each law school to publish the data using a standard XML schema, and then just made available to everyone the XML file containing each year’s information from the tables.
dave hoffman - March 21, 2011 at 7:38 am
James,
But that seems precisely wrong to me. Creating a disclosure standard for lots of private firms that requires (or assumes) the existence of some other third party intermediary to be useful is bad policy. Let the firms compete on how to best display the information – if you want – and let prospective applicants thereby understand how important salary and employment outcomes are for each school.
Also, why these particular slices of outcomes (quartiles? dividing between firms based on # of attorneys at the firms)? If the ABA really wants to meddle, why not force schools to survey all grads at a year and ask if they are happy with their jobs and if they make enough to cover expenses and debt. That would be just as meaningful. Which is to say, not very.
anon - March 21, 2011 at 10:36 am
“Let the firms compete on how to best display the information – if you want – and let prospective applicants thereby understand how important salary and employment outcomes are for each school.”
Sorry, I don’t follow. This is precisely the reason the ABA is talking about disclosure. Because law schools have no incentive to provide adequate disclosures.
Dave Hoffman - March 21, 2011 at 1:47 pm
Anon,
1. I don’t know why you think law schools have “no incentive to provide adequate disclosures.” Aren’t they competing for applicants? Assuming that the market is moving toward higher demand for information like this, I think supply would increase too.
2. But even if that weren’t true, my point is that law schools can compete on how to disclose, with the ABA providing a baseline of the substantive content and a requirement of fair display. The requirement, perversely, makes it all but impossible to understand the material because it is buried in an enormous, multi-factorial, enormous table. Who does that help?
anon - March 21, 2011 at 2:02 pm
1. Yes, law schools are competing for applicants That is why the incentive is to distort numbers to make the school look better to applicants.
2. Complexity is not a virtue when it comes to reporting, but neither is utter simplicity. Both can skew reality. Why not figure out a clear set of guidelines that provide information prospective students are interested in, and do it in a precise a clear way? Of course, some people will complain, but at least prospective students will have, in theory, a more accurate picture of each school.
James Grimmelmann - March 22, 2011 at 12:05 am
Dave, it’s only your final paragraph that I have an issue with: my argument is that if disclosure is to be mandated, than it should be raw-data disclosure. Let the law schools compete on the pretty pictures as well if they want, but raw data is the necessary baseline to enable third parties to get into the visualization and presentation game.
Government open data efforts are demonstrating the wisdom of the “If you build it, they will come” approach. See the Princeton Government Data and the Invisible Hand paper. Given scarce resources, it’s better to provide standardized data than polished visualizations, because it acts as an input to an open innovative ecosystem.
Dave Hoffman - March 22, 2011 at 9:18 am
James,
Totally fair, and I think I agree with you. The ABA could fairly require disclosure of an xml file to some central authority. It could also require schools to display the data on their websites, in whatever form the school deemed appropriate, so long as it was not misleading or obscuring.
The point is really that tables like these are the least useful form of disclosure (to no disclosure). The seem like they are precise and clear, but actually they are hard to read and often get ignored.
Leave a Reply