Empirical Studies at ALEA
posted by Dave Hoffman
Bill Henderson (at the ELS Blog) has a very useful round-up of empirical papers presented at the recent ALEA conference. Blog-traveller Kate Litvak comes in for special praise:
Kate Litvak [presented] “The Effect of the Sarbanes-Oxley Act on Non-US Companies Listed in the U.S.,” which was an extremely well-done event study that used a natural experiment approach to capture the market reaction to SOX (it was generally negative). In the last couple of years, Kate, who does not have a PhD, has spent a lot of time learning sophisticated econometric techniques. It really showed. Very impressive (and easy to follow) presentation.
To be frank, I’ve been quite skeptical of studies showing a negative relationship between SOX and equity prices, on several grounds: (1) my practice experience managing the creation of event studies that dealt with changing legal regimes suggested that results are rarely as robust as one might hope; (2)) the passage and eventual implementation of SOX were so attenuated that event studies would seem hard to perform; and (3) the debate is quite politicized, with folks already disposed to dislike federalization of corporate law leading the charge on the empirical front as well. But, having read Kate’s paper, I’m inclined to rethink my position. It is well-worth a read.
May 8, 2006 at 10:42 pm
Posted in: Articles and Books, Conferences, Contract Law & Beyond, Corporate Law, Economic Analysis of Law, Empirical Analysis of Law, Law School, Securities
Print This Post
Responses (4)
Joshua Wright - May 9, 2006 at 1:35 am
Dave, while I agree whole heartedly that Kate’s paper is worth the read (not to mention carefully done), I am not sure if I completely follow point (3) above.
Do you have some empirical studies in mind that are examples of folks who are disinclined to federalize corporate law? I am NOT asking you to name them here, but your post seems to suggest that this is what you have in mind. Without naming them, have you read specific studies finding a negative relationship that are intellectually dishonest or otherwise motivated by a dislike for federalization? Maybe I am misreading you.
I ask because I take one of the great benefits of empirical work (and modeling for that matter) to be that the type of work you refer to can be exposed as flawed. This sort of exposure is a real contribution to any empirical debate. But a vague and anonymous reference to such dishonesty is not productive in my opinion. Anyway, if you have papers in mind, I think you would be doing a service to your readers and those following the empirical SOX debate to point them out and identifying where they go wrong.
Dave Hoffman - May 9, 2006 at 7:30 am
Josh,
Nuts, I half-feared that last point would get misinterpreted – the perils of blogging in haste! Thanks for the clarifying question.
Briefly, I simply haven’t read the relevant literature with enough care. I agree with you that the promise of empirical work is that dishonesty can be exposed, but the peril is that if you don’t have time to dig into the method, readers may be tempted to take relevant conclusions on faith (i.e., you look at results w/out looking into the black box). With respect to event studies of stock market behavior in particular, my view – based on practice experience, again, so anecdotal and weakly held – is that the technique produces results that are easy to “move.” Thus, I’m naturally “suspicious” (in the best sense of the word) of ES-based legal scholarship. I’d like to read it carefully and think about it before buying on.
This is especially true in the context of the SOX debate, where folks are predisposed to want to find confirming evidence. Research suggests the psychology behind this desire, and doesn’t require intellectual dishonesty at all.
But I haven’t read the papers carefully enough to know whether my skepticism is grounded in much. Indeed, Kate’s paper is the first I’ve really carefully looked at – the others I’ve in mind I just skimmed – and I was really impressed with the care in which she set up the analysis, her control of various factors, and the cleverness of the approach. I’ve now listed as a summer project the task of going back and re-reading the earlier work (instead of skimming it).
Sorry if I was unclear. I do think you are right that “vague and anonymous reference” to dishonesty is not helpful, and I really didn’t intend to make that type of swipe. But pointing out that ELS raises the barriers to inquiring into the strength of a claim is no sin.
Joshua Wright - May 9, 2006 at 9:44 am
Thanks for the clarification Dave. The point re: the increased costs of assessing empirical work is well taken. I wonder, though (mostly with tongue in cheek), if the predisposition to find confirming evidence and otherwise bolster one’s priors also impacts “other” (non-ELS) work in legal scholarship? Lastly, “moving” one’s results in response to the desire to obtain a particular set of results (say, a negative coefficient on a variable of interest), at least in my book, is dishonesty.
Dave Hoffman - May 10, 2006 at 2:29 pm
Josh, I agree that non-ELS work can be shoddy and dishonest, but that dishonesty is often, but not always, easier to see. Sure, folks can mistate the holding of a case, but the costs of checking up are negligible, and if you are half-way familiar with the area of law, likely the fraud is evident. By contrast, ELS work requires significant investment in looking at the method and the data itself. That is, I suppose we should expect higher amounts of legal-academic “fraud” in ELS papers than in, say, conlaw papers, if we could ever find a way to measure such a thing. This – of course – suggests that ELS papers published in peer reviewed journals ought to be taken more seriously, as peer review is a screen, of a sort, against methodological trickery.
Leave a Reply