Modeling to the Ignorant
posted by David Zaring
The very brilliant Matthew Stephenson has recently published a positive political theory paper on the choice for Congress between delegation to agencies and delegation to courts. He thinks that a rational Congress will delegate decisions to agencies if it wants to create a regime that will be ideologically consistent across issues but variable over time (he studiously avoids examples, but perhaps agricultural subsidies – which could be changed in the future but in the present Congress wants done in a particular way – are the idea). And that Congress will delegate decisions to courts if it wants temporal consistency but ideological variability (perhaps a regulation requiring the alternating, as opposed to direct, current, or maybe rules that affect long-term government contracts, are cases where Congress would roll the dice on the content of the choice – delegate it to the courts – but hope that whatever choice is made becomes a predictable precedent that future courts follow).
The second part of Stephenson’s paper, though, is, quite literally, Greek to me:
Modeling is upon us. You see it in the Journal of Empirical Legal Studies – though empiricism is conventionally thought to be a form of social science that might be practiced without sophisticated modeling – and now you see it in the Harvard Law Review. Stephenson develops eight such formulas in the formal proof portion of his piece on legislative delegations.
What is the upside of this sort of modeling for lawyers and political scientists? God knows the latter do plenty of it, but there’s an audience cost – I expect that most of the legal scholars sympathetic to positive political theory apply its insights without even being close to being able to model like McNollgasts can. And I’ve been at conferences where law professors assessed models more with fear and trembling than with confidence in the clarity of the modeled insights. I am, after all, semi-like Stephenson, an eager and promising administrative law scholar [ed. – hoo boy. Might be a good time to disable comments.]. But I doubt that Stephenson was writing part II of his paper for my benefit, or if he was, he certainly won’t enjoy my incisive critique of it – I simply skipped it.
So other than noting, gentle reader, that perhaps you ought to try to learn the basics of modeling if you expect to be reading a lot of legal scholarship in the future, I wonder if I might trouble those of you who do enjoy reading modeling pieces to recommend legal scholarship (I guess I’ll leave economists and mathematicians out for now – I’ll assume without deciding that their modeling presents different concerns than does modeling in public law subjects) in which the thesis was inexpressible without the model. I’d also like advice on meta-scholarship – that is, good resources on how to read models. In short, I’d like to hear advice for a inexperienced consumer of formally modeled scholarship. I suspect you’d be doing more of the Co-Op’s readers a service than you might think.
April 6, 2006 at 3:39 pm
Posted in: Articles and Books
Print This Post
Responses (9)
Frank - April 6, 2006 at 9:29 pm
Very interesting post. My instinct is to be very wary of this mathematicization, given that the variables and coefficients are often close to meaningless. How do we measure ideology? What’s a conservative spending decision, now that the Republicans have increased spending more in 5 years in office than Clinton did in 8? Is a decision to prohibit certain GMO biotechnology conservative (ala Kass) or liberal (ala the Greens)?
It’s no surprise that Farber’s response to Stephenson says “the empirical evidence does not support the model.” See
http://www.harvardlawreview.org/forum/issues/119/feb06/farber.shtml
But I doubt a group of political scientists could even find a way to code the evidence in order to see if the the theory was true or false. So perhaps it fails the first criterion of (Popperian) science: falsifiability.
In any event, I would hate to see legal scholarship dominated by “fancy math” just because it has the patina of rigor. This happened to poli sci for a while, with unfortunate consequences. Here are a few sources who share my insight/bias:
Shapiro & Green, The Pathologies of Rational Choice Theory (1994). (This was written in response to rational choice theorists’ aggressive colonization of political science departments. They won at Harvard, lost at Yale, and still have enormous influence in the profession.)
Ian Shapiro, The Flight from Reality in the Human Sciences (2005). (critiquing mystification-via-mathematicization)
George Steinmetz, ed., The Politics of Method in the Human Sciences: Positivism and Its Epistemological Others (2005) (wonderful collection of essays, some of which discuss the way mathematicization functions as a strategy for consolidating authority in the social construction of expertise)
Oz Shy, Introduction to Network Economics (wryly commenting on the lack of utility of a great deal of mathematical modeling in economics).
David Zaring - April 7, 2006 at 9:20 am
All stuff that’s new to me. And I think Vermeule’s comments on Stephenson’s piece were sorta along the so what lines – so perhaps count it in the non-falsifiable camp. Rat choice always struck me as powerful because it simplified so much – but modeled rat choice … that’s much less simple.
Nate Oman - April 7, 2006 at 2:25 pm
David & Frank: The point of the modeling is to show you the interaction between different variables. The fact that the some of the variables are difficult or impossible to quantify doesn’t make the model invalid or useless. Also, beware of mathphobia cloaked as deep methodological critique.
The real tragedy of Matthew Stephenson’s work is that such a great mind is being frittered away in the barren wasteland of administrative law.
David Zaring - April 7, 2006 at 2:49 pm
I’ll grant that there may be mathophobia – but that doesn’t mean that the audience, even the audience a law professor would want, isn’t affected. And I took Frank’s point, and the point of the critiques (Stephenson’s coming out with a response, btw) to be that stylistized relationships that don’t have predictive real world power aren’t very helpful.
But fine, fine … I’ll only mildly note that though some don’t like administrative law, for others, it’s the only kind of public law worth taking seriously. I imagine that latter group includes most political scientists, who have, for better or ill, never been fans of traditional constitutional or public international law.
Frank - April 8, 2006 at 11:28 am
Nate, just to be clear: if the variable is impossible to quantify, you still think it’s useful to model it?
Joshua Wright - April 8, 2006 at 12:25 pm
Frank, of course it is still useful to model it.
Let me expand on Nate’s point. Nate is right that one of the primary reasons to model is to identify relationships and their implications. These models predict how a change in one variable of interest impacts others, and sometimes tell us something about the magnitude of the effect. Sometimes we can test all of the relationships we would like because we have the data. Sometimes we dont have that data and can only test some of the hypothesis we can derive from the model. One of the main contributions of modeling, though, is that it forces one to be precise about identifying those relationships and the underlying assumptions driving those relationships. This is true even if we cannot “quantify” the variable.
Here is a boring example from my own research: I want to model competition of firms for shelf space and its impact on consumers. I need a model of consumer behavior. I believe, and am willing to assume, that consumers value product location on the shelf but dont know how to “quantify” this effect because I only observe whether the consumer purchases the product or does not. By assuming in my model that premium shelf space increases consumer willingness to pay (and openly stating as much), I have held this interesting part of the problem constant in a reasonable manner such that I can look at interesting relationships between manufacturer and retailer behavior. Perhaps I can even derive some previously unexplored relationships from my model that ARE testable with data. I realize shelf space is not everybody’s thing, but it should not be difficult to think of an example that does the trick for you. The point is that modeling might tell us something (maybe even something important) about certain observed behaviors and their impacts even though not every variable in the model is measurable in the sense you seem to believe is necessary.
What is more, modeling compels the researcher to explicitly set out the assumptions and forces that are driving those relationships. If the assumptions are unwarranted, or do not truly support the outcome predicted by the author, such errors can be exposed. Legal arguments have long included assumptions about both the empirical state of the world and the behavior of economic agents on variables of interest. Frequently, scholars build on these assumptions and make predictions about variables of interest and predicted effects. Increasing the level of precision in specifying those assumptions and deriving conditions under which certain relationships or results hold (or do not hold) seems like a good thing to me.
Of course, excessive formality, i.e. using a fancier model than necessary to make the point, may be counterproductive. A model should start simple, and add features as neccessary. There is a good deal in economic modeling that is not helpful and overly formal. But this is not properly construed as an attack on modeling itself.
David Zaring - April 9, 2006 at 11:13 am
Okay, that’s a really helpful comment. Though I still take it that a variable that can *never* be tested creates Frank’s falsifiability problem.
My concerns (directed at political scientists, rather than economists) are more prosiac. I could see someone concluding that the thesis “because of stare decisis, political insulation, and the hokum/magesty of judicial decisionmaking, delegating decisions to courts are difficult to control in the near term, but are stable in the long term” being, if anything, easier to consume than Stephenson’s modeled conclusion, and just as falsifiable. If not more so. I wonder if the modeling limited the sort of conclusions he could draw while also costing readers.
But I don’t want to overstate the concerns. I’m interested in learning more about modeling because that’s what the rat choice crowd does, for better or worse (better, if their success in the academy is any indication).
Tried to post this yesterday, but comments closed, maybe because of spam? Lest any of you think that I’m ever far from my trusty laptop.
geoff manne - April 9, 2006 at 1:37 pm
Frank: What methodology would you prefer — by what allegedly better means would you assess complex relationships? Do you believe that the imperfections of formal modeling (and, importantly, formal modelers) are so clearly worse than the imperfections inherent in other modes of analysis? Beware the Nirvana fallacy (you know — the belief that Kurt Cobain was actually better than mediocre. Kidding; I kid). Seriously — it is not enough to point out that option A is imperfect; one surely must also demonstrate that option B is actually preferable.
Joshua Wright - April 9, 2006 at 2:39 pm
“Though I still take it that a variable that can *never* be tested creates Frank’s falsifiability problem.”
David, a variable that can never be tested might create a falsifiability problem with respect to THAT variable. But, if we are looking at complex relationships out there in the world with many, many interactions — some observable and some not (*never* can be tested, lets pretend we would know that) — making assumptions about one allows me to more rigorously examine the rest of the relationships involved. This is a primary feature of modeling. I can get MORE testable implications out of the model by doing this.
I realize none of this answers your question, and I certainly dont know much about political science modeling. But, I could not resist defending modeling as a useful tool against Frank’s critique.
Leave a Reply