Limitations of Technology and “Right Size” Challenges
posted by Miriam Cherry
Last week, I read that a hiker had come across the remains of adventurer Steve Fossett’s airplane near Mammoth Lakes, California (full story here). What was unique in the search for the downed plane last year was the nature and extent of the search and rescue effort. Thousands of people on their computers joined together to scour pictures taken by aerial photography in hopes of saving Fossett or at least finding the crash site. This distributed search network was part of Amazon’s Mechanical Turk – which breaks down massive tasks (like scanning through thousands of aerial photographs) into small portions that can be performed by thousands of individuals, taking up only a few minutes of their time.
Professor Yochai Benkler has been writing about these types of collaborative networks, most recently in his book The Wealth of Networks, and I’ve recently been thinking a lot about them too, especially what they might mean for the future of work (and accordingly, traditional labor and employment law doctrine). But before getting too excited about these new forms of distributed collaboration, I have to sound a note of caution.
After all, the Mechanical Turk searching didn’t end up finding Steve Fossett’s plane last year. Was it hubris to think that new technology could solve the problem of search and rescue? Well, perhaps it was one of those intractable problems – according to this account:
“rugged mountainous, tree-covered terrain gave … less than a 10 percent probability of detecting debris from the wreckage during aerial fly-overs. . .. The fact that a large portion of the small aircraft was fabric-covered and that the aircraft quite likely burned on impact leaving very little exposed fabric or metal, also made it harder to find.”
This doesn’t mean that the Mechanical Turk/distributed work technology isn’t useful, or that this type of search and rescue effort won’t be successful in the future. But perhaps this was too ambitious a task. My co-author Rob Rogers and I asked similar questions when I was writing about a different kind of collaborative knowledge gathering tool – prediction markets. Picking the “right size” challenge may be an important part of promoting these new technologies – instead of choosing problems that would be a wonderful advance, but which might be likely to fail. Which types of problems could be benefited from a distributed networking solution? Which will get them? Which will assist the development of the technology, and which may hurt it? In the meantime, RIP, Steve Fossett, who strove to explore new frontiers – even in death.
October 11, 2008 at 5:29 pm
Posted in: Technology
Print This Post
Responses (5)
A.J. Sutter - October 12, 2008 at 10:29 am
Prof. Cherry, I’ve taken a quick look at your paper, and I’m wondering why do you want to encourage the existence of prediction markets?
And what sorts of problems do you think they will solve? What are some problems in the past that you think might have been solved faster or better using the “knowledge-gathering” capabilities of such markets?
When you consider the financial markets, and their supposed ability to gather information about prices, etc., why do you think the current financial crisis took so long to happen? Or, just to take an example, why did the CDO/mortgage-backed security market not crash long ago? Knowledge about how sub-prime loans were being generated was hardly secret.
(I’m also curious about how you determined, for purposes of your paper, that the creation of markets is a matter of “random walk”. What are the random variables, and how are you quantifying them? A random walk is a very specific kind of statistical behavior. E.g., pace the ghost of Louis Bachelier, financial markets are not random-walk based, as pointed out about 40 years ago by Benoit Mandelbrot — a fact subsequently ignored by many, to their ruin. But at least in Bachelier’s case, the variables, price and time, were clear.)
Miriam Cherry - October 12, 2008 at 7:16 pm
Hi, AJ,
Thanks for your comment and for reading my article, there’s a lot here to respond to! Prediction markets could be used in any number of contexts – many companies use them internally for sales forecasts, they’re currently working on predicting the outcome of the upcoming election, and there are many that operate in the area of science & other inventions. As they pertain to law, I’d like to see them used to improve administrative agency decisions, predicting what the Supreme Court might decide.
Right now, as we all react to the subprime meltdown and its broader economic consequences, it’s a lot more fashion to be a market skeptic than supporter. We’ve seen bubbles before – whether in the context of real esate, the dot com takeoff, or in Dutch tulip bulbs.
Prediction markets, like any type of markets, are subject to many constraints – factors that may cause them to excel – or to fail. I talk about these factors in a different article, 100 Nw. U. L. Rev. 1141, 1163-67 (2006), available here http://papers.ssrn.com/sol3/papers.cfm?abstract_id=927502.
We used the “random walk” as a metaphor/analogy to discuss the somewhat haphazard development of these markets described in the article.
Best, MC
A.J. Sutter - October 13, 2008 at 12:28 am
Thanks for your reply. As for the “fashion” of being a market skeptic, it’s one whose vogue is long overdue. Unfortunately, law & economics scholarship ignores many of the more analytical criticisms of the efficacy of markets, even those advanced by neoclassical economists, such as Gerard Debreu and Nicholas Georgescu-Roegen, among others, decades ago. Debreu, BTW, got a Nobel for his contributions to proving the First and Second Welfare Theorems, which to date form the firmest analytical foundation economists have for asserting the superiority of markets in allocating resources; it is less firm than generally supposed.
One of the issues has to do with the notion of aggregation of individual preferences — a process all those nice, smooth, unlabeled demand curves in econ (and nowadays law) books rely on. Information markets rely on this too, to the extent that they “organize and aggregate individual knowledge into a collective result” (your Supreme Court paper @ 1143.) The mathematical justification for this aggregation is very shaky. The collection Agreement on Demand: Consumer Theory in the Tewntieth Century edited by Mirowski and Hands has some good essays on the Sonnenschein-Mantel-Debreu theorem, and on Gerard Debreu’s misgivings about aggregation.
Another issue is with the efficient markets hypothesis more generally; you rely on this in your Supreme Court information market paper. For a critique of EMH, some places to start might be Mandelbrot & Hudson’s The Misbehavior of Markets: A Fractal View of Risk, Ruin & Reward, and, for a more mathematical analysis, the introduction and Chapter E20 of Mandelbrot’s Fractals and Scaling in Finance. For a general appreciation of Mandelbrot’s impact on economics, see, e.g., Chapters 11 and 12 of Mirowski’s The Effortless Economy of Science?. Of course, had there been an expert market for economic theories, Mandelbrot, an IBM mathematician most famous for his characterization of “fractals” and for his coinage of that term, might have been considered a “sheep” as you describe in the section of your paper starting at p. 1169.
You also mention the “Hollywood Stock Exchange” in your paper, as a prediction market about blockbusters (as distinct from Oscars predictions). You might be interested to check out the book by Art De Vany, Hollywood Economics: How extreme uncertainty shapes the film industry to see why the EMH doesn’t apply to the statistical behavior of that setting, either. Also, I was puzzled by your paper’s reference to using information markets “to predict insolvency of financial institutions” (@1158). I thought that’s what financial markets, and the “efficient market hypothesis,” should have done this time. Similarly, comments like “Having each person trade independently of others reduces the herding and groupthink instincts and insulates the market from information cascades” (@1165) must today strike even you as a bit rosy, don’t they?
It’s all too typical of the L&E approach not to engage with the math of the models it invokes; your breezy use of “random walk” as a metaphor is another example. But unless you really understand the limitations of the assumptions that underlie those models, you’re just relying on ideology when you promote the use of markets — and, worse, you’re encouraging people to use that ideology as a basis for policy.
A.J. Sutter - October 13, 2008 at 9:24 am
PS: on re-reading, I see my most recent comment might come across as a bit impatient or “aggravated”, as we used to say on Long Island. My apologies; your papers aren’t by any means the only, or most egregious, examples of the attitude toward markets that I critique.
Miriam Cherry - October 13, 2008 at 11:44 am
Hi, AJ,
We’re both coming at this from different directions and starting premises; I looked for some of your work to understand your perspective, but as I don’t know your institutional affiliation and as “AJ” may be a nickname or a pseudonym, I didn’t have any luck. Since this discussion has more to do with the details or how markets work (or don’t), and little to do with the original topic I posted on, I’ll write you off blog to discuss.
Best regards, MC
Leave a Reply