Gamble on Gambling
posted by Michael Abramowicz
Keith Jacks Gamble has written a super paper on the Super Bowl. After each play in the Super Bowl, he noted the “bid’ and “ask” prices for a share on Tradesports that would pay off $10 if and only if the Colts won the Super Bowl. He used the midpoint of those prices to derive probability estimates, making it possible to assess, at least based on the market’s wisdom, how each play made more or less likely a Colts victory.
Perhaps someday sports pages will routinely post graphs of trading prices to summarize the games of the day before — especially if annotated with plays (as in this graph of the Super Bowl on Midas Oracle), it’s an excellent way to get a quick overview of what happened and what mattered.
I’m a bit more skeptical, however, of one use that Gamble makes of the data. He uses the prices to assess individual player performances. For example, Devin Hester is given 10.25 percentage points for his opening kick return for a touchdown. With this methodology, Gamble concludes that Bears QB Rex Grossman contributed 36.5 percentage points to the Colts winning.
Certainly, Grossman had a very bad day. But there are at least two reasons to be skeptical of this approach as a general way of assessing player contributions: (1) The simultaneity problem. Football is a team sport, and it is difficult to disaggregate all the players’ contributions. My hunch has always been that behind a great quarterback is a great offensive line. (2) The anticipation problem. Estimates of a particular player’s ability is already impounded into market prices. If Tom Brady were leading a last minute drive, a market might assume that he’ll probably be successful because he’s Tom Brady, thus understating the extent of his contribution. A better approach may be to give credits based on how different events contribute to winning in general. (See the Protrade markets, for examples of this approach.)
Gamble is also doing some very interesting serious research about how bounded rationality in financial markets can cause cascade behavior. I’d be curious to know also how seemingly random factors affect trading patterns — in the Super Bowl, did trading volume or direction change depending on how funny the ads were?
For those who have no idea what the relation of this to law, you’ll have to wait until my book, Predictocracy: Market Mechanisms for Public and Private Decisionmaking, comes out in the fall. In the meantime, thanks very much to Dan Solove for inviting me to guest blog! I enjoyed my time here, and very much appreciated all of the thoughtful comments on my posts.
February 12, 2007 at 10:10 am
Posted in: Economic Analysis of Law
Print This Post
Responses (2)
Daniel J. Solove - February 13, 2007 at 3:57 pm
Michael — Thanks for the great visit and your terrific and thought-provoking posts!
Keith Jacks Gamble - February 13, 2007 at 6:46 pm
Hi Prof. Abramowicz,
Thank you for your interesting and insightful response to my analysis. I agree that the simultaneity problem and the anticipation problem are confounding factors in assessing a player’s contribution to his team’s chance of winning. I’d like to say more on these issues.
(1) The Simultaneity Problem. Certainly, football is a team sport, and thus I suspect that cleanly identifying a player’s sole contribution play-by-play to his team’s chance of winning may be impossible. Certainly the performance of the offensive line has a ton to do with a team’s success. But even with standard statistics for measuring a player’s performance these issues remain. Quarterbacks get credit for completions, passing yards, and throwing touchdowns even though the coach’s play calling, the line’s blocking, the receiver’s route running, ect have everything to do with creating those numbers. Attributing actions to the primary actors on a play is a natural way to compute statistics when faced with the simultaneity problem. At least my net probability points statistic provides some weight to the game situation when measuring a player’s impact on a given play. Certainly, a 3 yard run on 3rd and 2 at the opponent’s 4 yard line means more than a 3 yard run on 3rd and 10 at a team’s own 20. This difference isn’t captured in rushing yards, but it plays a major role in my numbers. The Protrade win probability measure which similarly takes account of the game situation, also has this simultaneity problem.
Also on this issue of the simultaneity problem, there is at least one case in which a player’s contribution to his team’s chance of winning can be measured more cleanly, unexpected injuries and suspensions. For example, when Kevin Garnett was suspended for one game after throwing a punch (click here for the story), the drop in the odds of his team winning provides a nice measure of his impact.
(2) The Anticipation Problem. Certainly, the market builds in estimates of a team’s and players’ abilities. After all, the market gave the Colts a 68% chance of winning before they even stepped on the field. Certainly, this number says something about the Colts performance that isn’t captured in my net probability points statistic for each player. If the market expects that the Payton-Manning-led Colts offense will move the ball easily, then this expectation will be built into prices. As a result, Peyton Manning’s performance will indeed be understated by my measure relative to the performance of, for example, backup Jim Sorgi had he done the exact same thing as Peyton. However, it’s not so clear that Peyton’s performance will be understated relative to the performance of the Colts other offensive players. The anticipation problem holds for them as well. If the Colts offense is expected to move the ball easily, then receiver Reggie Wayne’s performance will also be understated relative to his performance without these high expectations.
Interestingly, if the market anticipated that the Bears wouldn’t have much success in moving the ball on the Colts, then this anticipation problem actually understates how bad Rex Grossman’s performance actually was! Despite the market’s low expectations of how the Bears would fair against the Colts, Rex contributed 36.5% (percentage points) to the Colts chance of winning by my measure, more than twice as much as any Colt. Just imagine how bad this number would have looked had the Bears actually been expected to beat the Colts.
There’s also a simultaneity problem with interpreting the market’s anticipation of the future as a pure problem. The fact that the market price incorporates expectations of the future can actually be an advantage for my measure. Just as the market price incorporates expectations of Peyton’s offensive ability, it also incorporates expectations of how bad the defense he’s playing against is. If the market expects the Colts offense to have success in part because the Bears’ pass defense is so poor, then my measure more accurately measures his performance than a measure that doesn’t take into account expectations of the Bears’ defensive ability. Certainly, a measure of a player’s performance should incorporate expectations of the defense’s ability. The Protrade’s current publicly available win probability measure doesn’t take expectations about particular teams’ defensive abilities into account, thus it will overstate the performance of offensive players when playing against a particularly poor defense.
I am big fan of Protrade’s attempts to measure a player’s performance in terms of his impact on the probability of his team’s chance of winning. However, estimates that only take into account historical data, and not market expectations, are bound to be inaccurate. Making estimates only taking into account historical data is like driving while looking in the rear-view mirror. There’s no problem as long as the road ahead is just like the road gone by. However, if the future isn’t like the past, then wham. (analogy courtesy Robert Merton). For example, just think about how the various new offensive strategies (recently, the spread in college football) can change the game. The market has an amazing ability to take into account all the particulars of the situation in a way that historical data cannot. Thus, I think probability estimates must take into account market expectations to measure accurately the probability impact of a play, which is essential to both of our measures of a player’s performance. The research director at Protrade assures me that they are working internally on a measure that takes team ability into account. I hope that their methods will be made public so that interested folks can see and test for themselves the accuracy of their measure.
In closing, I’m happy to see these probability based statistics getting more attention because I think they provide an exciting way to look at the game. Tom Brady provides an excellent example of the potential of these statistics. It’s a common notion that Tom Brady is better than his traditional statistics indicate because he has produced for his team in the moments when it mattered most. I’d love to see what a market-based probability performance measure would have looked like for Tom Brady in his many magical playoff performances and in his most recent dud.
Again, thanks for the great comments.
Keith
Leave a Reply