posted by Michael Kang
In my previous post, I argued that Michael Lewis’s influential bestseller Moneyball, widely cited in academia, ultimately relies too much on hyperbole to make its claims about the superiority of Billy Beane’s statistical methods in managing the Oakland Athletics baseball team. In this post, I assess the value of those Moneyball methods by examining the results of Oakland’s 2002 draft, a central event glamorized in the book as a showcase of Beane’s “scientific selection of amateur baseball players.”
This is a long, baseball-heavy post, so let me cut to the chase at the outset. Oakland’s Moneyball draft of 2002 was not the smash success that Lewis’s book forecasted. The seven years since the draft bear that out. Beane had seven first-round picks that year but drafted none of the eleven all-stars signed from the 2002 draft, despite exercising almost twenty percent of all first-round choices in the draft. In fact, only half the players on Beane’s wish list of the top twenty players in the draft ended up playing even a game in the major leagues. To tell a compelling story of Beane’s superiority, Lewis overstates what Cass Sunstein and Richard Thaler call the “blunders and the confusions of those who run baseball teams.” It turns out that other teams do a pretty good job of identifying talent too. In the 2002 draft, Oakland did not draft the best players available, while other teams using traditional methods did equally well or better.
So, what is the lesson for Moneyball? Lewis describes how Beane imported quantitative methods from the academic and financial worlds to identify assets, in this case baseball players, undervalued by the market. The modest notion that statistical methods can be invaluable in the search for these undervalued assets, particularly in an industry so obsessively numbers-oriented as baseball, is unassailable. But Lewis’s claims that Moneyball demonstrates the outright superiority of Beane’s quantitative methods in identifying the best talent are much more difficult to sustain, and Moneyball does not convincingly establish the backwardness of other teams, which had already begun erasing whatever advantages Oakland possessed by the time of Moneyball’s publication.
Instead, Moneyball demonstrates a slightly but importantly different lesson about Oakland’s successes during the early 2000s.
So, the familiar Moneyball theme of Beane outsmarting the league by finding great talents invisible to other teams clouded by their prejudices is largely overstated by Lewis. Not even the most committed Moneyball devotee would argue that the heroes of the book—Scott Hatteberg, Chad Bradford, Ricardo Rincon, Jeremy Brown—were star players. They were role players, and it’s in finding role players at a bargain where Moneyball was most useful. The advantage of Moneyball methods was not that they were so superior per se to traditional approaches in identifying top talent, or that traditional approaches were as wrongheaded as Lewis depicts. Moneyball, by offering a different playbook than other teams followed at the time, rarely identified star players with any more accuracy than traditional methods, but instead represented an alternate strategy that helped Oakland maximize the return from the pile of cheaper, marginal players that other teams did not covet. Moneyball was most valuable for finding cheaper players, not better ones. Finding cheaper is different from finding better, and Moneyball gave Oakland an advantage mainly at the former rather than the latter.
The best way to explain this lesson is by carefully assessing Oakland’s 2002 June amateur draft. Lewis documents this draft minute-by-minute in the book to illustrate Beane’s methods. And for good reason. Oakland held seven first-round picks (including sandwich picks), and seven of the first thirty-nine picks overall—a historic opportunity for Beane and his team. However, judged seven years later, Oakland’s results from the draft appear unremarkable. Among Oakland’s seven first-rounders, only three had more than a cup of coffee in the major leagues—Nick Swisher, Joe Blanton, and Mark Teahen—though none is too much more than an average player and all three were traded away by Oakland. As a Bay Area sportswriter summarized, “Basically, Moneyball happened, and it didn’t change the course of the A’s. Nice story. Good book. Not a lot of history made by those seven players.”
My point is not that Oakland’s 2002 draft was a failure, as some suggest, or decisively undercuts Moneyball as baseball philosophy. Neither is quite right in my view. But Beane and the Oakland braintrust, at least in Lewis’s account, dramatically overrate ex ante the success of their draft. Oakland’s draft is not just good in their view at the time, but “unfair,” nearly otherworldly. “It counted as one of the happiest days of Billy Beane’s career,” according to Lewis, and Beane warned “[t]his doesn’t happen. Don’t think this is normal.”
Thanks to the book, we have Beane’s internal ranking of the top twenty players available for the 2002 draft and can judge Beane’s skills at player evaluation with rare insight. Beane’s list, as documented by Lewis, were the “twenty players they’d draft in a perfect world” if “money were no object and twenty-nine other teams were not also vying to draft the best amateur players.” As a first cut, only ten of the twenty players on Beane’s list ever appeared in even a regular major league game. Although the major league draft is a “f_cking crapshoot” in Beane’s words, his list’s 50 percent success rate is less than the 60 percent rate for first rounders from 2000 to 2005. When compared to the first twenty players selected in the 2002 draft (as opposed to the entire first round), Beane’s list looks even worse by comparison. Ninety percent, eighteen of the first twenty picks in the 2002 draft reached the majors, including five all-stars.
To compare Beane’s acumen against the rest of the majors yet more precisely, we can focus on only the twelve players on his list who were not first-round picks of, nor likely considered first-round talents by other teams—Ben Fritz, Stephen Obenchain, John McCurdy, Jeremy Brown, Bill Murphy, Steve Stanley, John Baker, Mark Kiger, Brian Stavinsky, Shaun Larkin, Brant Colamarino, Mark Teahen. None of these players were ranked among Baseball America’s top forty prospects in the 2002 draft, but Beane staked his draft on them by drafting all but one of them. Mark Teahen was a nice find who has enjoyed a solid if unspectacular career with Kansas City. John Baker, who was released by Oakland, eventually caught on with Florida. But unfortunately for Oakland fans, Beane selected Fritz, Obenchain, McCurdy, and Brown with first-round picks, and took Murphy, Stanley, Kiger, Stavinsky and Colamarino later. Those nine players combined to appear in just twenty-three major league games total among them over eight seasons.
Lewis carelessly attributes the other teams’ disinterest in players like Brown and Colamarino, as two examples, to an irrational prejudice that they “do not look the way a young baseball player is meant to look.” But many teams weren’t interested for the simpler, less interesting reason that Brown and Colamarino just weren’t very good. In the end, Lewis was wrong to crow about how smart Beane was not to share other teams’ bias against Jeremy Brown’s lack of athleticism and defensive ineptitude—the very characteristics that killed his major-league prospects. Jeremy Brown, cited by Ian Ayres and Sunstein and Thaler among others as the poster child for Moneyball’s value, washed out of baseball not long after the book’s publication.
One aspect of Oakland’s 2002 draft truly was not “normal”—the Athletics took a college player with their first 23 picks and drafted 25 collegians among their first 28 picks. The 2002 draft turned out to be an unfortunate moment to be so strongly biased against high school players. Eight of the eleven players signed from the richly talented 2002 draft who were eventually selected as a major-league all-star came right out of high school: Zack Greinke, Prince Fielder, Scott Kazmir, Cole Hamels, Matt Cain, Jonathan Broxton, Brian McCann, and Josh Johnson. All but Johnson (fourth round) were taken in the first two rounds. Only three players signed out of college from the 2002 draft have been chosen as an all-star so far—Joe Saunders (first round), Curtis Granderson (third round), and Russell Martin (seventeenth round). It is worthwhile to point out as well that not a single one of these eleven all-star players was on Beane’s list of the top twenty players in the 2002 draft. As Beane admitted to Baseball Prospectus in a recent interview, “That list would look very different today.”
Oakland’s focus on college players was clearly by design, as Lewis colorfully explained by reference to Beane’s view of one particular high schooler in the draft. Lewis noted that Beane had no interest in Scott Kazmir, a high school pitcher coveted by Detroit and Milwaukee (though drafted by the New York Mets). As Lewis explained, “[P]icking a high school pitcher like Kazmir is exactly the sort of not-so-bright decision both franchises had a knack for making.” Yet Kazmir was one of six high school pitchers from that draft to become a major league all-star, as well as one of the first 2002 draftees to reach the majors.
The cost of dismissing high school players illustrates a tension in Lewis’s presentation of the Moneyball philosophy. Lewis explains that the Moneyball insight is that “when you rule out an entire class of people from doing a job simply by their appearance, you are less likely to find the best person for the job.” However, Oakland ruled out as a practical matter an entire class of prospects from consideration in the draft—high schoolers. As Lewis admitted, “[W]ith rare exceptions the new scouting directors toss all high school players immediately onto the dumping ground.” Oakland has backed away from this attitude since 2002, but it is revealing that the supposedly clear-eyed Moneyballers possessed a clear bias at the time that might have proved as costly to them as more familiar biases, criticized in the book, were for traditionally-oriented teams. Those teams correctly passed on busts like Jeremy Brown for their biases, while Beane’s own bias cost Oakland the chance to draft any of the eight all-stars who signed out of high school in the draft. In other words, it is not clear from Moneyball that Oakland managed to do a better job of identifying the best talent or was much less affected by inaccurate prejudices of its own.
Instead, the great value of Moneyball methods was not better prediction or fewer mistakes, but finding cheaper, though not necessarily better players, and making cheaper mistakes when they occur. Beane’s insight, which Lewis perhaps misidentifies, is not that Moneyball methods find more or better major league players than conventional approaches used by other teams. Each approach finds players that the other misses. And it isn’t clear whether Beane ended up finding more production among players that other teams ruled out, or whether other teams end up finding more production among players that he ruled out. Beane’s insight, instead, was to locate a different set of good players, and miss on a different set of good players, than most of the other teams do, and thereby avoid competing for the same talent and driving up his costs. Although the book emphasizes the supposed superiority of Beane’s methods, it’s the cost-effectiveness of asymmetry that is the more compelling point to take away from Moneyball.
In the end, Oakland’s decline on the field might be understood in part as a combination of two trends. First, as many note, other teams learned to co-opt the Moneyball playbook and negate some advantages Oakland enjoyed early on. But this is only a part of the story, and it is important to note that Oakland has gravitated nearly as much in the direction of traditional approaches since Moneyball’s publication.
Second, Moneyball methods proved somewhat limited over the longer run for Oakland. Moneyball has been insufficient help, at least thus far, for replacing the star players lost over time to free agency. If Moneyball provided a great advantage for finding top talent, Beane should have been able to draft replacements for Barry Zito, Miguel Tejada, Jason Giambi, and other departed all-stars. As Lewis explained, the 2002 draft was “the best chance [Oakland] might ever have to find several Barry Zito’s.” However, as effective as Moneyball proved for finding cheap complementary players like Hatteberg, Moneyball wasn’t as helpful for finding the next set of star players like Zito. Other teams, with their traditional methods, proved very likely to find those star players just as well as Oakland.
Lewis’s thesis of Moneyball superiority assumed the “blunders and the confusions” of other teams, but it turns out other teams aren’t usually so stupid—in fact, Kenny Williams, the White Sox general manager repeatedly mocked in the book, won a World Series only a couple years after the book’s publication. Oakland today still has nice role players discarded by other teams, such as Brad Ziegler and Justin Duchscherer, but it hasn’t been able so far to adapt Moneyball methods to outperform other teams in locating the critical new stars to lead them. Borrowing the words of one baseball scout, Moneyball helped Oakland find “a bunch of safe players,” but “it gets you a losing team” without the star players for them to support. Over the longer run, the greatest beneficiaries of Moneyball seem to be teams, like the Red Sox, that have nicely blended some Moneyball methods with traditional ones, rather than choosing one side or the other in what Lewis depicts too starkly as a baseball culture war.