- Concurring Opinions - http://www.concurringopinions.com -
Posted By Dave Hoffman On December 19, 2012 @ 9:56 am In Law and Psychology | 6 Comments
Like many others, I’ve been using Amazon Mechanical Turk to recruit subjects for law & psychology experiments. Turk is (i) cheap; (ii) fast; (iii) easy to use; and (iv) not controlled by the psychology department’s guardians. Better yet, the literature to date has found  that Turkers are more representative of the general population than you’d expect — and certainly better than college undergrads! Unfortunately, this post  at the Monkey Cage provides a data point in the contrary direction:
“On Election Day, we asked 565 Amazon Mechanical Turk (MTurk) workers to take a brief survey on vote choice, ideology and demographics. . . . We compare MTurk workers on Election Day to actual election results and exit polling. The survey paid $0.05 and had seven questions: gender, age, education, income, state of residence, vote choice, and ideology. Overall, 73% of these MTurk workers voted for Obama, 15% for Romney, and 12% for “Other.” This is skewed in expected ways, matching the stereotypical image of online IT workers as liberal—or possibly libertarian since 12% voted for a third party in 2012, compared to 1.6% percent of all voters. . . In sum, the MTurk sample is younger, more male, poorer, and more highly educated than Americans generally. This matches the image of who you might think would be online doing computer tasks for a small amount of money…”
Food for thought. What’s strange is that every sample of Turkers I’ve dealt with is older & more female than the general population. Might it be that Turk workers who responded to a survey on election habits aren’t like the Turk population at large? Probably so, but that doesn’t make me copacetic.
Article printed from Concurring Opinions: http://www.concurringopinions.com
URL to article: http://www.concurringopinions.com/archives/2012/12/unrepresentative-turkers.html
URLs in this post:
 found: http://dl.dropbox.com/u/7536991/Mechanical_Turk.pdf
 post: http://themonkeycage.org/blog/2012/12/19/how-representative-are-amazon-mechanical-turk-workers/
 : http://www.plosone.org/article/info:doi/10.1371/journal.pone.0051876
 : http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2138624
 : http://jme.bmj.com/content/early/2012/11/29/medethics-2012-100798.short?g=w_jme_ahead_tab
Copyright © 2010 Concurring Opinions. All rights reserved.