Working the Crowd
I’ve written before about crowdsourcing and its accompanying employment law issues. And so when I saw this recent article from the Economist, I thought that it encapsulated both the potential of virtual work as well as its downside. The article notes that psychologists are using the Mechanical Turk to broaden their psychological experiments so that they are truly global and representative of more than the perspectives of western college students. Sounds great! Er, well. Knowing how hard it is to earn minimum wage on the Mechanical Turk (because my research assistant and I have tried to earn it, and failed to do so on numerous occasions), it becomes a darker story. Here are some excerpts from the article:
Crowdsourcing is a way to get jobs like deciphering images, ranking websites and answering surveys done for money by online workers. Several firms offer the service, including oDesk, CrowdFlower and Elance. But by far the most popular for scientific purposes is Mechanical Turk, which is run by Amazon and is named after an 18th-century chess-playing machine in which a human secretly moved the pieces. Mechanical Turk has more than 500,000 people, known as Turkers, in its workforce. For the hard-pressed, cash-strapped psychologist, this is a godsend. Turkers, despite the fact that half of them have at least one degree, are willing to work for peanuts. (Their median wage is about $1.40 an hour.) Most, indeed, seem to regard the tasks they are set as more like a paying hobby than an actual job. And, crucially, they are growing more cosmopolitan with each passing year. Though 40% are still from America, a third are Indian and the rest come from about 100 other countries. . . .using Turkers instead of undergrads does offer some genuine diversity.
One researcher who has taken advantage of that diversity is David Rand, a lecturer in psychology at Harvard University. He is using Mechanical Turk to reconsider the results of several experiments originally conducted mainly on students. In a recent study of moral decision-making, for example, he recruited hundreds of Turkers to repeat a classic thought experiment known as the trolley problem. This confronts its participants with a dilemma—a runaway railway trolley will kill a group of people unless the subject of the study chooses to push a single individual in front of it, in order to slow it down. Doing so will kill that individual, so the dilemma is whether to kill one person deliberately, or several through inaction.… Dr Rand is re-running Dr Herrmann’s experiments on Mechanical Turk—at a tenth of the cost of the original work….
Questions of ethics have also arisen. Some people think research projects which pay wages of less than $2 an hour are exploitative—even though that is the going rate for other Turker activities. Conversely, according to Karen Fort, of France’s Institute of Scientific and Technical Information, at least one university has already prohibited the use of grant funds for this sort of study, for fear that Turkers could claim status as employees. For many researchers, though, the appeals of crowdsourcing—bargain prices, vast supply and enormous scale—are too attractive to ignore. Indeed, the new methodology might democratise the very practice of psychology, allowing those without a laboratory or university behind them to join in as well.
I have a very strong view on this. I believe that in its current form, the relations between workers and the psychologists described here are exploitative. Beyond that, there is a strong argument that employment on these terms violates the Fair Labor Standards Act. Researchers and their universities should be ashamed. It is ironic that someone ostensibly investigating ethics around the world cannot do a better job when designing his own experiment. Further questions: How is this getting past university IRBs? Has anyone at the Department of Labor taken notice of this? And where are the enterprising wage-and-hour labor attorneys? (The spokewoman for France’s Institute of Scientific and Technical Information has a point, in my view).
[Hat-tip: Jeffrey Harrison]