Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Adam J. Berinsky, Gregory A. Huber, Gabriel Lenz
Political Analysis · 2012 · 4,150 citationsOpen access
No retraction on record in OpenAlex.Check Crossref now
Cite this paper
Berinsky, A. J., Huber, G. A., & Lenz, G. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20(3), 351–368. https://doi.org/10.1093/pan/mpr057
In text: (Berinsky et al., 2012)
Ask a question about this research
One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.
Abstract
We examine the trade-offs associated with using Amazon.com 's Mechanical Turk (MTurk) interface for subject recruitment. We first describe MTurk and its promise as a vehicle for performing low-cost and easy-to-field experiments. We then assess the internal and external validity of experiments performed using MTurk, employing a framework that can be used to evaluate other subject pools. We first investigate the characteristics of samples drawn from the MTurk population. We show that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples—the modal sample in published experimental political science—but less representative than subjects in Internet-based panels or national probability samples. Finally, we replicate important published experimental work using MTurk samples.
Related papers
- Rayyan—a web and mobile app for systematic reviews2016
- Amazon's Mechanical Turk2011
- Snowball Sampling: Problems and Techniques of Chain Referral Sampling1981
- Concurrent and Predictive Validity of a Self-reported Measure of Medication Adherence1986
- External review and validation of the Swedish national inpatient register2011
Metadata from OpenAlex (CC0). Citations are generated from the published record.