Beyond the Turk: Alternative platforms for crowdsourcing behavioral research

Eyal Peer, Laura Brandimarte, Sonam Samat, Alessandro Acquisti

Research output: Contribution to journalArticlepeer-review

1749 Scopus citations


The success of Amazon Mechanical Turk (MTurk) as an online research platform has come at a price: MTurk has suffered from slowing rates of population replenishment, and growing participant non-naivety. Recently, a number of alternative platforms have emerged, offering capabilities similar to MTurk but providing access to new and more naïve populations. After surveying several options, we empirically examined two such platforms, CrowdFlower (CF) and Prolific Academic (ProA). In two studies, we found that participants on both platforms were more naïve and less dishonest compared to MTurk participants. Across the three platforms, CF provided the best response rate, but CF participants failed more attention-check questions and did not reproduce known effects replicated on ProA and MTurk. Moreover, ProA participants produced data quality that was higher than CF's and comparable to MTurk's. ProA and CF participants were also much more diverse than participants from MTurk.

Original languageEnglish
Pages (from-to)153-163
Number of pages11
JournalJournal of Experimental Social Psychology
StatePublished - 1 May 2017

Bibliographical note

Publisher Copyright:
© 2017 Elsevier Inc.


  • Amazon Mechanical Turk
  • CrowdFlower
  • Crowdsourcing
  • Data quality
  • Online research
  • Prolific Academic


Dive into the research topics of 'Beyond the Turk: Alternative platforms for crowdsourcing behavioral research'. Together they form a unique fingerprint.

Cite this