"But You Promised": Methods to Improve Crowd Engagement In Non-Ground Truth Tasks

Avshalom Elmalech, Barbara J. Grosz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Crowdsourcing platforms were initially designed to recruit people to perform tasks that were simple cognitively but difficult for computers. One challenge in these settings is to identify an incentive mechanism for motivating workers to complete tasks and do high-quality work. Previous research has studied the use of financial incentive mechanisms and social comparison as motivators. These mechanisms can only be applied to ground truth tasks, tasks for which there is an objective performance scale. In this paper, we define and compare three innovative methods for improving worker engagement on non-ground truth tasks drawing on a psychological theory of commitment. The three methods are similar in asking participants to promise they will complete a task, but they differ in terms of how the commitment is made. In the first method, participants commit by signing a contract; in the second, by listening to a recording; in the third, by recording a personal commitment. The last two methods significantly improved the task completion rate when compared to two baseline conditions. The methods we propose can be implemented simply, can be used for any task, and do not affect participants' behavior other than by improving their engagement.

Original languageEnglish
Title of host publicationProceedings of the 5th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2017
EditorsSteven Dow, Adam Tauman
PublisherAAAI press
Pages21-30
Number of pages10
ISBN (Electronic)9781577357933
StatePublished - 27 Oct 2017
Externally publishedYes
Event5th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2017 - Quebec City, Canada
Duration: 24 Oct 201726 Oct 2017

Publication series

NameProceedings of the 5th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2017

Conference

Conference5th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2017
Country/TerritoryCanada
CityQuebec City
Period24/10/1726/10/17

Bibliographical note

Publisher Copyright:
Copyright © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of '"But You Promised": Methods to Improve Crowd Engagement In Non-Ground Truth Tasks'. Together they form a unique fingerprint.

Cite this