TY - GEN
T1 - BudgetFIx
T2 - 13th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2014
AU - Tran-Thanh, Long
AU - Huynh, Trung Dong
AU - Rosenfeld, Avi
AU - Ramchun, Sarvapali
AU - Jennings, Nicholas R.
N1 - Publisher Copyright:
Copyright © 2014, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
PY - 2014
Y1 - 2014
N2 - Crowdsourcing is a multi-agent task allocation paradigm that involves up to millions of workers, of varying reliability and availability, performing large numbers of micro-tasks. A key challenge is to crowdsource, at minimal cost and with predictable accuracy, complex tasks that involve different types of interdependent micro-tasks structured into complex workflows. In this paper, we propose the first crowdsourcing algorithm that solves this problem. Our algorithm, called BudgetFix, determines the number of interdependent micro-tasks and the price to pay for each task given budget constraints. Moreover, BudgetFix provides quality guarantees on the accuracy of the output of each phase of a given workflow. BudgetFix is empirically evaluated on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and is shown that BudgetFix can provide similar accuracy, compared to the state-of-the-art algorithm for this workflow, but is on average 32% cheaper.
AB - Crowdsourcing is a multi-agent task allocation paradigm that involves up to millions of workers, of varying reliability and availability, performing large numbers of micro-tasks. A key challenge is to crowdsource, at minimal cost and with predictable accuracy, complex tasks that involve different types of interdependent micro-tasks structured into complex workflows. In this paper, we propose the first crowdsourcing algorithm that solves this problem. Our algorithm, called BudgetFix, determines the number of interdependent micro-tasks and the price to pay for each task given budget constraints. Moreover, BudgetFix provides quality guarantees on the accuracy of the output of each phase of a given workflow. BudgetFix is empirically evaluated on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and is shown that BudgetFix can provide similar accuracy, compared to the state-of-the-art algorithm for this workflow, but is on average 32% cheaper.
KW - Budgeted task allocation
KW - Crowdsourcing
KW - Find-fix-verify
KW - Performance guarantees
UR - https://www.scopus.com/pages/publications/84910155070
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
T3 - 13th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2014
SP - 477
EP - 484
BT - 13th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2014
PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Y2 - 5 May 2014 through 9 May 2014
ER -