CoRefi: A Crowd Sourcing Suite for Coreference Annotation

Aaron Bornstein; Arie Cattan; Ido Dagan

CoRefi: A Crowd Sourcing Suite for Coreference Annotation

Aaron Bornstein, Arie Cattan, Ido Dagan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Scopus citations

Abstract

Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present COREFI, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, COREFI provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. COREFI is open source and directly embeds into any website, including popular crowdsourcing platforms. COREFI Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi

Original language	English
Title of host publication	EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations
Editors	Qun Liu, David Schlangen
Publisher	Association for Computational Linguistics (ACL)
Pages	205-215
Number of pages	11
ISBN (Electronic)	9781952148620
State	Published - 2020
Event	2020 System Demonstrations of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020 - Virtual, Online Duration: 16 Nov 2020 → 20 Nov 2020

Publication series

Name	EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations

Conference

Conference	2020 System Demonstrations of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020
City	Virtual, Online
Period	16/11/20 → 20/11/20

Bibliographical note

Publisher Copyright:
© 2020 Association for Computational Linguistics.

Funding

The work described herein was supported in part by grants from Intel Labs, Facebook, the Israel Science Foundation grant 1951/17, the Israeli Ministry of Science and Technology and the German Research Foundation through the German-Israeli Project Cooperation (DIP, grant DA 1600/1-1).

Funders	Funder number
DIP	DA 1600/1-1
German-Israeli Project Cooperation
Intel Labs
Deutsche Forschungsgemeinschaft
Israel Science Foundation	1951/17
Ministry of science and technology, Israel

Cite this

Bornstein, A., Cattan, A., & Dagan, I. (2020). CoRefi: A Crowd Sourcing Suite for Coreference Annotation. In Q. Liu, & D. Schlangen (Eds.), EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations (pp. 205-215). (EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations). Association for Computational Linguistics (ACL).

Bornstein, Aaron ; Cattan, Arie ; Dagan, Ido. / CoRefi : A Crowd Sourcing Suite for Coreference Annotation. EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations. editor / Qun Liu ; David Schlangen. Association for Computational Linguistics (ACL), 2020. pp. 205-215 (EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations).

@inproceedings{347e3588580b4464b22257ff06fa3a28,

title = "CoRefi: A Crowd Sourcing Suite for Coreference Annotation",

abstract = "Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present COREFI, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, COREFI provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. COREFI is open source and directly embeds into any website, including popular crowdsourcing platforms. COREFI Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi",

author = "Aaron Bornstein and Arie Cattan and Ido Dagan",

note = "Publisher Copyright: {\textcopyright} 2020 Association for Computational Linguistics.; 2020 System Demonstrations of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020 ; Conference date: 16-11-2020 Through 20-11-2020",

year = "2020",

language = "אנגלית",

series = "EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations",

publisher = "Association for Computational Linguistics (ACL)",

pages = "205--215",

editor = "Qun Liu and David Schlangen",

booktitle = "EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations",

address = "ארצות הברית",

}

Bornstein, A, Cattan, A & Dagan, I 2020, CoRefi: A Crowd Sourcing Suite for Coreference Annotation. in Q Liu & D Schlangen (eds), EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations. EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations, Association for Computational Linguistics (ACL), pp. 205-215, 2020 System Demonstrations of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Virtual, Online, 16/11/20.

CoRefi: A Crowd Sourcing Suite for Coreference Annotation. / Bornstein, Aaron; Cattan, Arie; Dagan, Ido.
EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations. ed. / Qun Liu; David Schlangen. Association for Computational Linguistics (ACL), 2020. p. 205-215 (EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CoRefi

T2 - 2020 System Demonstrations of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020

AU - Bornstein, Aaron

AU - Cattan, Arie

AU - Dagan, Ido

PY - 2020

Y1 - 2020

N2 - Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present COREFI, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, COREFI provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. COREFI is open source and directly embeds into any website, including popular crowdsourcing platforms. COREFI Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi

AB - Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present COREFI, a web-based coreference annotation suite, oriented for crowdsourcing. Beyond the core coreference annotation tool, COREFI provides guided onboarding for the task as well as a novel algorithm for a reviewing phase. COREFI is open source and directly embeds into any website, including popular crowdsourcing platforms. COREFI Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo: https://github.com/aribornstein/corefi

UR - http://www.scopus.com/inward/record.url?scp=85173719648&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85173719648

T3 - EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations

SP - 205

EP - 215

BT - EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations

A2 - Liu, Qun

A2 - Schlangen, David

PB - Association for Computational Linguistics (ACL)

Y2 - 16 November 2020 through 20 November 2020

ER -

Bornstein A, Cattan A, Dagan I. CoRefi: A Crowd Sourcing Suite for Coreference Annotation. In Liu Q, Schlangen D, editors, EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations. Association for Computational Linguistics (ACL). 2020. p. 205-215. (EMNLP 2020 - Conference on Empirical Methods in Natural Language Processing, Proceedings of Systems Demonstrations).

CoRefi: A Crowd Sourcing Suite for Coreference Annotation

Abstract

Publication series

Conference

Bibliographical note

Funding

Other files and links

Fingerprint

Cite this