Creating a large benchmark for open information extraction

Gabriel Stanovsky, Ido Dagan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

100 Scopus citations

Abstract

Open information extraction (Open IE) was presented as an unrestricted variant of traditional information extraction. It has been gaining substantial attention, manifested by a large number of automatic Open IE extractors and downstream applications. In spite of this broad attention, the Open IE task definition has been lacking - there are no formal guidelines and no large scale gold standard annotation. Subsequently, the various implementations of Open IE resorted to small scale post-hoc evaluations, inhibiting an objective and reproducible cross-system comparison. In this work, we develop a methodology that leverages the recent QA-SRL annotation to create a first independent and large scale Open IE annotation,1 and use it to automatically compare the most prominent Open IE systems.

Original languageEnglish
Title of host publicationEMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings
PublisherAssociation for Computational Linguistics (ACL)
Pages2300-2305
Number of pages6
ISBN (Electronic)9781945626258
DOIs
StatePublished - 2016
Event2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016 - Austin, United States
Duration: 1 Nov 20165 Nov 2016

Publication series

NameEMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings

Conference

Conference2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016
Country/TerritoryUnited States
CityAustin
Period1/11/165/11/16

Bibliographical note

Publisher Copyright:
© 2016 Association for Computational Linguistics

Funding

We would like to thank Mausam for fruitful discussions, and the anonymous reviewers for their helpful comments. This work was supported in part by grants from the MAGNET program of the Israeli Office of the Chief Scientist (OCS), the Israel Science Foundation grant 880/12, and the German Research Foundation through the German-Israeli Project Cooperation (DIP, grant DA 1600/1-1). This work was supported in part by grants from the MAGNET program of the Israeli Office of the Chief Scientist (OCS), the Israel Science Foundation grant 880/12, and the German Research Foundation through the German-Israeli Project Cooperation (DIP, grant DA 1600/1-1).

FundersFunder number
DIPDA 1600/1-1
German-Israeli Project Cooperation
Chief Scientist Office
Deutsche Forschungsgemeinschaft
German-Israeli Foundation for Scientific Research and Development
Israel Science Foundation880/12
Office of the Chief Scientist, Ministry of Economy

    Fingerprint

    Dive into the research topics of 'Creating a large benchmark for open information extraction'. Together they form a unique fingerprint.

    Cite this