SemEval-2018 Task 9: Hypernym Discovery

Jose Camacho-Collados; Claudio Delli Bovi; Luis Espinosa-Anke; Sergio Oramas; Tommaso Pasini; Enrico Santus; Vered Shwartz; Roberto Navigli; Horacio Saggion

doi:10.18653/v1/s18-1115

SemEval-2018 Task 9: Hypernym Discovery

Jose Camacho-Collados
, Claudio Delli Bovi
, Luis Espinosa-Anke
, Sergio Oramas
, Tommaso Pasini
, Enrico Santus
, Vered Shwartz
, Roberto Navigli
, Horacio Saggion

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

78 Scopus citations

Abstract

This paper describes the SemEval 2018 Shared Task on Hypernym Discovery. We put forward this task as a complementary benchmark for modeling hypernymy, a problem which has traditionally been cast as a binary classification task, taking a pair of candidate words as input. Instead, our reformulated task is defined as follows: given an input term, retrieve (or discover) its suitable hypernyms from a target corpus. We proposed five different subtasks covering three languages (English, Spanish, and Italian), and two specific domains of knowledge in English (Medical and Music). Participants were allowed to compete in any or all of the subtasks. Overall, a total of 11 teams participated, with a total of 39 different systems submitted through all subtasks. Data, results and further information about the task can be found at https://competitions.codalab.org/competitions/17119.

Original language	English
Title of host publication	NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop
Editors	Marianna Apidianaki, Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, Steven Bethard, Marine Carpuat
Publisher	Association for Computational Linguistics (ACL)
Pages	712-724
Number of pages	13
ISBN (Electronic)	9781948087209
DOIs	https://doi.org/10.18653/v1/s18-1115
State	Published - 2018
Event	12th International Workshop on Semantic Evaluation, SemEval 2018, co-located with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 - New Orleans, United States Duration: 5 Jun 2018 → 6 Jun 2018

Publication series

Name	NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop

Conference

Conference	12th International Workshop on Semantic Evaluation, SemEval 2018, co-located with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018
Country/Territory	United States
City	New Orleans
Period	5/06/18 → 6/06/18

Bibliographical note

Publisher Copyright:
© 2018 Association for Computational Linguistics

Funding

The authors gratefully acknowledge the economic support in the construction of the datasets from the Maria de Maeztu-UPF Grant provided to Horacio Saggion, Luis Espinosa-Anke and Sergio Oramas; Google Research through the Google Doctoral Fellowship in Natural Language Processing to Jose Camacho-Collados; and Bar-Ilan University through Vered Shwartz. This work is partially supported by the TUNER project (TIN2015-65308-C5-5-R, MINECO/FEDER, UE), Spanish Ministry of Economy and Competitiveness. We would also like to thank the task participants who provided helpful inputs to improve the task through their comments in the Google Group.

Funders	Funder number
Google Group
Google
Bar-Ilan University	TIN2015-65308-C5-5-R
Ministerio de Economía y Competitividad
European Regional Development Fund

Access to Document

10.18653/v1/s18-1115

Cite this

Camacho-Collados, J., Delli Bovi, C., Espinosa-Anke, L., Oramas, S., Pasini, T., Santus, E., Shwartz, V., Navigli, R., & Saggion, H. (2018). SemEval-2018 Task 9: Hypernym Discovery. In M. Apidianaki, M. Apidianaki, S. M. Mohammad, J. May, E. Shutova, S. Bethard, & M. Carpuat (Eds.), NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop (pp. 712-724). (NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/s18-1115

Camacho-Collados, Jose ; Delli Bovi, Claudio ; Espinosa-Anke, Luis et al. / SemEval-2018 Task 9 : Hypernym Discovery. NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop. editor / Marianna Apidianaki ; Marianna Apidianaki ; Saif M. Mohammad ; Jonathan May ; Ekaterina Shutova ; Steven Bethard ; Marine Carpuat. Association for Computational Linguistics (ACL), 2018. pp. 712-724 (NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop).

@inproceedings{2993f9db1e8d442db2117f72e47a49b1,

title = "SemEval-2018 Task 9: Hypernym Discovery",

abstract = "This paper describes the SemEval 2018 Shared Task on Hypernym Discovery. We put forward this task as a complementary benchmark for modeling hypernymy, a problem which has traditionally been cast as a binary classification task, taking a pair of candidate words as input. Instead, our reformulated task is defined as follows: given an input term, retrieve (or discover) its suitable hypernyms from a target corpus. We proposed five different subtasks covering three languages (English, Spanish, and Italian), and two specific domains of knowledge in English (Medical and Music). Participants were allowed to compete in any or all of the subtasks. Overall, a total of 11 teams participated, with a total of 39 different systems submitted through all subtasks. Data, results and further information about the task can be found at https://competitions.codalab.org/competitions/17119.",

author = "Jose Camacho-Collados and \{Delli Bovi\}, Claudio and Luis Espinosa-Anke and Sergio Oramas and Tommaso Pasini and Enrico Santus and Vered Shwartz and Roberto Navigli and Horacio Saggion",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computational Linguistics; 12th International Workshop on Semantic Evaluation, SemEval 2018, co-located with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 ; Conference date: 05-06-2018 Through 06-06-2018",

year = "2018",

doi = "10.18653/v1/s18-1115",

language = "אנגלית",

series = "NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "712--724",

editor = "Marianna Apidianaki and Marianna Apidianaki and Mohammad, \{Saif M.\} and Jonathan May and Ekaterina Shutova and Steven Bethard and Marine Carpuat",

booktitle = "NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop",

address = "ארצות הברית",

}

Camacho-Collados, J, Delli Bovi, C, Espinosa-Anke, L, Oramas, S, Pasini, T, Santus, E, Shwartz, V, Navigli, R & Saggion, H 2018, SemEval-2018 Task 9: Hypernym Discovery. in M Apidianaki, M Apidianaki, SM Mohammad, J May, E Shutova, S Bethard & M Carpuat (eds), NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop. NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop, Association for Computational Linguistics (ACL), pp. 712-724, 12th International Workshop on Semantic Evaluation, SemEval 2018, co-located with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018, New Orleans, United States, 5/06/18. https://doi.org/10.18653/v1/s18-1115

SemEval-2018 Task 9: Hypernym Discovery. / Camacho-Collados, Jose; Delli Bovi, Claudio; Espinosa-Anke, Luis et al.
NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop. ed. / Marianna Apidianaki; Marianna Apidianaki; Saif M. Mohammad; Jonathan May; Ekaterina Shutova; Steven Bethard; Marine Carpuat. Association for Computational Linguistics (ACL), 2018. p. 712-724 (NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - SemEval-2018 Task 9

T2 - 12th International Workshop on Semantic Evaluation, SemEval 2018, co-located with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018

AU - Camacho-Collados, Jose

AU - Delli Bovi, Claudio

AU - Espinosa-Anke, Luis

AU - Oramas, Sergio

AU - Pasini, Tommaso

AU - Santus, Enrico

AU - Shwartz, Vered

AU - Navigli, Roberto

AU - Saggion, Horacio

PY - 2018

Y1 - 2018

N2 - This paper describes the SemEval 2018 Shared Task on Hypernym Discovery. We put forward this task as a complementary benchmark for modeling hypernymy, a problem which has traditionally been cast as a binary classification task, taking a pair of candidate words as input. Instead, our reformulated task is defined as follows: given an input term, retrieve (or discover) its suitable hypernyms from a target corpus. We proposed five different subtasks covering three languages (English, Spanish, and Italian), and two specific domains of knowledge in English (Medical and Music). Participants were allowed to compete in any or all of the subtasks. Overall, a total of 11 teams participated, with a total of 39 different systems submitted through all subtasks. Data, results and further information about the task can be found at https://competitions.codalab.org/competitions/17119.

AB - This paper describes the SemEval 2018 Shared Task on Hypernym Discovery. We put forward this task as a complementary benchmark for modeling hypernymy, a problem which has traditionally been cast as a binary classification task, taking a pair of candidate words as input. Instead, our reformulated task is defined as follows: given an input term, retrieve (or discover) its suitable hypernyms from a target corpus. We proposed five different subtasks covering three languages (English, Spanish, and Italian), and two specific domains of knowledge in English (Medical and Music). Participants were allowed to compete in any or all of the subtasks. Overall, a total of 11 teams participated, with a total of 39 different systems submitted through all subtasks. Data, results and further information about the task can be found at https://competitions.codalab.org/competitions/17119.

UR - https://www.scopus.com/pages/publications/85117572299

U2 - 10.18653/v1/s18-1115

DO - 10.18653/v1/s18-1115

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85117572299

T3 - NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop

SP - 712

EP - 724

BT - NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop

A2 - Apidianaki, Marianna

A2 - Mohammad, Saif M.

A2 - May, Jonathan

A2 - Shutova, Ekaterina

A2 - Bethard, Steven

A2 - Carpuat, Marine

PB - Association for Computational Linguistics (ACL)

Y2 - 5 June 2018 through 6 June 2018

ER -

Camacho-Collados J, Delli Bovi C, Espinosa-Anke L, Oramas S, Pasini T, Santus E et al. SemEval-2018 Task 9: Hypernym Discovery. In Apidianaki M, Apidianaki M, Mohammad SM, May J, Shutova E, Bethard S, Carpuat M, editors, NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop. Association for Computational Linguistics (ACL). 2018. p. 712-724. (NAACL HLT 2018 - International Workshop on Semantic Evaluation, SemEval 2018 - Proceedings of the 12th Workshop). doi: 10.18653/v1/s18-1115

SemEval-2018 Task 9: Hypernym Discovery

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this