SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection

Jordan Kodner; Salam Khalifa; Khuyagbaatar Batsuren; Hossep Dolatian; Ryan Cotterell; Faruk Akkuş; Antonios Anastasopoulos; Taras Andrushko; Aryaman Arora; Nona Atanelov Gábor Bella; Elena Budianskaya; Yustinus Ghanggo Ate; Omer Goldman; David Guriel; Simon Guriel; Silvia Guriel-Agiashvili; Witold Kieraś; Andrew Krizhanovsky; Natalia Krizhanovsky; Igor Marchenko; Magdalena Markowska; Polina Mashkovtseva; Maria Nepomniashchaya; Daria Rodionova; Karina Sheifer; Alexandra Serova; Anastasia Yemelina; Jeremiah Young; Ekaterina Vylomova

SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection

Jordan Kodner, Salam Khalifa, Khuyagbaatar Batsuren, Hossep Dolatian, Ryan Cotterell, Faruk Akkuş, Antonios Anastasopoulos, Taras Andrushko, Aryaman Arora, Nona Atanelov Gábor Bella, Elena Budianskaya, Yustinus Ghanggo Ate, Omer Goldman, David Guriel, Simon Guriel, Silvia Guriel-Agiashvili, Witold Kieraś, Andrew Krizhanovsky, Natalia Krizhanovsky, Igor MarchenkoMagdalena Markowska, Polina Mashkovtseva, Maria Nepomniashchaya, Daria Rodionova, Karina Sheifer, Alexandra Serova, Anastasia Yemelina, Jeremiah Young, Ekaterina Vylomova

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

26 Scopus citations

Abstract

The 2022 SIGMORPHON-UniMorph shared task on large scale morphological inflection generation included a wide range of typologically diverse languages: 33 languages from 11 top-level language families: Arabic (Modern Standard), Assamese, Braj, Chukchi, Eastern Armenian, Evenki, Georgian, Gothic, Gujarati, Hebrew, Hungarian, Itelmen, Karelian, Kazakh, Ket, Khalkha Mongolian, Kholosi, Korean, Lamahalot, Low German, Ludic, Magahi, Middle Low German, Old English, Old High German, Old Norse, Polish, Pomak, Slovak, Turkish, Upper Sorbian, Veps, and Xibe. We emphasize generalization along different dimensions this year by evaluating test items with unseen lemmas and unseen features separately under small and large training conditions. Across the six submitted systems and two baselines, the prediction of inflections with unseen features proved challenging, with average performance decreased substantially from last year. This was true even for languages for which the forms were in principle predictable, which suggests that further work is needed in designing systems that capture the various types of generalization required for the world's languages.

Original language	English
Title of host publication	SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop
Editors	Garrett Nicolai, Eleanor Chodroff
Publisher	Association for Computational Linguistics (ACL)
Pages	176-203
Number of pages	28
ISBN (Electronic)	9781955917827
State	Published - 2022
Event	19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, SIGMORPHON 2022 - Seattle, United States Duration: 14 Jul 2022 → …

Publication series

Name	SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop

Conference

Conference	19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, SIGMORPHON 2022
Country/Territory	United States
City	Seattle
Period	14/07/22 → …

Bibliographical note

Publisher Copyright:
© 2022 Association for Computational Linguistics.

Funding

We would like to thank Garrett Nicolai, Maria Ryskina, Ben Ambridge, Jeff Heinz, and all those who provided valuable advice and logistical support in the early stages of this project, Judit Ács, Duygu Ataman, Zigniew Bronk, Eleanor Chodroff, Sofya Ganieva, Włodzimierz Gruszczyński, Nizar Habash, Jan Hajicˇ, Jan Hric, Ritvan Karahodja, Christo Kirov, Elena Klyachko, Ritesh Kumar, Va-hagn Petrosyan, Matvey Plugaryov, Mohit Raj, Maria Ryskina, Elizabeth Salesky, Zygmunt Saloni, Danuta Skowrońska, Marcin Woliński, and Robert Wołosz, who prepared and authored lexicons used in this project, as well as Jeff Heinz, Sarah Payne, and Charles Yang, who provided feedback on this overview paper. The neural baseline system was trained on the SeaWulf HPC cluster maintained by RCC, and IACS at Stony Brook University and made possible by NSF grant #1531492.

Funders	Funder number
National Science Foundation	1531492

Cite this

Kodner, J., Khalifa, S., Batsuren, K., Dolatian, H., Cotterell, R., Akkuş, F., Anastasopoulos, A., Andrushko, T., Arora, A., Bella, N. A. G., Budianskaya, E., Ghanggo Ate, Y., Goldman, O., Guriel, D., Guriel, S., Guriel-Agiashvili, S., Kieraś, W., Krizhanovsky, A., Krizhanovsky, N., ... Vylomova, E. (2022). SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection. In G. Nicolai, & E. Chodroff (Eds.), SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop (pp. 176-203). (SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

Kodner, Jordan ; Khalifa, Salam ; Batsuren, Khuyagbaatar et al. / SIGMORPHON-UniMorph 2022 Shared Task 0 : Generalization and Typologically Diverse Morphological Inflection. SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop. editor / Garrett Nicolai ; Eleanor Chodroff. Association for Computational Linguistics (ACL), 2022. pp. 176-203 (SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop).

@inproceedings{2b2059c915f743e79f95f1beaef4b4eb,

title = "SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection",

abstract = "The 2022 SIGMORPHON-UniMorph shared task on large scale morphological inflection generation included a wide range of typologically diverse languages: 33 languages from 11 top-level language families: Arabic (Modern Standard), Assamese, Braj, Chukchi, Eastern Armenian, Evenki, Georgian, Gothic, Gujarati, Hebrew, Hungarian, Itelmen, Karelian, Kazakh, Ket, Khalkha Mongolian, Kholosi, Korean, Lamahalot, Low German, Ludic, Magahi, Middle Low German, Old English, Old High German, Old Norse, Polish, Pomak, Slovak, Turkish, Upper Sorbian, Veps, and Xibe. We emphasize generalization along different dimensions this year by evaluating test items with unseen lemmas and unseen features separately under small and large training conditions. Across the six submitted systems and two baselines, the prediction of inflections with unseen features proved challenging, with average performance decreased substantially from last year. This was true even for languages for which the forms were in principle predictable, which suggests that further work is needed in designing systems that capture the various types of generalization required for the world's languages.",

author = "Jordan Kodner and Salam Khalifa and Khuyagbaatar Batsuren and Hossep Dolatian and Ryan Cotterell and Faruk Akku{\c s} and Antonios Anastasopoulos and Taras Andrushko and Aryaman Arora and Bella, {Nona Atanelov G{\'a}bor} and Elena Budianskaya and {Ghanggo Ate}, Yustinus and Omer Goldman and David Guriel and Simon Guriel and Silvia Guriel-Agiashvili and Witold Kiera{\'s} and Andrew Krizhanovsky and Natalia Krizhanovsky and Igor Marchenko and Magdalena Markowska and Polina Mashkovtseva and Maria Nepomniashchaya and Daria Rodionova and Karina Sheifer and Alexandra Serova and Anastasia Yemelina and Jeremiah Young and Ekaterina Vylomova",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, SIGMORPHON 2022 ; Conference date: 14-07-2022",

year = "2022",

language = "אנגלית",

series = "SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "176--203",

editor = "Garrett Nicolai and Eleanor Chodroff",

booktitle = "SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop",

address = "ארצות הברית",

}

Kodner, J, Khalifa, S, Batsuren, K, Dolatian, H, Cotterell, R, Akkuş, F, Anastasopoulos, A, Andrushko, T, Arora, A, Bella, NAG, Budianskaya, E, Ghanggo Ate, Y, Goldman, O, Guriel, D, Guriel, S, Guriel-Agiashvili, S, Kieraś, W, Krizhanovsky, A, Krizhanovsky, N, Marchenko, I, Markowska, M, Mashkovtseva, P, Nepomniashchaya, M, Rodionova, D, Sheifer, K, Serova, A, Yemelina, A, Young, J & Vylomova, E 2022, SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection. in G Nicolai & E Chodroff (eds), SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop. SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 176-203, 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, SIGMORPHON 2022, Seattle, United States, 14/07/22.

SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection. / Kodner, Jordan; Khalifa, Salam; Batsuren, Khuyagbaatar et al.
SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop. ed. / Garrett Nicolai; Eleanor Chodroff. Association for Computational Linguistics (ACL), 2022. p. 176-203 (SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - SIGMORPHON-UniMorph 2022 Shared Task 0

T2 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, SIGMORPHON 2022

AU - Kodner, Jordan

AU - Khalifa, Salam

AU - Batsuren, Khuyagbaatar

AU - Dolatian, Hossep

AU - Cotterell, Ryan

AU - Akkuş, Faruk

AU - Anastasopoulos, Antonios

AU - Andrushko, Taras

AU - Arora, Aryaman

AU - Bella, Nona Atanelov Gábor

AU - Budianskaya, Elena

AU - Ghanggo Ate, Yustinus

AU - Goldman, Omer

AU - Guriel, David

AU - Guriel, Simon

AU - Guriel-Agiashvili, Silvia

AU - Kieraś, Witold

AU - Krizhanovsky, Andrew

AU - Krizhanovsky, Natalia

AU - Marchenko, Igor

AU - Markowska, Magdalena

AU - Mashkovtseva, Polina

AU - Nepomniashchaya, Maria

AU - Rodionova, Daria

AU - Sheifer, Karina

AU - Serova, Alexandra

AU - Yemelina, Anastasia

AU - Young, Jeremiah

AU - Vylomova, Ekaterina

PY - 2022

Y1 - 2022

N2 - The 2022 SIGMORPHON-UniMorph shared task on large scale morphological inflection generation included a wide range of typologically diverse languages: 33 languages from 11 top-level language families: Arabic (Modern Standard), Assamese, Braj, Chukchi, Eastern Armenian, Evenki, Georgian, Gothic, Gujarati, Hebrew, Hungarian, Itelmen, Karelian, Kazakh, Ket, Khalkha Mongolian, Kholosi, Korean, Lamahalot, Low German, Ludic, Magahi, Middle Low German, Old English, Old High German, Old Norse, Polish, Pomak, Slovak, Turkish, Upper Sorbian, Veps, and Xibe. We emphasize generalization along different dimensions this year by evaluating test items with unseen lemmas and unseen features separately under small and large training conditions. Across the six submitted systems and two baselines, the prediction of inflections with unseen features proved challenging, with average performance decreased substantially from last year. This was true even for languages for which the forms were in principle predictable, which suggests that further work is needed in designing systems that capture the various types of generalization required for the world's languages.

AB - The 2022 SIGMORPHON-UniMorph shared task on large scale morphological inflection generation included a wide range of typologically diverse languages: 33 languages from 11 top-level language families: Arabic (Modern Standard), Assamese, Braj, Chukchi, Eastern Armenian, Evenki, Georgian, Gothic, Gujarati, Hebrew, Hungarian, Itelmen, Karelian, Kazakh, Ket, Khalkha Mongolian, Kholosi, Korean, Lamahalot, Low German, Ludic, Magahi, Middle Low German, Old English, Old High German, Old Norse, Polish, Pomak, Slovak, Turkish, Upper Sorbian, Veps, and Xibe. We emphasize generalization along different dimensions this year by evaluating test items with unseen lemmas and unseen features separately under small and large training conditions. Across the six submitted systems and two baselines, the prediction of inflections with unseen features proved challenging, with average performance decreased substantially from last year. This was true even for languages for which the forms were in principle predictable, which suggests that further work is needed in designing systems that capture the various types of generalization required for the world's languages.

UR - http://www.scopus.com/inward/record.url?scp=85139098980&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85139098980

T3 - SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop

SP - 176

EP - 203

BT - SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop

A2 - Nicolai, Garrett

A2 - Chodroff, Eleanor

PB - Association for Computational Linguistics (ACL)

Y2 - 14 July 2022

ER -

Kodner J, Khalifa S, Batsuren K, Dolatian H, Cotterell R, Akkuş F et al. SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection. In Nicolai G, Chodroff E, editors, SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2022. p. 176-203. (SIGMORPHON 2022 - 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, Proceedings of the Workshop).

SIGMORPHON-UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection

Abstract

Publication series

Conference

Bibliographical note

Funding

Other files and links

Fingerprint

Cite this