Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss

Barbara Plank, Anders Søgaard, Yoav Goldberg

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

298 Scopus citations

Abstract

Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise. We address these issues and evaluate bi-LSTMs with word, character, and unicode byte embeddings for POS tagging. We compare bi-LSTMs to traditional POS taggers across languages and data sizes. We also present a novel bi-LSTM model, which combines the POS tagging loss function with an auxiliary loss function that accounts for rare words. The model obtains state-of-the-art performance across 22 languages, and works especially well for morphologically complex languages. Our analysis suggests that bi-LSTMs are less sensitive to training data size and label corruptions (at small noise levels) than previously assumed.

Original languageEnglish
Title of host publication54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers
PublisherAssociation for Computational Linguistics (ACL)
Pages412-418
Number of pages7
ISBN (Electronic)9781510827592
DOIs
StatePublished - 2016
Event54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Berlin, Germany
Duration: 7 Aug 201612 Aug 2016

Publication series

Name54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers

Conference

Conference54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
Country/TerritoryGermany
CityBerlin
Period7/08/1612/08/16

Bibliographical note

Publisher Copyright:
© 2016 Association for Computational Linguistics.

Fingerprint

Dive into the research topics of 'Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss'. Together they form a unique fingerprint.

Cite this