Training with exploration improves a greedy stack LSTM parser

Miguel Ballesteros; Yoav Goldberg; Chris Dyer; Noah A. Smith

doi:10.18653/v1/d16-1211

Training with exploration improves a greedy stack LSTM parser

Miguel Ballesteros, Yoav Goldberg, Chris Dyer, Noah A. Smith

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

40 Scopus citations

Abstract

We adapt the greedy stack LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles (Goldberg and Nivre, 2013) instead of assuming an error-free action history. This form of training, which accounts for model predictions at training time, improves parsing accuracies. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural network dependency parser.

Original language	English
Title of host publication	EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings
Publisher	Association for Computational Linguistics (ACL)
Pages	2005-2010
Number of pages	6
ISBN (Electronic)	9781945626258
DOIs	https://doi.org/10.18653/v1/d16-1211
State	Published - 2016
Event	2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016 - Austin, United States Duration: 1 Nov 2016 → 5 Nov 2016

Publication series

Name	EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings

Conference

Conference	2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016
Country/Territory	United States
City	Austin
Period	1/11/16 → 5/11/16

Bibliographical note

Publisher Copyright:
© 2016 Association for Computational Linguistics

Funding

This work was sponsored in part by the U. S. Army Research Laboratory and the U. S. Army Research Office under contract/grant number W911NF-10-1-0533, and in part by NSF CAREER grant IIS-1054319. Miguel Ballesteros was supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA). Yoav Goldberg is supported by the Intel Collaborative Research Institute for Computational Intelligence (ICRI-CI), a Google Research Award and the Israeli Science Foundation (grant number 1555/15).

Funders	Funder number
Israeli Science Foundation	1555/15
NSF CAREER
U. S. Army Research Laboratory
U. S. Army Research Office	W911NF-10-1-0533
National Science Foundation	IIS-1054319
Google
U.S. Army Aeromedical Research Laboratory
European Commission	FP7-ICT-610411, H2020-RIA-645012
Intel Collaboration Research Institute for Computational Intelligence

Access to Document

10.18653/v1/d16-1211

Cite this

Ballesteros, M., Goldberg, Y., Dyer, C., & Smith, N. A. (2016). Training with exploration improves a greedy stack LSTM parser. In EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 2005-2010). (EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/d16-1211

Ballesteros, Miguel ; Goldberg, Yoav ; Dyer, Chris et al. / Training with exploration improves a greedy stack LSTM parser. EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings. Association for Computational Linguistics (ACL), 2016. pp. 2005-2010 (EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings).

@inproceedings{b19d19f28e0c4b62805091102b312dd0,

title = "Training with exploration improves a greedy stack LSTM parser",

abstract = "We adapt the greedy stack LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles (Goldberg and Nivre, 2013) instead of assuming an error-free action history. This form of training, which accounts for model predictions at training time, improves parsing accuracies. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural network dependency parser.",

author = "Miguel Ballesteros and Yoav Goldberg and Chris Dyer and Smith, {Noah A.}",

note = "Publisher Copyright: {\textcopyright} 2016 Association for Computational Linguistics; 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016 ; Conference date: 01-11-2016 Through 05-11-2016",

year = "2016",

doi = "10.18653/v1/d16-1211",

language = "אנגלית",

series = "EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings",

publisher = "Association for Computational Linguistics (ACL)",

pages = "2005--2010",

booktitle = "EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings",

address = "ארצות הברית",

}

Ballesteros, M, Goldberg, Y, Dyer, C & Smith, NA 2016, Training with exploration improves a greedy stack LSTM parser. in EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings. EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings, Association for Computational Linguistics (ACL), pp. 2005-2010, 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, United States, 1/11/16. https://doi.org/10.18653/v1/d16-1211

Training with exploration improves a greedy stack LSTM parser. / Ballesteros, Miguel; Goldberg, Yoav; Dyer, Chris et al.
EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings. Association for Computational Linguistics (ACL), 2016. p. 2005-2010 (EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Training with exploration improves a greedy stack LSTM parser

AU - Ballesteros, Miguel

AU - Goldberg, Yoav

AU - Dyer, Chris

AU - Smith, Noah A.

PY - 2016

Y1 - 2016

N2 - We adapt the greedy stack LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles (Goldberg and Nivre, 2013) instead of assuming an error-free action history. This form of training, which accounts for model predictions at training time, improves parsing accuracies. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural network dependency parser.

AB - We adapt the greedy stack LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles (Goldberg and Nivre, 2013) instead of assuming an error-free action history. This form of training, which accounts for model predictions at training time, improves parsing accuracies. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural network dependency parser.

UR - http://www.scopus.com/inward/record.url?scp=85072834015&partnerID=8YFLogxK

U2 - 10.18653/v1/d16-1211

DO - 10.18653/v1/d16-1211

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85072834015

T3 - EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings

SP - 2005

EP - 2010

BT - EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings

PB - Association for Computational Linguistics (ACL)

T2 - 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016

Y2 - 1 November 2016 through 5 November 2016

ER -

Ballesteros M, Goldberg Y, Dyer C, Smith NA. Training with exploration improves a greedy stack LSTM parser. In EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings. Association for Computational Linguistics (ACL). 2016. p. 2005-2010. (EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings). doi: 10.18653/v1/d16-1211

Training with exploration improves a greedy stack LSTM parser

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this