The negochat corpus of human-agent negotiation dialogues

Vasily Konovalov, Ron Artstein, Oren Melamud, Ido Dagan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Annotated in-domain corpora are crucial to the successful development of dialogue systems of automated agents, and in particular for developing natural language understanding (NLU) components of such systems. Unfortunately, such important resources are scarce. In this work, we introduce an annotated natural language human-agent dialogue corpus in the negotiation domain. The corpus was collected using Amazon Mechanical Turk following the 'Wizard-Of-Oz' approach, where a 'wizard' human translates the participants' natural language utterances in real time into a semantic language. Once dialogue collection was completed, utterances were annotated with intent labels by two independent annotators, achieving high inter-annotator agreement. Our initial experiments with an SVM classifier show that automatically inferring such labels from the utterances is far from trivial. We make our corpus publicly available to serve as an aid in the development of dialogue systems for negotiation agents, and suggest that analogous corpora can be created following our methodology and using our available source code. To the best of our knowledge this is the first publicly available negotiation dialogue corpus.

Original languageEnglish
Title of host publicationProceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016
EditorsNicoletta Calzolari, Khalid Choukri, Helene Mazo, Asuncion Moreno, Thierry Declerck, Sara Goggi, Marko Grobelnik, Jan Odijk, Stelios Piperidis, Bente Maegaard, Joseph Mariani
PublisherEuropean Language Resources Association (ELRA)
Pages3141-3145
Number of pages5
ISBN (Electronic)9782951740891
StatePublished - 2016
Event10th International Conference on Language Resources and Evaluation, LREC 2016 - Portoroz, Slovenia
Duration: 23 May 201628 May 2016

Publication series

NameProceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

Conference

Conference10th International Conference on Language Resources and Evaluation, LREC 2016
Country/TerritorySlovenia
CityPortoroz
Period23/05/1628/05/16

Bibliographical note

Funding Information:
We thank Sarit Kraus, Avi Rosenfeld, Erel Segal-Halevi, Osnat Drein and Inon Zuckerman for their assistance and contribution. This work was partly supported by ERC Grant #267523.

Funding

We thank Sarit Kraus, Avi Rosenfeld, Erel Segal-Halevi, Osnat Drein and Inon Zuckerman for their assistance and contribution. This work was partly supported by ERC Grant #267523.

FundersFunder number
European Commission267523

    Keywords

    • Crowdsourcing
    • Dialogue systems
    • Negotiation corpora

    Fingerprint

    Dive into the research topics of 'The negochat corpus of human-agent negotiation dialogues'. Together they form a unique fingerprint.

    Cite this